More Personal Computing: Deep dive into the FamilyNotes sample app

  • Article

Screenshot from the FamilyNotes sample app, which shows how to use unique features of the Windows 10 platform, like face recognition, speech, ink, and voice commands. Get the code at https://github.com/Microsoft/Windows-appsample-familynotes. Graphic: WDG Content Publishing

Before I moved to DX, I worked on a great team of content developers, and we created the FamilyNotes sample app (FamilyNotes sample at GitHub) to show how to use unique features of the Windows 10 platform, like face recognition, speech, ink, and voice commands.

  • Create and present “sticky notes” for everyone in the family.
  • Add new notes by typing, inking, or dictating them.
  • Use Cortana and voice commands to activate the app and start a new note to the entire family or a specific family member.
  • Use face detection and face recognition to automatically display the notes applicable to a specific user.
  • App is written in C# and XAML, following a Model-View-Controller methodology.
  • Source code is generously documented.

Now, the content team is posting a series of deep dives on the design and code that went into FamilyNotes, and the first post is up: FamilyNotes: Introducing a Windows UWP sample using ink, speech, and face recognition.

I was responsible for implementing the speech recognition and speech synthesis features. John Kennedy, former teammate and all-around good guy, writes: 

My favorite bug happened when we were testing the dictation mode. Every time we started dictating a note, it was preceded with the string “dictate your note.” We thought we had left some debugging messages in our code, but we couldn’t find it in the source code. And it only happened on some computers, not all. It took a little while to realize that the computer was hearing itself say, “Please start dictating your note,” recognizing its own speech as a human voice and adding it to the note. The testers who had their volume turned down didn’t see the string appear because the computer couldn’t hear itself!

That was my bug. >< Fixing it required some synchronization between the speech recognizer and the speech synthesizer. I’ll show the code in a future post.

Meanwhile, get more details about the sample app here: FamilyNotes: Introducing a Windows UWP sample using ink, speech, and face recognition.

Technorati Tags: UWP,natural interface,Windows,Windows programming,C#,XAML