Microsoft’s MAVIS

I was in Redmond recently and got to hear from some of the wizards at Microsoft Research. They showed us some interesting technology that I can’t show to you. They also showed us audio indexing and I was sure they’d say “no Steve, you can’t blog about it” but I was pleasantly surprised to find that I could. So here it is.

Basically, audio indexing take an audio (or video) and indexes it – i.e. it works out what a speaker is saying and transcribes it to text. Then they search across that set of words and index those with and note where they appear. The outcome is you can search for the occurrence of a word in an audio or video stream and be taken directly to the point the word is spoken. MSR explain it better of course

“index the spoken content of recorded conversations, whether they are from meetings, conference calls, voice mails, presentations, online lectures, or even Internet video”.

Washington State Digital Archives are trying this out. Select audio recordings from their site and then search for a word like salmon (a big deal in the Pacific NW). Now be sure to click on the emboldened salmon in the results. You’re taken to the exact point in the audio stream where the word occurs.

The technology is known as MAVIS and there is plenty of information on it from MSR’s site. I was blown away by it and immediately thought of some places I’d love to see it applied. Top of the list is the TED talks which it’d be just great to search through on this way. You can easily see how it could be applied to all kinds of content though – everything from Hollywood movies (find your favourite line in a movie) to recordings of great speeches, or even radio programs.

What do think? Do you have a killer set of content you’d want to see this applied to?