Last Summer at the Imagine Cup World Finals, three computer-science students earned themselves a boot camp at Microsoft for the compassion ingenuity of their app, BoneyCare. BoneyCare is designed to help people with speech disorders improve their confidence and speaking quality. The app records, transcribes and analyzes a user’s speech patterns, which speech pathologists can then use to improve and accelerate their diagnoses and treatment plans.
Rongyu (left) goes to Beijing Normal University, Ariel (middle) attends UC Berkeley and Oscar (right) is at Northeastern University
Finding the inspiration for an impactful app
Team BoneyCare, consisting of Ariel Zhang, Oscar Li and Rongyu Wang, met in high school about six years ago. Oscar and Rongyu shared a class, and they met Ariel through their school’s model United Nations. The three students are now college juniors majoring in computer science.
Going into the Imagine Cup competition, Ariel, Oscar and Rongyu knew that they wanted to design something that would have a positive impact on people’s lives while using existing technology in an innovative way. For inspiration, they looked to the Imagine Cup World Finals winners who came before them.
Eyenaemia, whose anemia-diagnosing app secured a first-place win at the 2014 Imagine Cup World Finals, helped Team BoneyCare decide on their project’s concept.
Oscar says, “Eyenaemia used a camera for their app, so we thought, ‘What about a microphone?’”
The obvious choice was to do something around speech, which made Oscar remember his friend from primary school, Boney, who was bullied for having a stutter. Oscar shared the memory with Ariel and Rongyu, and their concept for an impactful app was born. “We designed this app for Boney,” Rongyu says, “to honor him, and to care for him and others who struggle with speech disorders.”
Oscar says, “There’s no universal therapy for a stutter, so we want to find out from this data what kind of therapy is better for one person versus others. We also want to find out if a patient’s current therapy is working or not since our software can provide a rating for speech fluency.”
Currently, patients have to go to their speech pathologists’ offices where they are recorded reading a passage or talking about a topic. A speech pathologist must wait for a patient’s recording to be transcribed, and then must physically search for and note disordered speech patterns. It can take months or years to find the right kind of treatment depending on the type and origin of a person’s disordered speech.
“Our cloud-based algorithm will find these patterns for them,” says Oscar.
Overcoming distance to develop their app
Sixteen hours and more than 10,700 kilometers (nearly 6,700 miles) separate Ariel from Oscar and Rongyu—Ariel goes to school in the United States, and Oscar and Rongyu attend schools in northern China. While developing BoneyCare, the team members also attended classes full time. Their physical distance and busy schedules made it challenging to find a time that would work for all of them to meet, so they would only hop on Skype to chat about critical tasks or issues.
But Microsoft Visual Studio Team Services, hosted on Microsoft Azure, made development of the app less complicated. Because the app is hosted in the cloud, the team can log on to Visual Studio and work on code whenever and wherever. Generally, Oscar does most of the coding. Ariel works on the machine learning aspect of the app, and Rongyu collects and manages the data needed to inform the project’s design and development.
With the BoneyCare app, users are given a topic to speak about for a certain amount of time. The app records and transcribes a user’s speech sample, and Microsoft Cognitive Services translates the speech into text. The team’s custom machine-learning algorithm then analyzes the transcript and correlates the types and patterns of the patient’s disordered speech with medical data collected by Rongyu.
“We have to be scientific,” Oscar stresses. The students know that, if developed the right way with the right datasets, BoneyCare could dramatically decrease the time it takes for speech pathologists to accurately treat speech disorders and identify the best treatments—which can help patients’ speech confidence and quality.
Imagine Cup success strategies
Throughout the Imagine Cup competition, the members of Team BoneyCare focused on improving their app rather than winning – it was about the end-goal of helping people with technology rather than placing first in a competition. This motivation, they think, was key to winning the boot camp and one they plan to maintain post-competition.
The desire to consistently improve helped the three students narrow down their interests in the broad field of computer science. Oscar has gotten into speech-processing and language recognition through his work on the BoneyCare app. Ariel finds herself into all things machine learning and has been working to improve the app’s algorithm since the Imagine Cup World Finals. Rongyu has invested her interests in data mining and analytics and is working to determine the best way to process data for the development of BoneyCare. She is also fascinated with virtual reality (VR) technology and wants to find a way to use VR in BoneyCare, which could further its reach as a speech training and therapy tool.
“What is really important to the project right now,” Ariel says, “is the data.” The team needs a large amount of data from a trustworthy, known source to continue development of BoneyCare. The more data the team collects, the more accurate BoneyCare will be. “Our hope is to find a clinic or organization to work with so we know exactly who is behind the voice-pattern data that we collect.”
The 2017 Imagine Cup season is underway with the finals just over two months out, and for those of you participating this year, Team BoneyCare reminds you to:
- Focus on improving your project and doing the right thing
- Look to the past winners for inspiration and motivation
- Talk to the Microsoft Student Partners (MSP) on your campus for information about the technology available to you
Microsoft Imagine, anywhere, anytime:
- Follow us on Twitter, Facebook, and YouTube.
- Subscribe to our blog to meet students just like you who are changing the world with their exciting new tech. Plus, stay on top of all the new products and offerings for students.
- Head on over to Microsoft Imagine where, if you haven’t already, you can create a Microsoft Imagine student profile, giving you access to developer tools, online training, developer competitions and the bi-monthly Microsoft Imagine newsletter.