(SACRAMENTO)

What if a video game could do more than just entertain children, but could provide daily speech therapy? That is the life-changing proposition of SpokeIt, a kid-friendly game for mobile devices designed to improve speech for children with cleft palates.

Speech powers up musical instruments in one SpokeIt minigame
Speech powers up musical instruments in one SpokeIt minigame

For more than seven years, Travis Tollefson, director of facial plastic and reconstructive surgery at UC Davis Cleft and Craniofacial Clinic, and UC Davis senior speech-language pathologist Christina Roth have partnered with UC Santa Cruz researchers to provide expert feedback and testing of this UC Santa Cruz-developed game. 

“Our pediatric patients have been able to take home iPads and play this at home,” said Tollefson, who asked more than 10 children between the ages of 2 to 7 to test drive the game over the years. “The program asks the child to produce a series of intelligible speech commands in order to progress through the game’s story.”

The player speaks “magic words” that have been dynamically selected for their specific speech therapy needs. Those words unlock actions on the screen. For example, children use their speech magic to imbue a sail with magic wind to propel a boat.

By playing minigames that test their speech and production of sounds, children unlock the next day’s game. While many children do not have access to speech therapy each day, SpokeIt provides a self-guided tool that incentivizes children to improve their speech every day.   

Approximately 7,500 babies are born in the United States with orofacial clefts each year. With a cleft palate, a child is unable to stop airflow through the nose and will often hold back the air at the larynx. This makes it difficult to create speech others can understand. Once cleft palates are surgically corrected, children still need help to produce normalized speech, which is where SpokeIt comes in.

“We are excited to see this technology help more children who don’t have access to speech therapy,” Tollefson said.

Tollefson and Roth were co-principal investigators, along with Su-hua Wang, professor of psychology at UC Santa Cruz, and Sri Kurniawan, primary investigator and chair of the UC Santa Cruz Computational Media Department, on a National Science Foundation grant in 2016.

Jared Duval, a UC Santa Cruz graduate student in computational media, is currently leading this project. He has enlisted the help of Smile Train, a nonprofit organization that empowers local cleft teams to provide comprehensive cleft care, to complete the game’s design. Smile Train supports care in more than 70 countries around the globe, which will help Duval achieve his goal of bringing this important resource to families affected by cleft lip and palate worldwide.

The next steps in their process are:

  • Finish making the game work in Android
  • Finish a web-based dashboard for parents and medical professionals to remotely control the game, access reports, and verify the machine learning
  • Release the app to start collecting longitudinal data
  • Translate the app into new languages for Smile Train to use

UC Davis cleft and craniofacial patients have provided feedback on the game’s development. Parents have also had the chance to rate the game’s effectiveness and parental satisfaction. Seventy percent of parents agreed that the game was engaging. Eighty percent agreed that the game was enjoyable to replay repeatedly. Eighty percent also agreed that the player perceived a sense of control in the game story.

“It’s been rewarding to have watched it develop from an initial concept to a game that our patients can engage with and have fun with,” Tollefson said. “We’re excited to see what the future will hold.