SpokeIt is A Co-Created Speech Therapy Game aimed to improve the at-home speech therapy experience for children born with Orofacial Cleft. SpokeIt is a highly polished speech therapy game for children born with orofacial cleft capable of diagnosing speech errors using novel machine learning approaches. It was co-created with children, speech therapists, families, surgeons, developmental psychologists, game designers, and artists. Through a partnership with Smile Train, SpokeIt will be delivered around the world in many languages. Screenshots of SpokeIt in action are shown below:
Boat Adventure Level Design
Space Trash Level Design
Boat Adventure Speech Interaction
Antagonist Eliza
Eliza Level Design
Level Selector
Musical Migs Speech Interaction
Cinematic
Boat Adventure Win Scene
Space Trash Speech Interaction
Check out the video about our partnership with Smile Train below!
Medium Fidelity Prototypes created in real-time with our players!
We conducted a day-long participatory workshop starting with low-fidelity paper prototypes and ending with real-time iterations of the medium-fidelity Adobe XD prototype embedded below. Participants with speech impairments had the opportunity to ideate the speech mechanics, give feedback on the level design, and discuss how each minigame is won. At the end of the workshop, participants tested their digital designs and presented them to a group of caretakers. These designs directly influenced the development of numerous SpokeIt minigames, including Musical Migs, Eliza, Boat Adventure, and Space Trash.
Check out how SpokeIt was co-created!
Keynotes about SpokeIt
Tangible design probes used in participatory sessions
Random number and letter generators, developmentally appropriate words, and game characters
Handmade felted SpokeIt characters embedded with magnets and antenna pipe cleaners
Completed felted Migs
Child with a cleft playing with design probes and designing new SpokeIt games
My Role
I am the founder of SpokeIt. I lead the design, research, and development of the app. I implemented the underlying framework, speech mechanics, and engine the minigames within the app are built on. The minigames were implemented by 50 high school interns, undergraduate research assistants, and Master's students under my supervision. I created and rigged the characters, but much of the level art was created by a hired artist. I developed the research materials and tangible probes.
