TalkMotion (Prototype)

TalkMotion is a mobile application prototype that recognizes particular programmed movements of the device to elicit particular text-to-speech phrases. For people unable to speak and with limited mobility to use keyboards or tablets, this enables faster and efficient social interaction which can greatly improve their quality of life. The application is not completed yet, it’s in the underdevelopment stage. Try this application prototype to explore the possibilities of communicative hand gestures using mobile phones!

Features on the TalkMotion app include:

* In the speech module, there are predefined 11 gestures users can perform. Moreover, users can change the audible responses associated with those gestures whatever they want.
* Users can input their own defined gestures in the application but cannot use them for speech production. But the gesture’s accelerometer data is being stored in our database. In the future, users can train the integrated deep learning model with the application to learn their own defined gestures for audible responses by pushing a button.

The application is an early, readily shared demonstration of gesture-to-speech. Further development is intended to enable additional gestures and improve recognition accuracy and speed, with the goal of enabling more efficient communication for individuals unable to speak.