FingerSound is an always-available gesture based input technology. We demonstrate its ability to recognize 14 distinct natural gestures, namely the digits 0-9 and cardinal directional swipes. These gestures are performed on a thumb mounted ring comprising a contact microphone and accelerometer and gyroscope sensors. They are unobtrusive and can be performed without the need to look at the device (eyes-free). We use Dynamic Time Warping for gesture classification. The contact microphone helps the system to know the gesture is being performed by analysing the sound generated during touching of the finger

 

As a bet, I was able to build this system within a very short time with help from two other teammates. This project helped me immensely to learn about the design aspects of HCI along with other technical knowledge. As a lesson from the project, I got to know how learning curve affects the usability of the system. The current evaluation suggests that the system works with more than 90% accuracy with a  short initial training time. This project will be appear in Ubicomp 17 conference.

Tags:

Wearable Tech, Acoustics, Gesture Recognition, HCI, Arduino