Project: Haptic Sense

The motivation

Imagine if you could feel sound? Seems like it’d be a neat novelty. But now imagine if you were deaf, now it could be an essential lifeline. And that’s what haptic sense is trying to solve, turning sound into something you can feel.

the prototype

And so I built it, using off the shelf arduino parts and some python code. The details are on the github repo. But basically all it took was the integrated microphone on my laptop, an arduino, the DRV2605L controller, the vibrating motor, and some relatively basic code.

There was a bit of fidgeting around the main python code however. Specifically switching from struct.unpack to numpy.frombuffer to convert the data to integers making the code easier to understand and faster. Also specifically how to process the volume whether it made sense to smooth it out with an average and how to scale the volume compared to the vibration motor output. In the end, I choose to not smooth it out to make it more responsive and to scale back the volume relative vibration output so that only truly loud noises reached maximum output.

the End result?

You could feel a person’s voice in a conversation. You could feel when a plate clattered in the kitchen. It does kind of work. I could only imagine how much it could change a deaf person’s experience; if they went to the movie theatre, it’d add a whole new dimension.

It’s so good in fact that it turns out Apple holds the patent for converting audio into vibration. I’m betting they were thinking about incorporating it into their Apple watches, although they haven’t released it yet so I’m guessing they’ve run into problems. Now that is understandable, our ears gives us sound in 2D through the frequency and amplitude along with the direction the sound comes from. The haptic sense is only 1D volume and no direction (at least when using only one mic and motor). Also leaving it on constantly in a noisy environment would rapidly drain the batteries. It’s also less useful in a noisy environment because there’d be so many competing sounds.

However, I still think giving a person something is better than nothing. Maybe the user-testing turned out different, but this is the lowest cost option, the whole setup can be vastly miniaturized. According to the UN, there are 70 million deaf people in the world with 80% of them living in developing countries. And sure there are actual cures for deafness now, but those treatments will be out of reach for most for a long time. Maybe the solution wasn’t good enough for Apple watch users, but for someone paying a handful of dollars for a battery-laden wristwatch that told them the volume of their environment it could be a lifesaver.