Raspberry Pi smart glasses translate in real time

Will Powell read about Google’s Project Glass, and decided he’d have a go at a DIY attempt to try to achieve something similar. And what he’s come up with is downright remarkable. Here’s his automatic translation system, which uses a couple of Raspberry Pis, a 3d headset, some microphones, a TV and an iPhone to display real-time subtitles in your glasses as you have a conversation.

28 comments

Avatar

Fantastic! That would be of great use to deaf people too for real-time language to subtitle provision.

Avatar

Deaf people usually do just fine reading lips. At least, the three deaf people in my family are. ;-)

Avatar

I am going deaf slowly, at the age of 51. This could be amazing for me in crowded spaces where all the audio turns to mush!

Avatar

Brilliant stuff. The word fantastic is rarely used appropriately but in this case it certainly seems ‘out of this world’!

Avatar

Finally, It actually feels like I’m living in the future. This is the kind of thing I was promised when I was a kid. Next stop hover-cars!

Avatar

I was thinking about something like this the other day, but in my mind I could see someone looking at a menu in a foreign language and it’d change the words on the menu to English.

Avatar

It exits as a iPhone app, so all we need is someone to incorporate it into some glasses and were there!
http://www.youtube.com/watch?v=h2OfQdYrHRs

Avatar

If it is possible for an iPhone to output video than it could be paired with an iTheatre type Glasses… Make sure you get the ones with better resolution.(800×600 or more) Some may come with an iPod/iPhone adapter.

Yes, I’ve got a pair of my own but just the 320×240 rez… :/

Avatar

And even more cool would be to read a news paper and the pictures of the news paper would “come-alive”, like shown in this TED talk

http://www.ted.com/talks/matt_mills_image_recognition_that_triggers_augmented_reality.html

This would be extremely cool, until the advertising companies would enter the game.

Avatar

Wow! At present, if the police detain a non-English-speaking foreigner, they have to figure out what language they speak, then arrange an interpreter. Maybe in the future all they’ll have to do is put on their glasses…

Avatar

While I don’t doubt it’s genuine, it’d obviously be very easy to ‘spoof’ a video like that ;)

The RaspberryPi(s) are “only” being used as video output, IMHO it’s the reliable speech recognition (the article doesn’t mention how/where that part’s being done?) that is the impressive part. But it’s a really nice demo of “plugging together” different bits of technology to create something impressive :)

Avatar

I think you’re using the term “real-time” a little loosely here. But it’s definitely a step in the right direction.

*this post has been edited by Raspberry Pi to remove the personal data of another user at the request of that user.

Avatar

I am gonna give this idea a Thumbs Up! Universal Translators are not Sci-Fi anymore? Great Idea!

Avatar

Next up is to shrink this setup down to the size of a combadge and the future is here.

Avatar

Is this for real? Has rt voice recognition and translation really become this good?

Avatar

How is the speech-to-text processing being handled? Thanks!

Avatar

For anyone who wants to know the “Magic” that is behind this idea!

Via: YouTube Channel…
“Published on 22 Jul 2012 by Will Powell
Inspired by google’s project glass I put together an application that can provide translated subtitles in real time. It allows me and my sister Elizabeth to have a conversation when I speak English and she speaks Spanish.

It uses two Raspberry Pi boards, Vuzix Star 1200, Jawbone mic. Audio is picked up and streamed from mobile device to a server that recognises, translates (uses microsoft translate API and caching layer) and pushes back. One pi drives the glasses and the other the TV that shows both halves of the conversation. Elizabeth uses a mic but would use the same Vuzix, Jawbone, Pi setup if I had another set of kit.

Find out more at http://www.willpowell.co.uk or follow me @WillPowellUK”

Avatar

All it says is “a server that recognises (sp)” the speech. But what is doing that recognition?

Avatar

would you be willing to write something for the magpi to tell us how you did the project?

Avatar

If this device can understand (let alone translate) the Spanish this girl is talking is better than me and I am native from Spain .

Avatar

Being deaf myself, that would be marvellous just to be able to read in my spectacles what people are saying!!

Avatar

Awesome project, but just for the record: Is well known that in Spain we have bullfighting and it is usually to see flags with the “spanish bull” on in. I have to say (as a spanish guy) that most people living in spain (90%+) hates bullfighting and see as an old tradition that should be removed. I don’t want to start a debate pro/con bullfighting but I just want to be clear that a lot of spanish people hate this “bull” on our flag. Congratulations for the projects and keep doing things like that ;)!

Avatar

Whats the song name in that YouTube clip?

Avatar

Thanks Liz, I thought this might not make it here, your the best.
Russ

Avatar

‘Only’ $5000 for the glasses? Seems a bit pricy… :-/

Avatar

The girl’s Spanish is terrible but….

ARE YOU FREAKIN’ KIDDING ME?

Great, honestly great project! It’d be interested to know what the TV and iPhone (actually it was an iPad I believe) do… Is the translation done by the Pis?

Avatar

What is the small display the other Pi is connected to? Has it got a touchscreen?

Avatar

Absolutely amazing. I too am somewhat deaf and constantly asking my kids to speak up drives them nuts. It would be nice to have these built into my prescription glasses so I could see subtitles when people talk. The time it would save me in confusion would be incredible! Kind of in that limbo of not truely 100% deaf but just to the point its annoying. I can’t wait. Hope its in my lifetime.

Leave a Comment

Comments are closed