Monty, a Raspberry Pi personal assistant

Liz: James Gallagher is 13 years old. He sent me a little bit of information about a voice-activated personal assistant project he’s been working on: I thought it was so neat that I asked him to write about it here. Over to James!

I envisioned an idea where you could rather than controlling your computer with a mouse, you can do it with your voice. This got me thinking, Siri is available for Apple devices and Cortana is available for Windows devices but what about Linux? So I embarked on a project to attempt to bring this ability to the Raspberry Pi, I know that it would take a long time and might not happen however I stuck at it and finally created a product. Meet “Monty”! Named The “Monty” Project, I made a piece of software where you could interact with your Raspberry Pi via Voice. So far, you can execute some basic commands that would normally be used in terminal and you can exchange pleasantries.

Some basic commands include asking it about the news and weather, some mathematical questions, launching programs and executing commands normally done through terminal. Also adding items to Crontab and lots lots more! This program takes advantage of the Web API’s available from Wolfram Alpha and RSS Feeds to send data to the program where it then speaks what you asked. Below you will see a video demo of the software in action.

Monty Program Official Showcase.

Showing off my Monty speech program with more advanced commands including navigate to a directory and list files in a directory. It also shows off the log it takes a little better and shows that at one point it misunderstood me. Thank you for watching.

I created this to provide an experience I couldn’t find anywhere else and to help others use computers or just have some fun. I showcased “Monty” at my school and allowing others to attempt to speak to my Raspberry Pi and proved effective. I hope to attend some Raspberry Pi conferences to show it off to the public and give others the chance to use it before release. When the software is released there will be an API with full documentation so that it will be easy for you to configure the software and are it your own.

Thanks for reading and have a nice day!

Note: Right now this software is still in Beta and not publicly available, the release window is July-December 2016. There are more features I hope to implement and still quite a few bugs to iron out. Please follow me on Twitter to keep up to date with the software. @James1Gallagher

36 comments

Avatar

Waaa! Reminds me of windows 95 and clippy! Then office assistant. And now we have suri, cortana. I forgot what google’s version is.
Well good luck lad. Another young brilliant engineer.
Seems like a lot of engineers coming from those islands.

Avatar

Great job! Keep it up, but consider putting your code on GitHub to grab some more attention and possibility to cooperation.

Avatar

Great job. I could never have imagined accomplishing something so complex when I was 13, pretty sure I still can’t do it. Though you might want to investigate the Jasper project as well as the Alexa Voice Service from Amazon before you get to much further. Could save you some time in implementing your additional features.

@Solar3000, Googles version is just “Ok Google” I believe, no real name for it.

Avatar

Google Now

Avatar

Thanks you guys for all the positive feedback, the code is going on github soon and I will look into Jasper. Thanks again, have a nice day!

Avatar

Just noticed, I made a mistake at the end. It should say: “When the software is released there will be an API with full documentation so that it will be easy for you to configure the software and make it your own.”. I just noticed.

Avatar

Hey, I’m 13 years old as well! What you have made here is awesome, and could change things forever. Carry on with this, I think this’ll go places!

Avatar

Hello MR. Gallagher,

Will your program be able to execute commands for a .MP3 player? I have an older brother who has mental retardation. He doesn’t speak clearly, kind of muddles his words. Although he is physically 52 years of age his mind is roughly equivalent to 4 to 10 years of age. He has been going blind but he dearly loves music. I have engaged the services of an online friend to come up with a .MP3 player for my brother using Raspberry Pi. When your software is done can it be used as the primary interface when the computer is turned on without having to do anything else?

Cheers,

Bruce Berryhill
Game Designer, Writer, Fractal Artist

Avatar

I am working on .mp3 integration with it however I am having a few troubles. It is a shame what happened, that just makes me want to work harder to get this out.

Avatar

I’m not sure how much work it’d be, but I wonder if you could integrate with http://www.freedesktop.org/wiki/Software/DbusProjects/ in order to allow Monty to control other programs?

Avatar

You should be able to adapt the audiobook player featured here into a music player?
https://www.raspberrypi.org/blog/sunday-grab-bag/

Avatar

It was great! Looking forward to seeing more great things coming from you, James.

Avatar

This was awesome! It would be nice if some of the cleaner text to speech systems were available for Linux/Pi, so the voice could be less robotic. I would like to know if this uses local voice processing, or if it uses the cloud based voice to text systems? For portability sake, it’s nice to have local voice recognition, but understand that the cloud systems can bring in so much more power and accuracy to the problem. Either way, this is so awesome! Looking forward to hearing more of this in the future!

Avatar

Well, I am running a Ubuntu Server on a Raspberry Pi but only as a backup of all files. I thought that cloud implementation would just take up too much time I don’t currently have. I am going to look into it just after I have worked on the API. It wouldn’t be fun to download the tool and download lots of huge files just to make it work. Do you have any cloud service recommendations I could use for files? As for the voice, I couldn’t program something that didn’t sound robotic so I stuck with the current voice. Thanks for the feedback, I will take cloud file and voice hosting into consideration. Have a nice day!

Avatar

I must mention, I did not know there was voice recognition programs available already so this was independent until you told me about Jasper and espeak. I shall look into them just to see what they are all about. Have a nice day.

Avatar

So, are you using the pocket Sphinx package? I found that really cool, but it had issues with noise on my older model B’s and the USB mics that I had. The cloud based voice recognition is usually done with the google engine. There’s an API. There is also an engine through Apple, I think. You just send the WAV file up to the system, and it sends back the text file to be parsed. There are tutorials on that. It just means the Pi has to be tied to the network to work. I work on the Internet, I rather my things not be dependent on it. (Tw/oI) :-)

Avatar

Excellent job! Another great example of the vast functionality of the Pi.

Avatar

looks (Sounds) good – is this an off shoot of Jasper / Jarvis ?

using espeak ?

will look forward to seeing progress on this
Excellent Job

Avatar

Outstanding work. Looking forward to seeing it in full glory and then playing with it myself.

Avatar

Lovely! What extra hardware is necessary? Getting live audio *into* a pi ain’t the easiest task without.

Avatar

Another Jasper ?
https://www.youtube.com/watch?v=UzaqNF6NlBA

Take a Look at Jasper Forum, if you want to build a similar project….
https://groups.google.com/forum/#!forum/jasper-support-forum

Jasper Source Code
https://github.com/jasperproject/jasper-client

Avatar

Fantastic work James. I will show this to our coderdojo club and know it will inspire them.
We are using rasberry p’s in the club for advanced coding.
Looking forward to seeing your progress with the voice project

Rory

Avatar

Wow. I am 14 and got into coding a little over a year ago. To see someone younger than me that has done something that I have no clue how to do is very humbling. I wish you the best in your endeavors and I hope that some day I will be able to do something just as amazing.

P.S. You can click on my name above to see what I would consider my biggest project.

Avatar

You have no idea how those of us three times his age feel!

Avatar

Absolutely fantastic! When I was 14 years old, I played video games.
Great job!

Avatar

James or other interested parties, you may want to take a look at Mycroft.
https://mycroft.ai

It is a fairly new open source project that does language analysis etc. It has been touted as the next generation of speech to text and actions for Linux. e.g. a competitor to Siri/ Cortana/ Alexa etc.
I think Mycroft’s analysis is done in the cloud when used on the Pi, but that may change.

I have no idea how you have done this so far, good work & good luck :)

Avatar

Absolutely fantastic job, James!! It doesn’t matter if your project parallels the Jarvis project or others…it spoke to you (literally) and you made it happen! It would be lovely to see you continue with this work, either independently or in conjunction with others like Jarvis. As each programmer has their unique style and way of interpreting a problem, you have the ability to analyze, theorize, and capitalize on Monty. Again, very impressive, ESPECIALLY at your age! Good work and best wishes in everything you pursue :-)

Avatar

Very nice, good job !
I started a similar project https://github.com/vsilent/smarty-bot 2 years ago, and I see people interested in this :) . That’s motivates me to continue !

Avatar

There is also the similar Gladys project (http://gladysproject.com/)

Avatar

Good work!
Please continue with the good work. It would be nice to see your work on github, because I want to learn about it. Dont worry about it not beeing finished or unpolished we are all here to learn! Even a 36 year old guy like me. Best wishes James!

Avatar

If I create something and wouldn’t use it myself then I would not release it. I do use “Monty” but at the moment everything is done in the least productive way possible. I am at the cleanup stage right now, to this point i just wanted it to work but now whilst cleaning I am finding more obstacles to ruin me. That is the way I have always been but I am considering a private beta for a limited amount of people to join and test then I can release it. Follow me at @James1Gallagher if you don’t already to get more info as it happens. Thanks for the advice and I hope you have a nice day.

Avatar

Sounds nice! Are you using an API to reach an external speech to text service or doing it all local to the Pi? I’m building a Jasper system at the moment (hooking up a high quality mic to it this weekend) to play with. I want to use the Julius STT engine so all the processing is local but it’s a pain to install. Best of luck with your project.

Avatar

heeyy greaattt project!! can i have the step by step process along with the codes???if u dnt mind

Avatar

Very nice work, congratulations.

I have an interest that might use a stripped down version of your code. I have a remote activated trolling motor on my fishing boat that uses a small hand-held radio control with 5 buttons to Stop, Slow down, Speed up, Turn right and Turn left. I would like to implement a voice control so I do not have to stop fishing to push buttons. The simplest thing would be to get another hand control (which the trolling motor has to recognize) and disassemble it and wire the button circuit switches to an RPi that would activate the buttons based on 5 voice commands (Halt, fast, slow, right, left or H,F,S,R and L if that’s easier). Would it be difficult to modify your code to do such a thing? Thanks!

Avatar

I want the program code :P

Leave a Comment

Comments are closed