Page 1 of 1

Kalliope (Jarvis like assistant)

Posted: Tue Mar 21, 2017 2:04 pm
by sispheor
Hi!

We made a little bot assistant for the Rpi !
It's more like a framework where you you can build your own assistant like Jarvis in the language of your choice.
Some modules (neurons) are already availlable to make your brain(eg: gmail, gcalendar, facebook, date,...).
And you are free to create your own and propose them to the community.

Demo video
The website
The source code
Already available modules

Re: Kalliope (Jarvis like assistant)

Posted: Tue Mar 21, 2017 3:33 pm
by YCN-
Wonderfull I like this kind of project !
What did you use for voice recognition ?
I like the way you did the "synapses" stuff. I'm not really sure that I exactly get what you meant. Is it like plug-in ?
Is one built for the use of relays ?

YCN-

Re: Kalliope (Jarvis like assistant)

Posted: Tue Mar 21, 2017 4:19 pm
by sispheor
For the voice recognition you can select a STT engine. There is also a self hosted one.

Yes a synapse is the link between input and output actions.
The input is your order, the output is a neuron (a plugin).

There is no neuron for controling GPIO direclty. But you can use the neuron script
or shellto do it.

Re: Kalliope (Jarvis like assistant)

Posted: Tue Mar 21, 2017 4:41 pm
by bacardi55
Multiple STT and TSS can be setup (with multiple languages), I'm mainly using google STT for now as on a single RPI, it can become a bit slow to install too much stuff on it (already have Kalliopé and Mopidy) ^^.

Sispheor has explained the synapses, but the idea is really simple:
Links your voice orders (input) and neurons (functionality and message templates - the output).
So you can create orders and templates in the language you need and just reuse data sent back by the neurons. Readme of neurons can give you more ideas of what I mean :).

For the GPIO connection or other domotic devices, I'm using domoticz to manage them, and created a neuron for Kalliopé to interact with domoticz.

If you are interested, I wrote some stuff about my (heavy?) kalliopé usage here
That being said, Kalliopé is a cool tool and have now some cool neurons

Re: Kalliope (Jarvis like assistant)

Posted: Wed Mar 22, 2017 9:28 am
by YCN-
Wow !

I love it !! I was thinking about doing this kind of assistant myself, but I would have done it with a low level approach, using C and my I/O would have been very user-specific, but I like your style, it's better and more user-friendly.

I think that I'll soon get involved in the project, did you consider adding an http server so that you can drive kalliope in a silent mode? May be with a web site for the user where you'll only see something as the facebook messenger app. I'm not really into web devellopment but that could be something I'd like to have.

What do you think ? What's your current approach for voice recognition ? Do you translate words into strings and then annalyse them ? If this is how you do it, the server might be pretty straight forward, I'll just have to forward to kalliope the http string data.

YCN-

Re: Kalliope (Jarvis like assistant)

Posted: Wed Mar 22, 2017 9:37 am
by sispheor
Currently Kalliope has an API.
A projecthas been started for controlling Kalliope via web interface.
And a Phone app should be available soon.

And yes we convert voice into string and then we analyse them to match a declared order in the conf.

Re: Kalliope (Jarvis like assistant)

Posted: Wed Mar 22, 2017 9:41 am
by bacardi55
YCN- wrote:Wow !

I think that I'll soon get involved in the project, did you consider adding an http server so that you can drive kalliope in a silent mode? May be with a web site for the user where you'll only see something as the facebook messenger app. I'm not really into web devellopment but that could be something I'd like to have.
I'm not sure what you mean by "silent mode" ^^.
Kalliopé provide an API so you can start synapse (or order) via API. You can even send an audio file via API and then Kalliopé will use the TTS as usual.
That being said, when you use the API, the synapse is executed on the rpi, so if your synapse has an audio return, it will be said on the rpi that leverage kalliopé. I have created an issue to maybe modify this behavior, but it's not started yet. Issue can be found here

Also, I have started a web remote app (using emberjs) that leverage the API to run synapse. I have stopped for some time to contribute to more neurons but I'll restart working on it soon. It will be able to send free text (orders) or an audio file, so you won't need a mic or being in the same room as your Kalliopé
App can be found here
YCN- wrote: What do you think ? What's your current approach for voice recognition ? Do you translate words into strings and then annalyse them ? If this is how you do it, the server might be pretty straight forward, I'll just have to forward to kalliope the http string data.
YCN-
Yes, Voice recognition works by transforming audio into strings that are then analyse by kalliopé.

Re: Kalliope (Jarvis like assistant)

Posted: Wed Mar 22, 2017 9:57 am
by YCN-
Well I was meaning instead of talking to kalliope finding a way to write to kalliope. For instance with a chat interface such as the messenger app. I don't really like the idea of talking to a robot. That why I'd like to have a web interface where I would be able to write what I would have said.

You'll go to a web page and then just write and have responce in real time with kalliope. Just like the FaceBook messenger app or whatever messenger you use (whatsapp...).

I'll give it a try soon I don't think it will be too complicated.

YCN-

Re: Kalliope (Jarvis like assistant)

Posted: Wed Mar 22, 2017 10:00 am
by bacardi55
YCN- wrote:Well I was meaning instead of talking to kalliope finding a way to write to kalliope. For instance with a chat interface such as the messenger app. I don't really like the idea of talking to a robot. That why I'd like to have a web interface where I would be able to write what I would have said.

You'll go to a web page and then just write and have responce in real time with kalliope. Just like the FaceBook messenger app or whatever messenger you use (whatsapp...).

I'll give it a try soon I don't think it will be too complicated.

YCN-
That is the goal of the web app i've done. So far, it's only listing available synapse and you can click to start them. I had in mind this ability to just run free text so it's in the roadmap, just had no time to tackle it yet.

Otherwise, I started a small CLI tool to do so, where you have a prompt and can simply type "order my great order" and it will send the order via API ("my great order"). You can find it here but it is just started (but this feature works).
I had in mind other features like installing new neurons or even configuring kalliopé via this tool.

*edit* : the link is: https://github.com/bacardi55/kalliope-cli

But all this is still yound and need more work :)

Re: Kalliope (Jarvis like assistant)

Posted: Sun Apr 02, 2017 2:50 am
by morecoffee
bacardi5,

I'm trying to run you CLI interface on a raspberry pi B to get an idea of how things works. I am using the v0.4.3 image. I'm having some trouble getting started. All I have done is setup the image and download your CLI program and changed it to executable.

When I run the command

Code: Select all

kalliope start
I get a long message that ends with:
in swig_import_helper
import _snowboydetect
ImportError: No module named _snowboydetect
When I try

Code: Select all

./kalliope-cli.py
The results were:
Traceback (most recent call last):
File "./kalliope-cli.py", line 10, in <module>
from cmd2 import Cmd
ImportError: No module named cmd2

Re: Kalliope (Jarvis like assistant)

Posted: Thu Apr 06, 2017 9:16 am
by sispheor
Is that ok for you now @morecoffee?

Re: Kalliope (Jarvis like assistant)

Posted: Thu Apr 06, 2017 1:43 pm
by morecoffee
I'm fine for now. I plan on getting a Pi 3B with a mic and try again later. Thanks for the help on the gitter chat. I look forward seeing how this project continues to develop.

Re: Kalliope (Jarvis like assistant)

Posted: Wed Jun 28, 2017 10:00 am
by sispheor
Version 0.4.4 online! https://github.com/kalliope-project/kal ... tag/v0.4.4

Better performance on Rpi !

Re: Kalliope (Jarvis like assistant)

Posted: Sun Jan 14, 2018 10:11 pm
by sispheor
V0.5.0 online !!
New text to speech, hooks support and more. See the complete changelog here.

Re: Kalliope (Jarvis like assistant)

Posted: Thu May 31, 2018 2:03 pm
by sispheor
v0.5.1 online.
You can now chain your orders!

New doc: https://kalliope-project.github.io/kalliope/

Re: Kalliope (Jarvis like assistant)

Posted: Sun Jul 15, 2018 6:49 pm
by NOsen
First of all thank you for your work on this.

Question i got it running on my raspberry it talks i can chat with it from my phone
however when i use the hotword "snowboy" it do not respond at all im using google speech to text paid version with a proper API key

If i record from the mic it works and it sounds good when i play up it but it just wont respond to any spoken word.

my asoundrc looks like this

Code: Select all

 cat .asoundrc
pcm.!default {
    type hw
    card 1
}
ctl.!default {
    type hw
    card 1
}

and like i wrote it do record my voice when i run arecord.


My settings file

Code: Select all

default_speech_to_text: "google"

speech_to_text:
  - google:
      language: "en-EN"
      key: "APICODE"

it seems like its not even trying to connect to google at all my google account has 0 hits and im running the API unrestricted so it should work

Kalliope log (there is no errors in the eror log

Code: Select all

Starting REST API Listening port: 5000
Starting Kalliope
Press Ctrl+C for stopping
zy loading)
Starting order signal
 * Environment: production
I'm ready
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: on
Any suggestions would be appreciated!

Re: Kalliope (Jarvis like assistant)

Posted: Mon Jun 03, 2019 7:44 am
by Mistgun
Salut,

J'ai actuellement installé Kalliope et Matrix voice sur ma Raspberry.

-J'ai créer un mini programme lumière avec Matrix qui se trouve: cd /root/matrix-hal-project/matrix-hal-project/app
"app" étant compilé, est exécutable avec la commande ./app

-J'ai modif mon brain.yml pour créer une nouvelle commande:

- name: "lumieretest"
signals:
- order: "lumieretest"
neurons:
- shell:
cmd: "cd /root/matrix-hal-project/matrix-hal-project/app"
- say:
message: "lumiere active"


Problème: Il n'exécute pas "app", comment je dois faire pour qu'il l'exécute (./app) ?
(sachant que "app" est en C++ et a été compilé avec g++)

Re: Kalliope (Jarvis like assistant)

Posted: Sun Jun 16, 2019 8:48 pm
by sispheor
c'st pas cmd: "cd /root/matrix-hal-project/matrix-hal-project/app"

mais

cmd: "cd /root/matrix-hal-project/matrix-hal-project/ && ./app"

je pense

Re: Kalliope (Jarvis like assistant)

Posted: Tue Jun 18, 2019 7:09 am
by Mistgun
Salut,
Oui effectivement c'est sa, j'avais trouvé mais je n'avais pas publié la réponse ici.

J'ai une 2ème question: J'ai fait un script bash/shell (api.sh) qui en gros récupère des valeurs sur un site puis les affiches (via "echo") et j'aimerai savoir comment faire pour que Kalliope puisse lire ces valeurs

EDIT:
J'ai fait une fonction à l'intérieur de mon shell (api.sh) et prenons comme exemple $valeur = 1 :

Code: Select all

function apitest {
code 
}

apitest() {
echo "valeur: "$valeur 
}

apitest
Le problème est que quand le fichier est exécuté par Kalliope: Tout s'affiche correctement sauf la valeur de "$valeur"
Il me met:

Code: Select all

valeur: 
à la place de:

Code: Select all

valeur: 1
Que faire ?
Merci (et oui je suis sans doute pas sur le bon forum pour cette question ;) )

Re: Kalliope (Jarvis like assistant)

Posted: Wed Jul 17, 2019 2:34 pm
by salomeleduc
Hello everyone, I know those post are old but I'm trying Kalliope and I have an issue : I'm using the Kalliope_starter_fr and pronounce "Kalliope" but nothing happened.
My microphone is working and when I was working with CMU Sphinx it was listening to me, but here, I have nothing.
Could you help me?
Thanks !!
Salomé

Re: Kalliope (Jarvis like assistant)

Posted: Fri Jul 26, 2019 7:00 am
by evansusu
EditDelete postReport this postQuote
Fri Jul 26, 2019 3:00 pm

https://www.thegeekpub.com/16187/contro ... pberry-pi/
Hi Guys, I followed this tutorial to light up a WS2812b led strip with 60 LEDS using a raspberry pi. Unfortunately it doesn't work.

'import board
import neopixel
pixels = neopixel.NeoPixel(board.D18, 60)

pixels[0] = (255, 0, 0)

After running it, I got this error in the images