mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

MQTT Python paho library for IoT

Sun Sep 04, 2016 10:40 pm

JFI
If you want to send IoT msgs around your house/work/school/world I highly recommend to checking out the MQTT protocol system and the paho Python library as its a very easy to use system

Quick MQTT primer
Run a program on any computer (pi/pc/mac/linux) called a broker (just run it on any computer that is basically kept running 24/7)

Then on say a Pi connected to your fridge light, send msgs on state of light to the broker.

On another Pi in your bedroom, connect to the broker computer and wait for the broker to forward the msgs from your fridge Pi

Connect a LED to your 2nd pi and get it to light up when fridge light is lit.

Use this setup to remotely check anyone sneaking food out of the fridge in the middle of the night

PS If you use an external broker on the internet - you can even check the fridge light from ANYWHERE in the world :)
Last edited by mattmiller on Sun Mar 04, 2018 4:32 pm, edited 1 time in total.

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Mon Sep 05, 2016 6:49 am

Also don't use the ancient version of Mosquitto that's shipped with Jessie. Get the latest from
http://mosquitto.org/2013/01/mosquitto- ... epository/
that version has the websockets support.

Install the paho stuff using pip and/or pip3.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Mon Sep 05, 2016 6:59 am

Whats the problem with the one shipped with Jessie?
I've been running clients for a few months now and not noticed any problems

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Mon Sep 05, 2016 7:11 am

mattmiller wrote:Whats the problem with the one shipped with Jessie?
I've been running clients for a few months now and not noticed any problems
It's ancient history and missing websockets.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Mon Sep 05, 2016 12:26 pm

Ta
Luckily I don't need websockets (don't even know what they are!)

Massi
Posts: 1691
Joined: Fri May 02, 2014 1:52 pm
Location: Italy

Re: MQTT Python paho library for IoT

Mon Sep 05, 2016 1:16 pm

DAMN paho 1.2 is out and dougie didn't alert me!!!!!!

subscribe simple! FINALLY! this is changing my life! definitelly! :lol:

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Fri Oct 07, 2016 8:02 pm

@dougielawson
OK Obiwan - I now need MQTT websockets :)

I followed this http://www.switchdoc.com/2016/02/tutori ... pberry-pi/

but I'm wondering if I should be compiling a version instead

HELP!!! :)

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Fri Oct 07, 2016 9:39 pm

What version do you have?

Code: Select all

pi@aplus ~ $ mosquitto -v
1475876298: mosquitto version 1.4.10 (build date Thu, 25 Aug 2016 10:12:09 +0100) starting
1475876298: Using default config.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Fri Oct 07, 2016 9:54 pm

1475877277: mosquitto version 1.4.10 (build date Thu, 25 Aug 2016 10:12:09 +0100) starting
1475877277: Using default config.
1475877277: Opening ipv4 listen socket on port 1883.
1475877277: Error: Address already in use

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 7:49 am

OK. So we can update the config file for that. Create a new file /etc/mosquitto/conf.d/ws.conf

Code: Select all

port 1883
protocol mqtt
listener 8883
protocol mqtt

listener 9001
protocol websockets
then grab a copy of https://github.com/jpmens/simple-mqtt-w ... master.zip and unzip it to /var/www/somedirectorygoeshere so you can access it with your web server.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 10:25 am

Done it all but
/var/www/somedirectorygoeshere so you can access it with your web server.
What webserver?

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 1:26 pm

mattmiller wrote:Done it all but
/var/www/somedirectorygoeshere so you can access it with your web server.
What webserver?
All three Apache2, Lighttpd or Nginx all serve HTML, CSS and JavaScript from /var/www/ or /var/www/html/
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 3:55 pm

OK - which is the simplest to get going?

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 4:22 pm

mattmiller wrote:OK - which is the simplest to get going?
Apache2 but it's the heaviest resource hog.

Apache2 uses /var/www/html as the document root directory.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 4:28 pm

Ta
I'm trying this out on a Pi3 so should be OK hopefully

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 4:43 pm

OK
Got mosquitto running
Published a message to it - msg appears in LX terminal window on Pi

Got appache2 running
put all the files from the .zip into /var/www/html

On PC stick 192.168.0.18 into broswer

Get this
0.png
0.png (11.37 KiB) Viewed 11949 times

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 6:15 pm

Edit the ../config.js file change the IP address in there to 192.168.0.18 then reload the web page.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

DieOma
Posts: 86
Joined: Mon Aug 03, 2015 4:02 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 7:37 pm

Hi,

I think, the easiest way to work with mqtt is node-red .
No Coding... just like scratch
Generally suitable for IoT

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 8:51 pm

Edit the ../config.js file change the IP address in there to 192.168.0.18 then reload the web page.
Obvious when you point out the config file :)
But still no joy

Does Apache2 need to be configured for websockets?

This websocket thing is definitely not the plug and play that the Paho python lib is!

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 8:55 pm

I think, the easiest way to work with mqtt is node-red .
No Coding... just like scratch
Generally suitable for IoT
Agreed for us geeks doing the IoT engineering stuff - but looking to see if I can make a web page for users to interface to the MQTT world so householders can control stuff simply

NodeRed interface doesn't work very well on a 5in screen :)

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 9:24 pm

Any idea what this is for in index.html?

Code: Select all

    function MQTTconnect() {
	if (typeof path == "undefined") {
		path = '/mqtt';
	}

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 10:13 pm

Apache2 is only serving the html, css & javascript. It needs nothing more.

The websocket is a connection between your browser and your MQTT server. This is what I see when it works for me.

Code: Select all

root@apollo ~ # netstat -tlnpu  | grep -e 'apache' -e 'mosqu'
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN      9324/mosquitto
tcp        0      0 0.0.0.0:9001            0.0.0.0:*               LISTEN      9324/mosquitto
tcp        0      0 0.0.0.0:8883            0.0.0.0:*               LISTEN      9324/mosquitto
tcp6       0      0 :::1883                 :::*                    LISTEN      9324/mosquitto
tcp6       0      0 :::443                  :::*                    LISTEN      1285/apache2
tcp6       0      0 :::80                   :::*                    LISTEN      1285/apache2
tcp6       0      0 :::8883                 :::*                    LISTEN      9324/mosquitto
root@apollo ~ # netstat -tnpu  | grep -e 'apache' -e 'mosqu'
tcp        0      0 192.168.1.11:9001          192.168.1.67:62667         ESTABLISHED 9324/mosquitto
root@apollo ~ #
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

asandford
Posts: 1998
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 10:14 pm

mattmiller wrote:
I think, the easiest way to work with mqtt is node-red .
No Coding... just like scratch
Generally suitable for IoT
Agreed for us geeks doing the IoT engineering stuff - but looking to see if I can make a web page for users to interface to the MQTT world so householders can control stuff simply

NodeRed interface doesn't work very well on a 5in screen :)
NodeRed doesn't have an 'interface'.

And NR does websockets, MQTT, Pi GPIO, Arduino Firmata, native file IO, DB access, email, Twitter and many other things. It is used by many people for home automation (including me albeit a greenhouse), mainly as it takes the hassle ofsimultaneous processing. I use freeboard to display the various sensor and actuator values on a webpage (but I'm now 'playing with'* NR Dashboard).

* I don't care if some people object to that phrase, if I'm not getting paid for it then it's recreational.

mattmiller
Posts: 2247
Joined: Thu Feb 05, 2015 11:25 pm

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 10:30 pm

I'm only get listen stuff on 1883 - no mention of 9001
Checking to see if I've made spullig mistook somewhere
NodeRed doesn't have an 'interface'.
Well thats the name I was giving to the web page that interfaces to the stuff going on behind it
Seemed a reasonable description to me :)

User avatar
DougieLawson
Posts: 39576
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: MQTT Python paho library for IoT

Sat Oct 08, 2016 10:32 pm

The /mqtt path is a special path for websockets. The whole tcpdump looks like

Code: Select all

GET /mqtt HTTP/1.1
Host: 192.168.1.11:9001
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: http://dougie.example.com
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
DNT: 1
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en
Cookie: cPrompt_useCookies=2; compact_display_state=false
Sec-WebSocket-Key: ujniwiOw8uZZgcMFCDlqxw==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Sec-WebSocket-Protocol: mqtt

HTTP/1.1 101 Switching Protocols
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Accept: Dp860mUltJ0pHYNGbAaRSrkZkw4=
Sec-WebSocket-Protocol: mqtt

.. ...0...m...$... ...B..... .........G..~..;..........0..
test_topicpayload..g..ad.....

Have you updated your mosquitto config with
/etc/mosquitto/conf.d/ws.conf

Code: Select all

root@apollo ~ # cat /etc/mosquitto/conf.d/ws.conf
port 1883
protocol mqtt
listener 8883
protocol mqtt

listener 9001
protocol websockets
root@apollo ~ #
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All fake doctors are on my foes list.

Return to “Python”