User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Mon Sep 04, 2017 9:59 pm

I rebuilt for pypi installation and now installs on both Jessie and Stretch without having to jump through hoops. Never had to use wheels before, but things have changed in that direction.

On Jessie, for python 2, first do a

Code: Select all

sudo apt-get install python-dev
Followed by:

Code: Select all

sudo pip2 install python-banyan
All others combinations - python 2 and python 3:

Code: Select all

sudo pip2 install python-banyan
Note on Jessie, it will install a newer version of numpy and this takes a while to build.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Tue Sep 05, 2017 12:52 am

pumpkinpi wrote:
Mon Sep 04, 2017 3:48 am
In the meantime, I tried asandford's recommendation. I did this:

Code: Select all

sudo apt-get install mosquitto
sudo pip install paho-mqtt
And these two things installed no probs.

I did some reading here:
https://pypi.python.org/pypi/paho-mqtt/1.3.0

but quickly got lost. Have not yet found a tutorial for dummies on how to make this work on my raspberry pi's running simple python scripts. I am just looking to pass strings and simple numbers between them.

If anybody knows of guides, I would appreciate direction.

I think I just need to see a few examples. To late to keep googling...

PPi.
Starting on a fresh stretch lite image:

Code: Select all

 sudo apt update; sudo apt upgrade -y
 sudo apt install -y mosquitto mosquitto-clients
 
You've now installed the mosquitto MQTT broker (think of it as a server) and the clients (publisher and subsriber)

Now type the following:

Code: Select all

[email protected]:~ $ mosquitto_sub -d -t hello/world
Client mosqsub/1389-raspberryp sending CONNECT
Client mosqsub/1389-raspberryp received CONNACK
Client mosqsub/1389-raspberryp sending SUBSCRIBE (Mid: 1, Topic: hello/world, QoS: 0)
Client mosqsub/1389-raspberryp received SUBACK
Subscribed (mid: 1): 0

Open another terminal or putty session, and type this:

Code: Select all

[email protected]:~ $ mosquitto_pub -d -t hello/world -m "Hello from Terminal window 2!"
Client mosqpub/1416-raspberryp sending CONNECT
Client mosqpub/1416-raspberryp received CONNACK
Client mosqpub/1416-raspberryp sending PUBLISH (d0, q0, r0, m1, 'hello/world', ... (29 bytes))
Client mosqpub/1416-raspberryp sending DISCONNECT
[email protected]:~ $
In your first window you should see:

Code: Select all

Client mosqsub/1389-raspberryp sending PINGREQ
Client mosqsub/1389-raspberryp received PINGRESP
Client mosqsub/1389-raspberryp sending PINGREQ
Client mosqsub/1389-raspberryp received PINGRESP
Client mosqsub/1389-raspberryp received PUBLISH (d0, q0, r0, m0, 'hello/world', ... (29 bytes))
Hello from Terminal window 2!
Client mosqsub/1389-raspberryp sending PINGREQ
Client mosqsub/1389-raspberryp received PINGRESP

So this shows that the broker and clients are working.

Crtl-C out of the 'subcriber' screen and type:

Code: Select all

[email protected]:~ $ mosquitto_sub -d -t ledStatus
Install pip and paho-mqtt from you 'publisher' console:

Code: Select all

 sudo apt install -y python-pip
  pip install paho-mqtt
Collecting paho-mqtt
  Downloading paho-mqtt-1.3.0.tar.gz (79kB)
    100% |████████████████████████████████| 81kB 1.0MB/s
Building wheels for collected packages: paho-mqtt
  Running setup.py bdist_wheel for paho-mqtt ... done
  Stored in directory: /home/pi/.cache/pip/wheels/a8/fb/b8/25cf89a4ebec3846c24b8feec8469f8402e7706c2c962c84b4
Successfully built paho-mqtt
Installing collected packages: paho-mqtt
Successfully installed paho-mqtt-1.3.0
Now create a test python program and run it:

Code: Select all

[email protected]:~ $ vi testmq.py
[email protected]:~ $ cat testmq.py
import paho.mqtt.publish as publish
import time
print("Sending 0...")
publish.single("ledStatus", "0", hostname="localhost")
time.sleep(1)
print("Sending 1...")
publish.single("ledStatus", "1", hostname="localhost")

[email protected]:~ $ python testmq.py
Sending 0...
Sending 1...
[email protected]:~ $
And on your 'subscription' console you should see:

Code: Select all

[email protected]:~ $ mosquitto_sub -d -t ledStatus
Client mosqsub/6104-raspberryp sending CONNECT
Client mosqsub/6104-raspberryp received CONNACK
Client mosqsub/6104-raspberryp sending SUBSCRIBE (Mid: 1, Topic: ledStatus, QoS: 0)
Client mosqsub/6104-raspberryp received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/6104-raspberryp received PUBLISH (d0, q0, r0, m0, 'ledStatus', ... (1 bytes))
0
Client mosqsub/6104-raspberryp received PUBLISH (d0, q0, r0, m0, 'ledStatus', ... (1 bytes))
1
Loads more info here for all sorts of IoT devices.

Node-Red has inbuilt javascript clients and the msg.payload var contains just the text that is sent / received, and as it's non-blocking and async, you don't have to write horrible callbacks - it just works.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Tue Sep 05, 2017 2:22 am

Oh, this is cool stuff. Thanks for all the help folks. It is going to take me a few weeks to start playing around with this in my spare time. This is definitely what I need for my little home control IOT project.

I'll report back on progress.

PPi.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Thu Sep 07, 2017 3:18 am

Ran through your mini tutorial tonight @asandford and it WORKS. So cool. Replaced "localhost" with an ip address on my lan and can send messages between Pi's too. So cool.

Thanks!

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Thu Sep 07, 2017 7:56 pm

pumpkinpi wrote:
Thu Sep 07, 2017 3:18 am
Ran through your mini tutorial tonight @asandford and it WORKS. So cool. Replaced "localhost" with an ip address on my lan and can send messages between Pi's too. So cool.

Thanks!
Glad you found it useful.

I used just one Pi for convenience, but usually the clients and broker(s) are on different servers (although inter-process communication can be done fairly easily with MQs on a single host).

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Fri Sep 08, 2017 2:46 am

Ok, trying this inside some existing python scripts and I must be missing something.

So, this is a section from my computer (picontrol) that is subscribing to a topic and taking action if it gets a message:

Code: Select all

import datetime
import time
import paho.mqtt.client as paho

def on_message(pihmi1, userdata, msg):
        audiblealert()
        print(msg.topic+" "+str(msg.qos)+" "+str(msg.payload))

pihmi1 = paho.Client()
pihmi1.on_message = on_message
pihmi1.connect("192.168.0.4", 1883)
pihmi1.subscribe("kitchenoccupancy")
pihmi1.loop_start()
Basically, this is supposed to make an audible alert and print on the screen when somebody walks into the kitchen or it is deemed unoccupied by a motion sensor.

This is the publisher code (pihmi1):

Code: Select all

import time
import datetime
import RPi.GPIO as io
import paho.mqtt.client as paho
io.setmode(io.BCM)
io.setup(18, io.IN)

occupiedcount = 0
picontrol = paho.Client()
picontrol.connect("192.168.0.5", 1883)
picontrol.loop_start()
occupied = 'NO'
picontrol.publish("kitchenoccupancy", occupied)

while True:
    oldmotion = io.input(18)
    time.sleep(2)
    occupiedcount += 1
    if io.input(18):
        if occupied == 'NO':
            occupiedcount = 0
            occupied = 'YES'
            picontrol.publish("kitchenoccupancy", occupied)
    if occupiedcount > 1350:
        if occupied == 'YES':
            occupied = 'NO'
        picontrol.publish("kitchenoccupancy", occupied)
I tried to leave "optional" stuff out to see if it would work this way. Basically, the code does not crash, but I don't see any effect either.

I am running the first one with python3 and the second with python if that makes any difference. I would not expect so.

PPi

mattmiller
Posts: 2113
Joined: Thu Feb 05, 2015 11:25 pm

Re: Sharing data between python scripts through a LAN

Fri Sep 08, 2017 1:49 pm

those are two different ip addresses- but lets ignore that as people never paste the code they are ACTUALLY using :)

Use another MQTT tool to make sure the broker is actually working and receiving and sending messages

I use MQTTBox Chrome app on my Win10 machine to do that

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Fri Sep 08, 2017 6:10 pm

Oh, that is interesting. I thought the addresses had to be different. Should they be the address of the publisher computer or the subscriber?

They are not the actual addresses, but are different in my code.

I did do the stuff above (command line stuff) so I know it is working fine - just not in my code.

Thanks,
PPi

mattmiller
Posts: 2113
Joined: Thu Feb 05, 2015 11:25 pm

Re: Sharing data between python scripts through a LAN

Fri Sep 08, 2017 8:03 pm

Oh, that is interesting. I thought the addresses had to be different. Should they be the address of the publisher computer or the subscriber?
The address on all machines needs to be the address of the broker you are using

MQTT is based on there being a broker running on a computer somewhere (can be on your own LAN or on the internet like test.mosquitto.org )

All computers that want to share data on a particular topic must publish and/or subscribe to the broker that is carrying that topic

Are you running a broker on your LAN?

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 1:39 am

pumpkinpi wrote:
Fri Sep 08, 2017 6:10 pm
Oh, that is interesting. I thought the addresses had to be different. Should they be the address of the publisher computer or the subscriber?

They are not the actual addresses, but are different in my code.

I did do the stuff above (command line stuff) so I know it is working fine - just not in my code.

Thanks,
PPi
In the mini tutorial, the clients and server are all installed on one host. All the references to localhost should refer to the hostname of the device running the broker. The client IP addressess do not need to be used, or even known, for a successfull connection. Think of the process like this forum. You need to know the address to post a message, and I need to know it to read it. You don't need to know who will read your post, and I don't need to know the poster - hence the forum is a broker of sorts.

There are advanced cases where clients can connect to different bokers, and the brokers are bridged, but I don't think you're at that stage yet :)

The use of a broker provides better QoS (the broker knows which clients have received a message and which ones have missed it) and can redeliver if need be. MQTT is designed to be lightweight on resources and there are more feature rich protocols available. I use the RabbitMQ package which can broker MQTT messages, as well as a host of other protocols.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 1:55 am

Ok, so I got it to work by putting 'localhost' in the one machine (broker). In the other one, I left the IP address of the broker and it worked. Putting localhost in both, does not work.

This is cool stuff. I will just set one Pi on my network as the broker and have all other machines reference to that one. If I do that, can one "client" machine send messages to another "client" if they are both subscribed to the same topic?

I am having an issue with reading the message that comes back. It prints a b'YES'. I believe that means byte. I just want strings.

Code: Select all

kitchenoccupancy 0 b'YES'
Can't figure out how to just get strings to come through.

Thanks,
PPi

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 1:57 am

Figured out the conversion. It was just:

Code: Select all

if msg.payload.decode("utf-8") == "YES":
PPi

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 2:36 am

pumpkinpi wrote:
Sat Sep 09, 2017 1:55 am

This is cool stuff. I will just set one Pi on my network as the broker and have all other machines reference to that one. If I do that, can one "client" machine send messages to another "client" if they are both subscribed to the same topic?
Yes. If one client publishes to a topic, all subscribers to that topic will receive it. It's worth pointing out that a client that is publishing may also be subscribing, so the flow of data can be complex.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 7:14 pm

IT WORKS! IT WORKS!

This is the coolest thing I have done on my Pi's. I wish I had figured this out years ago. I have about 12 Pi's in my house doing lots of different jobs and the one thing I have struggled with the most is getting them to coordinate with each other. I have a lot of code to sort through now and rewrite.

This is so much more powerful than other solutions I have used.

Thanks a million for the assistance folks.

I just can't believe how easy this is in the end.

PPi

mattmiller
Posts: 2113
Joined: Thu Feb 05, 2015 11:25 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 8:12 pm

I just can't believe how easy this is in the end.
I'm with you :)
The fact that its available to use and run from Desktop PCs to little ESP8266s is brilliant :)

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Sat Sep 09, 2017 10:29 pm

pumpkinpi wrote:
Sat Sep 09, 2017 7:14 pm
IT WORKS! IT WORKS!

This is the coolest thing I have done on my Pi's. I wish I had figured this out years ago. I have about 12 Pi's in my house doing lots of different jobs and the one thing I have struggled with the most is getting them to coordinate with each other. I have a lot of code to sort through now and rewrite.

This is so much more powerful than other solutions I have used.

Thanks a million for the assistance folks.

I just can't believe how easy this is in the end.

PPi
Now try Node Red and MQTT and you can have asynchronous, non-blocking, event driven code. I have a green house monitoring system reading sensors and controlling switches with it. It also has easy database integration and message transformation, native gpio control on a Pi and Arduino support with firmata. It's already built in to Raspbian, and is developed by IBM.

Return to “Python”