RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Run a asyncio socket function a a thread

Wed Jun 26, 2019 7:48 pm

I'm learning all about threads, asyncio, all because I wish to run a non blocking server.
I'm conscious that there are several variants according to which version of python.
I'm running on a Pi3 with python 3.4.2 or 2.7.9
I'm running micropython version 1.11 on several esp32's

I'll constrain my example to python 3.4.2 for simplicity

Why do I wish to do this?
The Pi3 runs the out control loop: with some interrupt handling, but mainly tkinter control and monitoring pages.
It serves a ring of ESP32 microcontrollers running micopython, who communicate with each other via WIFi local network (one can act as an access point). One is nominated to receive and distribute messages via tcp sockets from the array of ESP32's which are responding to interrupts from sensors, sending PWM control to servos. This is the inner loop monitoring and control.

There are multiple threads on each controller, interruptible

I'm an aging SW engineer chap who has been familiar with inner and outer loop control, ISR and so forth.

So with onboard wifi and a plethora of interrupt pins + PWM what better way than use sockets.

The first thing that I learned was that socket do block by default, so no main programme can run.
The second thing that I discovered that threads become similarly blocked.
So I ended up with asyncio, with threads.

However:
I'm having difficulty (mainly because of the several differing implementations coupled with sparse descriptions of operation of functions) understanding the mechanisms and interaction of asyncio with threads.

This is a cut down version of the programme snippet. It would represent one small element of the application. It merely responds to clients.
The socket mechanism works and the connection behaves. The tasks run. However, the asyncio loop and run_until_complete are in the main programme and block all main body code.

Code: Select all

#asta.py
import asyncio, socket, time,threading

class mythread(threading.Thread):
    def run(self):
        while True:
            print ("In Thread",self.name)
            time.sleep(2)

th1=mythread()
th1.setName("T1")
th1.start()
th2=mythread()
th2.setName("T2")
time.sleep(1)
th2.start()



@asyncio.coroutine
def handle_client(client):
    request = None
    while request != 'quit':
        request = (yield from loop.sock_recv(client, 255)).decode('utf8')
        response = request + '\n'
        yield from loop.sock_sendall(client, response.encode('utf8'))
    client.close()

@asyncio.coroutine
def run_server():
    while True:
        client, _ = yield from loop.sock_accept(server)
        loop.create_task(handle_client(client))

HOST='192.168.1.241'
PORT=8888
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST,PORT))
server.listen(8)
server.setblocking(False)

loop = asyncio.get_event_loop()
loop.run_until_complete(run_server())
while True:
	print ("Running")
	time.sleep(.05)

So the next stage was convert the "loop = asyncio.get_event_loop(), loop.run_until_complete(run_server())" to either a thread class, insatiate an object and start it, or as here, define it as a function, and start it as a thread

Code: Select all

#astb.py
import asyncio, socket, time,threading

class mythread(threading.Thread):
    def run(self):
        while True:
            print ("In Thread",self.name)
            time.sleep(2)

th1=mythread()
th1.setName("T1")
th1.start()
th2=mythread()
th2.setName("T2")
time.sleep(1)
th2.start()

def rserver():
    loop=asyncio.get_event_loop()
    loop.run_until_complee(run_serer())

@asyncio.coroutine
def handle_client(client):
    request = None
    while request != 'quit':
        request = (yield from loop.sock_recv(client, 255)).decode('utf8')
        response = request + '\n'
        yield from loop.sock_sendall(client, response.encode('utf8'))
    client.close()

@asyncio.coroutine
def run_server():
    while True:
        print("Start Server")
        client, _ = yield from loop.sock_accept(server)
        loop.create_task(handle_client(client))

HOST='192.168.1.241'
PORT=8888
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((HOST,PORT))
server.listen(8)
server.setblocking(False)

#loop = asyncio.get_event_loop()
#loop.run_until_complete(run_server())

ts=threading.Thread(target=rserver)
ts.start()

while True:
	print ("Running")
	time.sleep(.05)


In this case I get:-
"^CException ignored in: <module 'threading' from '/usr/lib/python3.4/threading.py'>
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 1294, in _shutdown
t.join()
File "/usr/lib/python3.4/threading.py", line 1060, in join
self._wait_for_tstate_lock()
File "/usr/lib/python3.4/threading.py", line 1076, in _wait_for_tstate_lock
elif lock.acquire(block, timeout):
KeyboardInterrupt
"

In the case of instantiating a class object, just silence.

How can I run a server as a non blocking task?
It seems there are hundreds of us wishing to do this, but I've not yet found a workable elegant and not overly complicated example for such a simple task.

I'd really welcome some help.
Thanks
Robin

blimpyway
Posts: 148
Joined: Mon Mar 19, 2018 1:18 pm

Re: Run a asyncio socket function a a thread

Thu Jun 27, 2019 10:12 am

I'd use either asyncio or threads to implement the server, not both.

You might also want to use a newer python (3.5.3 on my Pi) since the asyncio api changed significantly.

Also you may consider a higher level API, like mqtt with a broker on PI (e.g. mosquitto) and python & micropython clients on either Pi and ESPs

Since mqtt queues all messages there-s a chance to get the python clients running in polling mode on the message queue(s), without the need of multithreading.

PS by "python clients" above I mean MQTT clients sending/receiving messages from the MQTT broker (==server). From your application POV one of these will be your python server process

PS2 I just found this example/tutorial on using upython mqtt on esp32 with a PI mosquitto broker https://randomnerdtutorials.com/micropy ... 2-esp8266/

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Thu Jun 27, 2019 11:29 am

My thanks Blimpyway, I'm more than grateful.

I'll certainly try this avenue.

Before I do, I wonder if I could ask:-

The critical issue here is 'real time'.
The signals that my ESP32's are interrupted by are time critical:- they're used for calculating speed across the sensors distributed around the multiple esp's.

They'll be time tagged (I'm aware of clock drift on the ESP's), but latency is an issue, with the risk of data becoming stale before new data is triggered.

I'm loath to use loops in any main for polling. I'd prefer a trigger and callback on a message receipt, or as intended, the messages arriving via a thread (one of many).

Is that possible. I just can't countenance a loop hogging the processor, nor latency of more than 250ms.
Thanks
Robin

User avatar
MrYsLab
Posts: 371
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Run a asyncio socket function a a thread

Fri Jun 28, 2019 11:55 am

You may want to look at python-banyan https://mryslab.github.io/python_banyan/# It is being used by Palace Games and there is an article on the RPi blog that mentions it https://www.raspberrypi.org/blog/raspbe ... cape-room/

Banyan requires neither threading or asyncio since it provides concurrency, but you may add threading or asyncio where it makes sense.

If you are staying within the confines of a LAN, MQTT is not as efficient as Banyan and there are comparative benchmarks to illustrate this. https://mryslab.github.io/python_banyan/#benchmark/

Banyan has a demo using the Raspberry Pi and ESP-8266 using the RPi to control and monitor the ESP. The MicroPython script for the ESP should be easily portable to an ESP-32. It uses interrupts to detect GPIO changes.

When the ESP sends a status GPIO status change, Python Banyan time stamps the event. Also, Banyan guarantees that it will receive messages in the order that they were sent.

There are 2 GUIs for the demo, one a tkinter GUI and the other a Web Based GUI. The tkinter gui will work with the standard Python 3 version included with Stretch, and both GUIs work with Python 3.7 included with Buster.

Here are screenshots of GUIs. If the images do not appear in this posting, please use the links:

https://github.com/MrYsLab/python_banya ... b_page.png

https://github.com/MrYsLab/python_banya ... r_demo.png

Image


Image

blimpyway
Posts: 148
Joined: Mon Mar 19, 2018 1:18 pm

Re: Run a asyncio socket function a a thread

Fri Jun 28, 2019 11:56 pm

@Robin
Pi's ability to handle the messages from esp32-s on time depends mostly on:
- how many messages are received per second
- how much processing is needed on the python/Pi app for each message.
- keep in mind that even with multi-threading, python interpreter uses a single core
- and the underlying communication framework indeed. Each one introduces more or less overhead, with mqtt being significantly better than a HTTP message-per-request approach.

You need a means to synchronize time on ESPs. Most obvious is NTP, I do not know how accurate it is. What difference/inaccuracy between ESPs is acceptable?

------------------

@MrYsLab - banyan looks cool, on closer look however:
- it seems it is layered upon ZeroMQ I assume its purpose is to simplify zeroMQ configuration. From OP's perspective he may use Zero MQ instead
- The performance improvement is mostly due to the fact ZMQ lacks a broker and is a lower level/complexity than MQTT
- Their page mentions their intent to provide an multi-language messaging solution for PIs, ESPs and Arduinos, but I found no further info upon whether or where are available client libraries for ESP32

User avatar
MrYsLab
Posts: 371
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Run a asyncio socket function a a thread

Sat Jun 29, 2019 10:57 am

The previous poster is partially correct. It is true, there is no native implementation of ZMQ for MicroPython. Banyan provides a MircoPython script that runs on the ESP and it communicates with a Banyan component called the ESP gateway over TCP/IP. The gateway publishes the information on the Banyan network.

Where @blimpyway is incorrect is that Banyan has its own broker, called the backplane If one runs the Banyan vs MQTT benchmarks, Banyan comes out ahead. The benchmark assumes all traffic is staying within the confines of a LAN, which I am assuming the OP's application does. If the application transmits data beyond the LAN, then MQTT may be the better route. I have not benchmarked things going over the WAN.

Lastly, Banyan does not rely on HTTP. The GUIs I posted are part of a demo. Banyan can be integrated with HTTP if one wishes, but it certainly does not use it for data transport.

blimpyway
Posts: 148
Joined: Mon Mar 19, 2018 1:18 pm

Re: Run a asyncio socket function a a thread

Sat Jun 29, 2019 11:07 am

UDP could work too, with minimum overhead.
You can find examples for both python, asyncio python and esp8266
I tested the basic asyncio UDP listener below with python 3.5 on my pi zero, and it was able to receive 1500messages/sec with a 50% cpu overhead of the receiving python process:

Code: Select all

import asyncio

class DiscoveryProtocol(asyncio.DatagramProtocol):
    def __init__(self):
        super().__init__()
        self.ctr = 0
    def connection_made(self, transport):
        self.transport = transport
    def datagram_received(self, data, addr):
        self.ctr += 1
        if self.ctr % 10000 == 0:
            print(self.ctr)

def start_discovery():
    loop = asyncio.get_event_loop()
    t = loop.create_datagram_endpoint(DiscoveryProtocol,local_addr=('0.0.0.0',5006))
    loop.run_until_complete(t)
    loop.run_forever()

start_discovery()
This was a slight change of original code found here: https://stackoverflow.com/questions/469 ... th-asyncio. All you need to do is to write your own datagram_received() to parse incoming data.

------------
The sender used to test it was plain python:

Code: Select all

import socket
import time

UDP_IP = "127.0.0.1"
UDP_PORT = 5006
MESSAGE = "Hello, World! bla bla bla bla"

print "UDP target IP:", UDP_IP
print "UDP target port:", UDP_PORT
print "message:", MESSAGE

sock = socket.socket(socket.AF_INET, # Internet
             socket.SOCK_DGRAM) # UDP

ctr = 0
start = time.time()
while ctr < 100000:
    ctr += 1
    time.sleep(0.0005)
    sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))


print("Sent {} messages in {} ms".format(ctr, 1000.0 * (time.time() - start)))
inspired from here: https://stackoverflow.com/questions/187 ... udp-packet

----------------------
You should find plenty examples online upon how to send/receive udp packets on ESP

PS the sender since it prints without parantheses, needs to be run with python 2.7, the receiver, since it uses asyncio, needs to be run with python3. Both were running on localhost.

PS2 Sorry it is a Pi 3 A+, not zero. I got confused since its name is "pizerow" cause I reused a Zero's SD card.

blimpyway
Posts: 148
Joined: Mon Mar 19, 2018 1:18 pm

Re: Run a asyncio socket function a a thread

Sat Jun 29, 2019 12:03 pm

@Robin,

In your code examples from your first message,

What locks down the thread (and async tasks are never executed) is using time.sleep() in eg:

Code: Select all

while True:
	print ("Running")
	time.sleep(.05)
Should use asyncio.sleep() instead, that passes control back to the event loop which in turn runs scheduled tasks.

Or loop.run_forever() instead of the whole while loop if you don't need to print "Running".

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Sat Jun 29, 2019 1:10 pm

I'm more than grateful for all the responses, so very cogent replies here.

I've looked at MQQT.

It's blocking.

I'll reply later on the nature of the apps, but essentially as in the OP the esp's do low level interrupt handling around what may be called a track, IR sensor's triggering interrupts at known positions, thereby calculating speed and direction. The interrupts are time stamped. The esps also do ad hoc servo control via PWM but that isn't time critical at all.

The speed and position is fed back to a Pi3, running tkinker. Speed errors are corrected by the PI3 by a custom controller via relays. Not particularly time critical, response 1/2 sec say.

The PI3 also does configuration management, mode select, and monitoring.

So if MQTT is blocking, and I instantiate a thread client and server, will they block all other threads and main?

Looping isn't an option, they've other event responsive things to do

I take it that the MQTT callback loop is useless for this.

I'll keep trying and investigate other options later.

Thanks again
Robin

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Sat Jun 29, 2019 7:25 pm

blimpyway wrote:
Sat Jun 29, 2019 12:03 pm
@Robin,

In your code examples from your first message,

What locks down the thread (and async tasks are never executed) is using time.sleep() in eg:

Code: Select all

while True:
	print ("Running")
	time.sleep(.05)
Should use asyncio.sleep() instead, that passes control back to the event loop which in turn runs scheduled tasks.

Or loop.run_forever() instead of the whole while loop if you don't need to print "Running".
I'm afraid that asyncio.sleep(2) doesn't, that is, sleep
I should use await asyncio.sleep(2). This function isn't available until 3.5
However, the latest release of Python3 for the Pi3 is 3.4.2.
I could upgrade by downloading version 7 and compiling it, but these functions seem quite undefined and non-prediactive, changing from one release to another.

blimpyway
Posts: 148
Joined: Mon Mar 19, 2018 1:18 pm

Re: Run a asyncio socket function a a thread

Mon Jul 01, 2019 8:10 am

What raspbian do you have?

On my Zeros and 3A+

Code: Select all

$ cat /etc/os-release

Shows me
PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"

Python version is 3.5.3

Code: Select all

 $ python3
Python 3.5.3 (default, Sep 27 2018, 17:25:39)
-----------------------------------------------
mqtt is not client and server but publisher and subscriber. The mqtt server known as broker, runs as a separate, non python process - this can be mosquitto or any other implementation. All python mqtt code is clients which send messages on named topics and receive them via registered callback methods.
-------
Client side - python process :
- all clients subscribe to or publish to topics.
- there is indeed a mqtt loop (at least in paho mqtt) which handles sending and receiving of messages. Receiving is handled through callbacks.
- that may conflict with Tk's own loop (cant run both at the same time), yet there are options like..:
- use tkinter's own scheduler to repeatedly execute mqtt loop for short times.
- run either mqtt loop or tkinter's in a separate thread
See examples e.g. https://stackoverflow.com/questions/459 ... event-loop

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Mon Jul 01, 2019 6:38 pm

I'm going to be quiet for a while.
A major rethink is necessary.
My esp32's aren't living up to sales hype of easy comms and low level monitoring/control in real time.
The dilemma is that I've too much invested in the tkinter application on the Pi3, several thousand lines of code.

The esp32's run micropython, which has asynci (usyncio) , mttq, sockets, and threads, but slightly different flavours from mainstream python.

I shall be removing any control from the Pi3 application and divorce it from real time monitoring and control, leaving it to provide superficial monitoring, and setting modes, speeds and configuration.

I shall migrate the closed loop control and monitoring from the Pi3 completely to the esp32s. But only when I'm convinced that the comms and interrupts on the esp32s are harmonious.

But first:-
I shall test the examples on the micropython site by Peter Hinch of running asyncio with MTTQ, but even so, they're inordinately over complex with cluges to overcome limitations. I'm not sure that they'll be fit for purpose with all sorts of caveats and untested statements.

I shall download the latest Python fit for the Pi3, compile it, install and set up its environment. Pain in the tail.
I'll test whether my tkinter will run in this version of Python. I know that it won't. I know I'm in for a horror story. I'll then decide whether it's worth the effort of getting an asynchronous thread running MQQT beside tkinter without further horrors. Unlikely

I'll probably revert to a simple time event triggered loop in tkinter to run a crude MQQT client purely for the top level montoring, mode setting, and configuration, removing any 'real time' responsibility, coupled with a MQQT Broker coordinating the broadcasts and responses to topics.

I'm not sure how many 10's of hours I've expended getting this one feature to operate, and this element is merely one enabling feature. I could revert, chuck the ESP32's away, and revert to the lot (interrupts, control) sitting with the tkinter application and dispense with brokers, blocking comms, lost interrupts (my bane of life in the 1970's) with all this overly complex workarounds for such a simple requirement.

Shame, there are hundreds of posts out there trying to do what I am, and none yet succeeded.

This needs a standard solution.

Failing that I'll revert to my own skills and write a low level driver in machine code. At least then I'm in control and know what's going on.

Despite all of this, I do have to thank you all for expending time and thought on helping.

It is truly appreciated, and such altruism is deeply recognised.

Be sure, I'll be back, and hopefully may help with whatever I discover on the way.


Your Robin

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 4:13 pm

"import asyncmqtttest.py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "asyncmqtttest.py", line 39
SyntaxError: invalid syntax"

It's the
"await client.connect()" in def main below

Why oh for goodness why?

Code: Select all

from machine import Pin
import socket
import sys
import network
import time

internalled=Pin(2,Pin.OUT)
externalled=Pin(4,Pin.OUT)


station=network.WLAN(network.STA_IF)
station.active(True)
station.connect("TP-LINK_EB850C", "arthurfrabb")
while (not (station.isconnected())):
        print("waiting to Connect")
        print (station.ifconfig())
        time.sleep(1)
station.ifconfig()
pin2.on()
time.sleep(1)
pin4.on()
pin2.off()
Time.sleep(1)
print ("Connected")
pin4.off()

from mqtt_as import MQTTClient
from config import config
import uasyncio as asyncio
SERVER='192.168.1.241'

def callback(topic,msg):
        print((topic,msg))

async def conn_hann(client):
        await client.subscribe('testTopic',1)

async def main(client):
        await client.connect()
        n=0
        while True:
                await asyncio.sleep(5)
                print('publish',n)
                #If the wifi is down the following will pause for the duration.
                await client.publish('result','{}'.format(n),qos=1)
                n +=1

config['subs_cb'] = callback
config['connect_coro'] = conn_han
config['server'] = SERVER

MQTTClient.DEBUG = True #Optional: print diagnostic messages
client=MQTTClient(config)
loop=asyncio.get.event_loop()
try:
        loop.run_until_complete(main(client))
finally:
        client.close() #Prevent LmacRxBlk:1 errors





I've followed the following to the letter:-
https://github.com/peterhinch/micropyth ... /README.md

Why oh why.
Have I made a simple typo or hasn't this been validated?

Robin Mosedale
Last edited by RobinMosedale on Fri Jul 19, 2019 7:56 pm, edited 1 time in total.

User avatar
MrYsLab
Posts: 371
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 7:04 pm

You probably should create an issue on GitHub.

If you want to give Python Banyan a stab, let me know, and I could walk you through it. I know for a fact that it works since it was thoroughly tested.

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 8:01 pm

I shall create a fault report.
Banyan?
Perhaps
However this mechanism is the third avenue that I've attempte.
So please do point my to where I might find the correct variant for the esp32 together with a compatible variant for the pi3

mattmiller
Posts: 2099
Joined: Thu Feb 05, 2015 11:25 pm

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 8:09 pm

I've looked at MQQT.

It's blocking.
it can be setup to block or it can be setup to not block

http://www.steves-internet-guide.com/lo ... tt-client/

I added it recently to a python project and just set it to use its loop_start() threaded callback mechanism and the rest of my program carries on without blocking just fine

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 8:32 pm

I know now that it can be set non-blocking.
I'd be grateful for your example

User avatar
MrYsLab
Posts: 371
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Run a asyncio socket function a a thread

Fri Jul 19, 2019 11:52 pm

I forgot you were looking for esp32 support. I would just need to port my esp8266 micropython code to the esp32. Do you need anything beyond simple digital I/O?

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Sat Jul 20, 2019 12:14 pm

An array of 6 esp32's.
One acting as accesspoint
1 rpi3 running a large app in tkinter.
mqtt broker on pi
all will need to rx/tx
interrupts in from esp32's time tagged (I know about the drift)
Speeds calculated, commands tx to control
Some non realtime pwm to control servos
All need compatible comms non blocking

User avatar
MrYsLab
Posts: 371
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Run a asyncio socket function a a thread

Sat Jul 20, 2019 12:47 pm

* An array of 6 esp32's.
This should not be an issue
* One acting as accesspoint
This should not be an issue
* 1 rpi3 running a large app in tkinter.
Not sure of your architecture, but Banyan is easily integrated with tkinter
* mqtt broker on pi
Python Banyan does not use MQTT, but does provide a simple gateway to do protocol conversion between MQTT and Banyan
* all will need to rx/tx
My MicroPython script uses TCP/IP and is configured to run full duplex
* interrupts in from esp32's time tagged (I know about the drift)
Time tagging is a feature of Banyan
* Speeds calculated, commands tx to control
I don't know what you mean here.
* Some non realtime pwm to control servos
PWM should not be an issue
* All need compatible comms non blocking
Banyan is non-blocking

Here is what I propose to do:
1. Update my MicroPython script to support the ESP32 using esp32spiram-20190720-v1.11-167-g331c224e0.bin (latest) http://micropython.org/download#esp32
2. Support both digital i/o and pwm
3. Provide a tkinter demo that will monitor digital inputs using interrupts (with time stamps), control both digital and pwm output.
4. Full instructions on installation and use.

If this is acceptable, it should take me 2 to 3 days to accomplish. You can then test and scale up to your 6 ESPs (I only have one). I will provide support and guidance if necessary.

Just let me know.

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Sat Jul 20, 2019 7:01 pm

That's really most generous MrYsLab.

Hold fire a little while, I'm in conversation with the author, Peter.

There's something odd with my version of uasyncio

Robin

RobinMosedale
Posts: 44
Joined: Tue Jan 29, 2013 10:55 am

Re: Run a asyncio socket function a a thread

Thu Jul 25, 2019 7:41 pm

So versions all scrubbed and the test asyncio mqtt demo programme works

A minor alteration to include a publish function worked.
Good progress
The a trvial asyncio function to switch an external LED.
runs for a few seconds



and then I get :-
"IndexError: queue overflow"

see below

Can't see what I'm doing wrong, anyone advise please?

Thanks
Robin

Code: Select all

# range.py Test of asynchronous mqtt client with clean session False.
# (C) Copyright Peter Hinch 2017-2019.
# Released under the MIT licence.

# Public brokers https://github.com/mqtt/mqtt.github.io/wiki/public_brokers

# This demo is for wireless range tests. If OOR the red LED will light.
# In range the blue LED will pulse for each received message.
# Uses clean sessions to avoid backlog when OOR.

# red LED: ON == WiFi fail
# blue LED pulse == message received
# Publishes connection statistics.

from mqtt_as import MQTTClient, config
from config import wifi_led, blue_led
import uasyncio as asyncio
import time

loop = asyncio.get_event_loop()
outages = 0

async def pulse():  # This demo pulses blue LED each time a subscribed msg arrives.
    blue_led(True)
    await client.publish('foo_topic', 'Message received')
    await asyncio.sleep(1)
    blue_led(False)

def sub_cb(topic, msg):
    print((topic, msg))
    loop.create_task(pulse())

async def wifi_han(state):
    global outages
    wifi_led(not state)  # Light LED when WiFi down
    if state:
        print('We are connected to broker.')
    else:
        outages += 1
        print('WiFi or broker is down.')
    await asyncio.sleep(1)

async def conn_han(client):
    await client.subscribe('foo_topic', 1)

async def main(client):
    try:
        await client.connect()
    except OSError:
        print('Connection failed.')
        return
    n = 0
    while True:
        await asyncio.sleep(5)
        print('publish', n)
        # If WiFi is down the following will pause for the duration.
        await client.publish('result', '{} repubs: {} outages: {}'.format(n, client.REPUB_COUNT, outages), qos = 1)
        n += 1

import machine

#pin2=machine.Pin(2,machine.Pin.OUT) #the inernal blue led
pin4=machine.Pin(4,machine.Pin.OUT) #externally attached led

async def testfunction ():
    while True:
        print ('LED On')
        pin4.on()
        time.sleep(1)
        print('LED Off')
        pin4.off
        await asyncio.sleep(1)
        print('Out')
    
# Define configuration
config['subs_cb'] = sub_cb
config['wifi_coro'] = wifi_han
config['will'] = ('result', 'Goodbye cruel world!', False, 0)
config['connect_coro'] = conn_han
config['keepalive'] = 120

# Set up client
MQTTClient.DEBUG = True  # Optional
client = MQTTClient(config)
loop.create_task(testfunction())
try:
    loop.run_until_complete(main(client))
    loop.run_until_complete(testfunction())
finally:  # Prevent LmacRxBlk:1 errors.
    client.close()
    blue_led(True)

results in:-

MicroPython v1.11-167-g331c224e0 on 2019-07-22; ESP32 module with ESP32
Type "help()" for more information.
>>>
>>> import rangec
LED On
LED Off
Checking WiFi integrity.
Out
LED On
LED Off
Out
LED On
LED Off
Out
LED On
LED Off
Out
LED On
LED Off
Out
LED On
LED Off
Got reliable connection
Connecting to broker.
Connected to broker.
We are connected to broker.
(b'foo_topic', b'first boring message')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
Out
LED On
LED Off
(b'foo_topic', b'Message received')
(b'foo_topic', b'Another boring Message')
Out
LED On
LED Off
(b'foo_topic', b'first boring message')
(b'foo_topic', b'Another boring Message')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'first boring message')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
(b'foo_topic', b'Message received')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "rangec.py", line 93, in <module>
File "rangec.py", line 88, in <module>
File "/lib/uasyncio/core.py", line 180, in run_until_complete
File "/lib/uasyncio/core.py", line 159, in run_forever
File "/lib/uasyncio/core.py", line 58, in call_later_ms
File "/lib/uasyncio/core.py", line 63, in call_at_
IndexError: queue overflow
>>>

Return to “Python”