theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

how to set up multitasking

Tue Jul 18, 2017 11:27 pm

I have a small automation project using python. I have some pushbuttons connected to the digital inputs running some code. I also have a temperature sensor connected to the 1-wire bus running some other code. The program runs at a fixed cycle time of 1s using the pygame clock.tick() function. It all works but the pushbuttons are not responsive enough. When I set the cycle time to 100ms it actually runs at a little over 800ms. Commenting out the code that reads the temperature sensor allows it to run at 100ms so the read function is the problem. I actually don't need to read the temperature every 100ms. 5s would be fine. I came up with a scheme of using nested loops to run some code at 100ms and some at 5s but realised it wouldn't work. The 100ms code would go to sleep for 800ms every 5s while the sensor was being read.
I really need to set up some sort of multitasking so the 2 sets of code run independently.
Some hints on what to research and how to get started would be appreciated.

ghans
Posts: 7362
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: how to set up multitasking

Wed Jul 19, 2017 5:18 am

You could either use the "threading" or the "multitasking" module.

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

Re: how to set up multitasking

Wed Jul 19, 2017 6:17 am

Thanks ghans. I have had a quick search on your suggestions and they look like what I'm after. Let's see if I can get something to work.

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Wed Jul 19, 2017 7:07 am

Threading should work, there was example of the above in a recent issue magpi.
But look at Datalogging with the sensehat.

I use the same thing for DS18's 1-wire in my Datalogging SenseHat program (just not worked out a nice way to add it to the CSV file)
Worksheet2 near the end iirc.
https://www.raspberrypi.org/learning/se ... worksheet/


The DS18B20s take about 750ms to give their temperature by default.

scotty101
Posts: 2295
Joined: Fri Jun 08, 2012 6:03 pm

Re: how to set up multitasking

Wed Jul 19, 2017 8:21 am

Why not just increase the rate at which your main loop executes?

Say every 100ms and if there are tasks that only need to happen once a second, you just use a counter to only enable them every 10th time.

People have been creating systems like this for much longer than there has been multithreading/tasking.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Wed Jul 19, 2017 10:19 am

scotty101 wrote:Why not just increase the rate at which your main loop executes?

Say every 100ms and if there are tasks that only need to happen once a second, you just use a counter to only enable them every 10th time.

People have been creating systems like this for much longer than there has been multithreading/tasking.
There would still be a 750ms block in the code where it would wait for feedback from the sensor?

theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

Re: how to set up multitasking

Fri Jul 21, 2017 8:29 am

Got it 99% working by using threading. Only 1 outstanding issue. My main loop uses a pygame event to look for the Esc key to stop the program. This is in the same loop that reads the digital inputs. The digital inputs work correctly but the Esc key no longer stops the program and I have to power down the pi to get out. Probably fine for the finished product but hopeless while I'm debugging.
I think the problem is that my thread is still running. How do you stop it?

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Fri Jul 21, 2017 8:34 am

Might not be able to help, but stick the code up and see if anyone can see the problem (does CTRL+C not break it eventually)
(don't forget the 'code' tags)

theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

Re: how to set up multitasking

Fri Sep 15, 2017 3:11 am

Thought I'd already given an update a month or 2 ago but can't see it anymore. I fixed the esc key not working by setting the thread as a daemon. This allows the main program to shut down even while the threads are still running.

Anyhow my project has developed further. My program uses an Adafruit TFT as a user interface. It is set up in pygame. I want to use tornado as a web server to show live data from my application on a web browser. I got it to run separately but when I integrate it into my program, the web server part takes over and the pygame part doesn't display anything any more. I thought I would be clever and use my new found skills to run the tornado code in a separate thread. Doing this restores the pygame code but the tornado code stops working. There are no error messages.

My question is are there any limitations that anybody knows about running pygame and tornado together or for running tornado in a separate thread? Sorry about not including and sample code. It is quite long and spread over several modules.

User avatar
OutoftheBOTS
Posts: 342
Joined: Tue Aug 01, 2017 10:06 am

Re: how to set up multitasking

Fri Sep 15, 2017 3:22 am

You can either manually build your own threading function (you will need to deal with button debounce issues) see https://www.tutorialspoint.com/python/p ... eading.htm

Or you can use the builtin threading routines for GPIO (known as call backs) see https://sourceforge.net/p/raspberry-gpi ... ki/Inputs/

theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

Re: how to set up multitasking

Fri Sep 15, 2017 3:47 am

You obviously haven't read the posts. I have got threading working on all of my I/O's etc. I just doesn't want to play with the tornado library.

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Fri Sep 15, 2017 6:43 am

As for code showing.
People (incl RaspberryPi) use github.
Have a look at putting it up there, it also means you have a backup.

User avatar
OutoftheBOTS
Posts: 342
Joined: Tue Aug 01, 2017 10:06 am

Re: how to set up multitasking

Fri Sep 15, 2017 10:18 am

Ok I have read your post a little closer and you want to be able to run to bits of code side by side in parallel rather than code in series on a thread that blocks the other threads when it runs.

I have done this before using the multiprocessing library see https://docs.python.org/2/library/multiprocessing.html

I found doing multi processing can open up a can of worms and I personally have had hit and miss with it. The main problem with multiprocessing over threading is threading only runs 1 thread at a time so you never end up with 2 threads writing or reading from the same bit of memory at the same time. Once you get multiprocessing running different processes on different cores at the same time (in parallel) you can run in the problem of memory corruption when 2 processes access the same bit of memory at the same time. I found sometimes the program ran fine then at other times at random the program can crash, google "multiprocessing race conditions" for more info on this.

Python isn't the best language for multiprocessing as it is a high level language and there is a lot happening under the hood that isn't seen to the programmer that can conflict with other processors on different cores.

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Fri Sep 15, 2017 10:48 am

I notice you reference python2 there.
Python3 (now 3.5 on Raspbian) should be better by now. 3.6 iirc has brought more improvements.
(All from just reading, not actual use ;-))

User avatar
OutoftheBOTS
Posts: 342
Joined: Tue Aug 01, 2017 10:06 am

Re: how to set up multitasking

Fri Sep 15, 2017 2:57 pm

Ha Ha Ha. I am an old schooler. I still use python 2.7.

Either way python isn't the best language to be doing multiprocessing in.

My little dip into multiprocessing was with a computer vision project I was doing using OpenCV with of course Numpy arrays. I was timing how long my robot took to do a task and also counted how many image frames it processed the divided the frames by the time take to get my fps. I was playing around with what sort of processing I was doing in order to get a higher fps. I of the things that I tried was multiprocessing to split it up and speed up things but it made it slower not faster and although it ran fine ost of the time at random it could crash because of race conditions. What I discovered was that OpenCV and Numpy array are already highly optimized and was already split up into multiprocessing on all 4 cores and when I tried to also put thing on other cores it slowed down what was already optimized in a low level language like C written by super smart programmers. I think tornado might be the same that it has been written in a low level language and already operates outside the GIL and OS.

theegg
Posts: 6
Joined: Tue Jul 18, 2017 11:12 pm

Re: how to set up multitasking

Sat Sep 16, 2017 6:19 am

I'm pretty sure that the problem relates to the line

tornado.ioloop.IOLoop.instance().start()

The tornado documentation and the code I cut and pasted from say this line should be at the end of the main section of the program. This plus the name of the command implies to me that this starts off a control loop internally somewhere. In my case the main section needs to run as a loop scanning I/O's and running the TFT display amongst other things. My theory is that the program is stuck running the tornado command and hence stops executing my main loop. It does however set up the webpage which I can view with a web browser so the tornado code works.

My solution was to run the tornado code in a separate thread. When I did this the rest of my code started running again so my theory is at least plausible. My current issue is that while my separate tornado thread doesn't spit out any errors, it also does't spit out any web page. I'm thinking tornado doesn't like being shunted off into another thread but don't understand enough to check for this. Is there any method of debugging a separate thread in python?

User avatar
OutoftheBOTS
Posts: 342
Joined: Tue Aug 01, 2017 10:06 am

Re: how to set up multitasking

Sat Sep 16, 2017 9:31 am

Ok I haven't used tornado and have learnt theses sorts of problems r common with threading and multiprocessing so I try to aviod using it when it possible to code without using them.

It does seem to be a blocking problem. The first way you had it Tornado was blocking your main thread and the second way you have it your main thread is blocking the tornado.

Have you read this page from tornado docs http://www.tornadoweb.org/en/stable/gui ... l#blocking

Hoagie
Posts: 118
Joined: Sun Sep 23, 2012 7:19 pm

Re: how to set up multitasking

Sat Sep 16, 2017 5:42 pm

I skip read the last few posts after reading that it wouldn't shut down anymore. This sounds like you have left the other thread running. I don't know how you can send values between threads in Python, but if your main one can set a flag in the background thread it can use this flag to exit its own loop and allow the shutdown. There are usually ways to make the main thread wait until the child threads have exited too.

bensimmo
Posts: 1740
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: how to set up multitasking

Sat Sep 16, 2017 5:59 pm


Return to “Automation, sensing and robotics”

Who is online

Users browsing this forum: raissahbibi and 11 guests