PiTommi
Posts: 25
Joined: Tue Sep 03, 2019 3:45 pm

GPIO and subprocess module Python3

Thu Jan 30, 2020 2:02 am

Hi guys!

So my main goal is to try to connect native RPi terminal commands to GPIO responses like led light, oled screen etc. My go to module in this is subprocess. I tried this out but a couple of problems popped up. First, the strategy to intercept the output is through:

Code: Select all

proc=subprocess.check_output()
First, I tried a couple of commands, but its seems that not all commands outputs (nothing in terminal but looks like it's waiting) like "netstat -c" or "top". I confirmed that if using:

Code: Select all

proc=subprocess.call()
then it outputs in terminal but obviously I can't do anything with this since it´s not saved.

Second, those commands that do work (don't remember which) using the first code are continously outputting data but since they never close the code can't continue and can't be processed.

I guess my question is how would you do this? Maybe my strategy is not optimal. For example say if you are using top command and want to monitor a particular PID either comes up or disappears during continous monitoring then a GPIO led lights up/lights out.

For the subprocess problem, is there other subprocess functions to use to get the output working for those that doesn't store output with the first code? Lastly for the continous "monitoring" commands what should I do to be able to process the data? Maybe output to txt or tmp file?

Hope it makes sense :)

Thanks

Aydan
Posts: 710
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: GPIO and subprocess module Python3

Thu Jan 30, 2020 1:40 pm

Check_output will only return after the command has finished.

For interactive processes and/or processes that have a lot of output use subprocess.Popen with stdin, stderr and stdout set to PIPE.
Then you can use the pipes to communicate with the process.

ATTENTION: the process will block when the STDOUT pipe has filled the buffer, so do not forget to read periodically from it.

Regards
Aydan

PiTommi
Posts: 25
Joined: Tue Sep 03, 2019 3:45 pm

Re: GPIO and subprocess module Python3

Tue Feb 11, 2020 1:47 am

Thanks Ayden!

Your advice helped a little bit. So this is an working example of what I did:

Code: Select all

import subprocess as sub
from subprocess import PIPE

proc=sub.Popen(['netstat' ,'-c'], stdin=PIPE, stdout=PIPE, stderr=PIPE, shell=True)
print (proc.stdout.read().decode('utf-8'))
So running the above will yield identical output as if run from terminal and it can also be interacted with by regex etc.

However, I'm not sure if I really understand the extent of your warning. When the data is read is the buffer resetted? Can you control the buffer?
Can you also control how long time the data is being read by subprocess?

The other thing is that I tried above code together with "top" that is also dynamic but with no luck. In the terminal I just get a blank line. I also tried other ones as well but it just get stuck without proceeding, it doesn't take the snapshot like for the "netstat -c" example. Tapping into native processes doesn't seem that straight forward :(

Any Idea of what I'm missing?

Aydan
Posts: 710
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: GPIO and subprocess module Python3

Tue Feb 11, 2020 9:34 pm

Code: Select all

>>> s=subprocess.Popen('/usr/bin/top',stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>>> s.stdout.read()
b'top: failed tty get\n'
'top' needs a tty, not a pipe, I'd use python3-psutil instead.

Rearding reading from the pipe, the pipe has a buffer. Once the buffer is full, the process writing into the pipe will stall. If you have a long running process, or a process which outputs more than a few KB of data, then you have to read the stdout multiple times and/or periodically until the process is finished.

Regards
Aydan

PiTommi
Posts: 25
Joined: Tue Sep 03, 2019 3:45 pm

Re: GPIO and subprocess module Python3

Fri Feb 14, 2020 1:38 am

Ok, so I would like to clarify that I thought initially that subprocess could just tap into native processes which is kind of what I wanted to map out.
However, it seem now that subprocesses is very limited in this regard. I'm aware of psutil but the point was to explore capabilites of subprocess for an easier way to interact with GPIO. How do I know if a command in terminal is going to be able to be run and interacted with in subprocess or not, if I get alot of 'b' ' ' lines? I have not come across the tty thing so I don't understand it. Can this be implemented with subprocess? Command 'top' itself is not the important thing but the connection between python (GPIO) and native raspbian processes.

Aydan
Posts: 710
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: GPIO and subprocess module Python3

Fri Feb 14, 2020 6:50 am

Code: Select all

b''
just means it's a bytes object, not a string. Bytes-Object means raw byte data, instead of character representation like in a string. Think of e.g. greek characters. bytes will show the raw codes (e.g. UTF-8) while string will show the actual characters when you 'print' them. you can convert one into the other with b'xxx'.decode() and 'xxx'.encode() an give it the appropriate encoding format as an argument. Default is ASCII.
It would help to know what exactly you want to achieve. If you want to send keycodes to an application, then it might be better to actually emulate keystrokes via python-evdev.

Regards
Aydan

User avatar
HermannSW
Posts: 2007
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: GPIO and subprocess module Python3

Fri Feb 14, 2020 8:24 am

Aydan wrote:
Tue Feb 11, 2020 9:34 pm

Code: Select all

>>> s=subprocess.Popen('/usr/bin/top',stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
>>> s.stdout.read()
b'top: failed tty get\n'
'top' needs a tty, not a pipe,
I did run in similar issue with "stty" with simple solution -- just give it tty as stdin!
https://lb.raspberrypi.org/forums/viewt ... 7#p1602021

Code: Select all

🍓 cat size.py 
import subprocess
subprocess.call(["stty", "size"], stdin=open("/dev/tty", "r"))
🍓 
🍓 echo foobar | python3 size.py 
24 80
🍓 
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

PiTommi
Posts: 25
Joined: Tue Sep 03, 2019 3:45 pm

Re: GPIO and subprocess module Python3

Fri Feb 14, 2020 5:06 pm

Thanks for the inputs!

I suspected the "b' ' " to be related to bytesize but I already tweaked the code for decoding without success it doesn't output anything. Either the terminal finish with blank line or gets stuck. I think I have seen a code somewhere with sending keystrokes using subprocess but what does that help if the original command doesn't work.
It would help to know what exactly you want to achieve.
I'm more interested in monitoring of processes and say in case something pops up or exceeds then the GPIO will output say to the LCD or lit a lamp. Just basics, connecting stuff together :D

I tried your code HermannSW and it didn't seem to work for interactive commands i.e check_output, Popen maybe it would be good for just "calls" that are more "stubborn" :)

Maybe I should ask a better question. What would your advice be to have a more stable interaction in what I want to achieve? Output to a txt or csv file from the native commands and feed it to a python code opening it say with csv module. The other example would be just straight up getting better at bash codes I guess. At this point I really prefer python though.

Return to “Python”