reggie4
Posts: 81
Joined: Wed Jan 15, 2014 2:09 pm

Sharing a numpy array between 2 running Python scripts

Fri Jun 05, 2015 11:31 am

I have two opencv2/Python scripts running at the same time on the same Raspberry PI.

1) A streaming server script based on https://github.com/berak/opencv_smallfr ... g_serve.py that outputs a live camera feed to the web
2) A motion detection script, that detects movement from a camera

Only one of these scripts can access the NOIR camera board at once, so I need a way of passing/sharing each frame gathered to the other script.

They both grab the frame from the NOIR camera board using opencv2's

Code: Select all

rc,img = capture.read()
; so i'll have to remove that from one of them and somehow pass the img to the other script when rc is true.

As the camera frame rate is substantial (24f/s), I don't think it's feasible to save the img (numpy array) to a file on a disc that they can both access; but I may be wrong!

I've been scratching my head and searching the net for examples on scripts that pass data, but alas i'm stuck.

How could I share a numpy array (approximately a megabyte of data) between two running python programs on the same Raspberry PI, and could someone kindly provide me some example code?

User avatar
paddyg
Posts: 2587
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Sharing a numpy array between 2 running Python scripts

Fri Jun 05, 2015 6:41 pm

I don't know how up-to-speed you are with numpy and multiprocessing but I think you can do something like this using numpy ctypes so long as you start the second process from the first using multiprocessing. There is a facility in that for defining shared memory.

PS after a quick search here's something somebody put up doing this kind of thing. http://thousandfold.net/cz/2014/05/01/s ... nd-ctypes/ Interested to hear how you get on!
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

reggie4
Posts: 81
Joined: Wed Jan 15, 2014 2:09 pm

Re: Sharing a numpy array between 2 running Python scripts

Mon Jun 08, 2015 10:53 am

Thanks paddyg for your advice. It's going to take me some time to digest your suggestions, because, i'm a multiprocessing novice! Am I correct that it's probably not feasible to store the numpy array on the PI's sd card 24 times a second? So storing it in the PIs ram (somehow) is the best approach? Also how secure it this method? I'm hoping to pasword protect the streeming server. If i store an image in the PI's memory will it be secure to other processes/ users?

User avatar
paddyg
Posts: 2587
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Sharing a numpy array between 2 running Python scripts

Mon Jun 08, 2015 4:38 pm

I think reading and writing to SD 24 time per second is not a good idea. It might be feasible to write to a ramdrive http://www.domoticz.com/wiki/Setting_up ... spberry_Pi and that might be simpler than the multiprocessing route as I imagine that you want to avoid reading while another process is writing and this might be controlled automatically by the os.

Would it not be easier to combine the two scripts into one? You could make the motion detection script a module and import it into the streaming script if you want to keep it modular and allow it to be used elsewhere.
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

reggie4
Posts: 81
Joined: Wed Jan 15, 2014 2:09 pm

Re: Sharing a numpy array between 2 running Python scripts

Wed Jun 17, 2015 11:04 am

Sorry Paddyg for the delay in responding. Iv'e tried to merge the scripts, by placing the motion script code in a function, and calling it around line 21 of the streaming server code.

But the streaming server script waits for a get request, prior to it starting the motion function; which consequently only starts motion detection when someone accesses the live stream.

This is a problem as I want motion detection and streaming server to run independently of whether someone accesses the stream or not.

I cannot figure out a way to fool the server script that someone has accessed the stream; hence a different approach.

Both the scripts use opencv2

Code: Select all

some_variable = capture.read()
to capture a frame from the NOIR Camera board, so they cannot run together.

So I either:
1) find a way to fool the server script into thinking someone has accessed the stream
2) try and have one script capture the frame and store it for, or share it with, the other script (ram drive looks promising)
3) ask clever people than me for approaches, because i'm well and truly stuck!

User avatar
paddyg
Posts: 2587
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Sharing a numpy array between 2 running Python scripts

Wed Jun 17, 2015 12:28 pm

It's rather hard to suggest anything very specific without actual code (do you have an online repository, or could you post code?) but assuming it's similar to the link in your first post, I think I would try:

- Have a function running in another thread with a while loop making it stay there for ever!
- In that thread/function have capture,rc,img as global variables (I know, not ideal but you've already got capture as a global) and have the image capture here only
- Also do the movement detection in this function.
- Start the thread before starting the server
- In the server do_GET miss out the line with capture.read() relying on picking them up from the global variables.

I think you will have to use two threads as you need two loops running
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d

reggie4
Posts: 81
Joined: Wed Jan 15, 2014 2:09 pm

Re: Sharing a numpy array between 2 running Python scripts

Thu Jun 18, 2015 4:47 pm

Woo Hoo, that approach worked!

Thankyou very much for your advice ;)

User avatar
paddyg
Posts: 2587
Joined: Sat Jan 28, 2012 11:57 am
Location: UK

Re: Sharing a numpy array between 2 running Python scripts

Thu Jun 18, 2015 6:08 pm

Well done. Looks like I will have to roll my sleeves up and have a go at shared memory in multiprocessing myself. Someday...
also https://groups.google.com/forum/?hl=en-GB&fromgroups=#!forum/pi3d


Return to “Python”