ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

reading file from rpi1 with rpi2

Wed Nov 04, 2015 4:54 am

Ok, so I've spent a good week trying to solve this problem. I've looked at samba, NIS... just about everything and I haven't found something that I can tell will work. Heres the situation....

I have one RPi with the official touchscreen on it. (RPI1)
I have a second RPi with nothing, as of now, going to put a 10.1 screen used as a HUD. (RPI2)

Both RPis will be connected with Ethernet to a switch, there will most likely be several when this is all said and done. I want RPI1 to be able to read/write from RPI2 (and possibly vice versa, but is optional) RPI2 would have variable files that are shared between all RPis connected to the switch.

How the code looks (very basic)
###var_store.py###
var1 = x
var2 = y
var3 = z
################

###var_read_write###
import var_store
print var_store.var1
var_store.var1 = a
print var_store.var1
#################

which would output something like
>>>
x
a
>>>


The var_store.py file would be kept on the 'master' pi (RPI2) and read from by RPI1. This is where I am stuck. I need a method that will allow RPI1 to see values on RPI2 and be able to modify them.

Any help is greatly appreciated as I don't know much about linux....

User avatar
rpdom
Posts: 17206
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: reading file from rpi1 with rpi2

Wed Nov 04, 2015 5:00 am

There's lots of ways you can do this, but your description isn't very clear.

You could have a shared mount using NFS so files in a certain area on Pi A can be read and written by Pi B, and write the values to those files.

You could look at using network sockets to send a value from a client (Pi A) to a server (Pi B).

That's just two for a start.

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 1:12 am

So I'll try and explain it a little better. I want 'global' variables that I can transfer from one pi to another. I'm looking for quick look-up speed, wanting to make a 'refresh' code that checks the status of the values every half second. I hope thats a better explanation. I'd be more than willing to answer questions :) I'm so grateful for the help!

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 2:41 am

ritter4242 wrote: The var_store.py file would be kept on the 'master' pi (RPI2) and read from by RPI1. This is where I am stuck. I need a method that will allow RPI1 to see values on RPI2 and be able to modify them.
I would think you could just store the values in a file in a directory that was on the tmpfs. You could mount the directory using sshfs or you could using scp, which will be faster if you keep the ssh connection open.

But the issue as you grow this is really going to be how to deal with the parallel nature of the access. Are you going to use locks? If you use more than one, you can have problems. Since you are not really saying what you are doing, it might be that you can just assume that all data is current and not worry about some of these issues. When I used this method to test security cameras, I just assumed that everyone could run updating at some speed, say every second, and the system looking for the images could also look every second. If it couldn't read the file, because it was being written, just use the old version. That works for a security camera test that is rapidly updated and can deal with the loss of an image every once in a while. That might not work for your application.

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 3:58 am

My code will be updating the state of relays, a display that shows various states, and possibly writing to OBD2. I would like some sort of 'master' variable file so that at any given instance the variables being used in the code are all the same (that way the display doesn't show relay n being off when in actuality it is on). Programs will write the values to this file and then re-read them. Variable are set three ways, logical switches (if these sets of conditions are met turn something on), physical switch/tkinter, and finally a telegram bot. When I use Tkinter I have to run two programs, otherwise the system 'jams' (yes I can use threading, and I do so so I can run update programs in the background to make sure that everything is up to date.

The reason I have two raspberry pis is the 'Official touch screen' only allows one screen (locks the HDMI) at the current time, so one raspberry pi will have a tkinter program running on the touch screen, the other will have the variable display, 10.1 inch hdmi. I was planning on using the second raspberry pi for the master file storage since it wont be particularly busy. As I add features I will most likely take my older model A and use it for the master file storage. (these will all be connected with ethernet, not wifi so its more secure)

How I'm getting around the read write lock issues is I'm importing the files:
##variable_file.py##
var_1 = 'Off'
###############

##code##
import variable_file
-or-
from variable_file import *
~
~
########


so the variables will be referenced with variable_file.var_1 or in the later case just var_1


Does that clear it up? I'm wanting a way to import a file off another pi (which I'm assuming works the same way any other file access does)

Some other ways I've considered, but I'm really wanting to have 1 master variable file, use I2C or some sort of serial. Each pi would have a copy of the master file and a program would listen for a new list of variable (or just the ones that have been changed) and update the local master file as needed.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 4:43 am

ritter4242 wrote:Programs will write the values to this file and then re-read them.
How do the programs know that they can open the file for writing, I mean if more than one program is writing to a single file? Are they going to try, fail, wait, try again, wait, finally succeed? If you have a file for each variable, then you might be able to get around this issue, unless more than one program is changing each variable.
I'm wanting a way to import a file off another pi (which I'm assuming works the same way any other file access does)
I suggested sshfs or scp holding the ssh connection open. With sshfs, you would just have a directory on the pi that you want to read the file from but it would physically be on the other pi. It's just mounting it through ssh using fuse.

I'm sceptical that this is enough for what you want to do, I suspect that ultimately you'll want to use message passing using the actor model or something like that. But you can probably gain some insight into your situation by setting this up using sshfs or scp or something from within your programming language itself.

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 11:39 am

Exchanging the values via a file with NFS is doable. I'm doing something similar using Perl and shell scripts for example to write the CPU-temperature of all my Pi's and the relay status of my PiFace module. Reading example in Perl:

Code: Select all

open FILE, "/usb1/rpi/temp-raspicam.txt" or die "Couldn't open file: $!";
  $rpi02 = <FILE>;
close FILE;
The only challenge is the timing as it's bad pactice to constantly read/write a file so often.
At least this will severly reduce the life expectancy of the SD card. You should really look at ways to make this smarter i.e. that you only write when something changes and only read when necessary.
If you really need something that fast, you should look at creating a ramdisk (don't know if that's available as a package, never needed it)

/emgi

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 4:51 pm

ok, I think I've realized now why I think I've been explaining what I'm trying to do, but failing...

I want to be able to import a file into python, stored on PiA, into a file on PiB. This gets around the whole read/write try/fail issue. I've done this on one pi before and it doesn't cause any issues as its referencing the value, not opening and closing the file. Now if that can be done with SSH then cool! but I know so little about it making an attempt at it seems insane. I'm not entirely sure how the import process works anyway...

if the method above (importing the files directly from PiA into PiB) doesn't work. I will need to set up some sort of communication system between PiA and PiB so A can tell B var_1 has changed. I would like another program that will send and array or a binary string (for checking purposes).

What I know:
I can import and reference variables stored in a file from more than one python script.

what I don't know:
How to import a file across a network.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 5:37 pm

ritter4242 wrote:I can import and reference variables stored in a file from more than one python script.
Of course you can, but you seem to be saying is that you are going to have various scripts *changing* the file at the same time. If it's just a text file, how can you open it for writing from pi A and pi B at the same time? If that's not going to happen, only one script will open the file to write to it, then you still have to deal with the fact that if you are rewriting it completely each time, it will not be there when it is being rewritten. What if the reader pi is trying to open it for reading then?

This problem absolutely happened to me in testing my security cameras. I found that by delaying the writes and the reads by about one second on both systems, I got much better throughput than just letting everything run at full speed. I think this is because less time was spent with the file in a state where it couldn't be read. But you have been saying that you can't really afford to drop access to the current true value of your remote variables. With my security system, I just used the old picture until a new one came through the next second.
How to import a file across a network.
I suggested using sshfs to mount the directory locally so you could just write to it. This seems like the most trivial way to at least start. if you use scp, you would write to a file and then copy it to the other pi, e.g. scp file bobby@whatever:/tmp and then read from your local /tmp from your python script. Note that /tmp is not on tmpfs on the default pi installation so you may wish to reduce your writes to flash by using some place that is, see what df says on your system. scp will prompt for your password so if you use it in production, you'll want to make it use some other means of authentication. A dedicated non-sudo account seems sensible to me.

If you've got various things collecting information that then get all appended to the variable file and this file is then read by various scripts, it seems like you are going to have to move to some sort of message passing mechanism.

Regarding getting sshfs working, the line I usually use at this point is:

sshfs -o reconnect -o follow_symlinks -o allow_other user@whatever localDirectory

If it gets hung up, you might have to run sudo umount -l localDirectory

It has a config file in /etc that you can use to allow any user to mount with it.

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 5:54 pm

ritter4242 wrote:ok, I think I've realized now why I think I've been explaining what I'm trying to do, but failing...

I want to be able to import a file into python, stored on PiA, into a file on PiB. This gets around the whole read/write try/fail issue. I've done this on one pi before and it doesn't cause any issues as its referencing the value, not opening and closing the file. Now if that can be done with SSH then cool! but I know so little about it making an attempt at it seems insane. I'm not entirely sure how the import process works anyway...

if the method above (importing the files directly from PiA into PiB) doesn't work. I will need to set up some sort of communication system between PiA and PiB so A can tell B var_1 has changed. I would like another program that will send and array or a binary string (for checking purposes).

What I know:
I can import and reference variables stored in a file from more than one python script.

what I don't know:
How to import a file across a network.
Exactly. And the communication system could be nfs.
Simply store the file on an nfs share and read it from the other Pi.

/emgi

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 6:00 pm

sorry stderr, I'm not writing the file, I'm changing the variables value.

##var.py##
var_1 = 'off'
########

>>>import var
>>>print var.var_1
off
>>>var.var_1 = 'on'
>>>print var.var_1
on


That ^ is what I'm trying to do, but with var.py being on a different raspberry pi (sorry I realized I'd been using the wrong term, read/write, instead of reference and overwrite) Once the master variable file is created the file itself wont be changed. It wont be opened or re-written. I will only be reading values (by import) and possibly changing them depending on which module is using the master variable file.

Again I apologize for not explaining what I'm trying to do (import a file across a network) and am always grateful for your help :)


emgi, will that (nfs) allow me to import that file? sorry I know very little about networks and the such

User avatar
emgi
Posts: 357
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 6:49 pm

If I have to explain it in nono terms, nfs is the samba of linux and any other .ux.
What it does is allow sharing files between machines, which seems to be what you are looking for.
The pi which writes the file should ideally host the nfs share.
If you have a basic understanding of linux file-system security you will have little problems setting this up.
No problem if you don't know where to start, there is a lot of info on this forum or via google.

/emgi

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 8:19 pm

ritter4242 wrote:sorry stderr, I'm not writing the file, I'm changing the variables value.
You kept referring to files, so I thought that is the way you wanted to do this. It now seems to me like you want to change a var on one pi in its instance of python and it will be changed on the other pi in its instance of python. The first problem with doing that is that you have two different pythons running.

I think you are talking about the message passing thing I was saying you'd probably need to get this to really work in the real world. I found the following which I think is the kind of thing you were trying to get at: https://www.safaribooksonline.com/libra ... 11s07.html .

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 8:27 pm

ritter4242 wrote:sorry stderr, I'm not writing the file, I'm changing the variables value.
##var.py##
var_1 = 'off'
########
>>>import var
>>>print var.var_1
off
>>>var.var_1 = 'on'
>>>print var.var_1
on
In just looking at this some more, aren't you actually changing what the values are in the var.py file as your system monitors the value changes on the other pi? I don't know how you are changing those values but it's like you want to run a loop that keeps importing the var.py file which would update the value in the script you are running. But how is the var.py file itself being updated?

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Fri Nov 06, 2015 10:46 pm

emgi, I'll look into that and see if it'll work!

stderr, once I've imported the file var.py, it is not modified anymore. what the pointer from the import statements is looking at does. (sorry I took a little bit of computer science a LONG time ago and I'm trying to remember how to explain this). So the var.py file sets up all the variables and then gives them their initial value ('Off') and from that point on its only used to give other scripts access to the values (what var_1) is pointing at, its value is changed. The values aren't changed in the file, they are only changed in memory (that way when I turn everything on its in the sate I'm expecting). then the memory references are changed, hopefully that makes sense. This allows one script of python to use values from var.py that are being actively changed by another without having the read write issue. (or at least my testing so far has shown that this is an option.) when I get home this evening I'll try and set something up so I can show you what I'm doing. So if I can import the file from a second pi, it would then be referencing values in that pi's memory. I've attached some of the code I used to test my theory *it seems you can't attach .py code... so its below*. its been a busy week unfortunately /:
################
vari.py
################

#light states
#set with vari.l1 = 'On' or 'Off'
#get with vari.l1

l1 = 'Off'
l1s = 'Off'
l2 = 'Off'
l2s = 'Off'
l3 = 'Off'
l3s = 'Off'
l4 = 'Off'
l4s = 'Off'



####################
main.py
####################
import threading
import Tkinter
import time
import vari


def l_update():
arr = [vari.l1,vari.l2,vari.l3,vari.l4]
arr_s = [vari.l1s,vari.l2s,vari.l3s,vari.l4s]
print arr
print arr_s
i = 0
while i < len(arr):
if arr!= arr_s:
arr_s = arr
i+=1
print arr
print arr_s
return
def set_l1():
vari.l1 = 'On'
vari.l2 = 'Off'
vari.l3 = 'Off'
vari.l4 = 'Off'
return
def set_l2():
vari.l1 = 'Off'
vari.l2 = 'On'
vari.l3 = 'Off'
vari.l4 = 'Off'
return
def set_l3():
vari.l1 = 'Off'
vari.l2 = 'Off'
vari.l3 = 'On'
vari.l4 = 'Off'
return
def set_l4():
vari.l1 = 'Off'
vari.l2 = 'Off'
vari.l3 = 'Off'
vari.l4 = 'On'
return
def set_Off():
vari.l1 = 'Off'
vari.l2 = 'Off'
vari.l3 = 'Off'
vari.l4 = 'Off'
return

def tk_pop():
app = Tkinter.Tk()
l1 = Tkinter.Button(app,text = "L1", command = set_l1)
l1.pack(side = 'left')
l2 = Tkinter.Button(app,text = "L2", command = set_l2)
l2.pack(side = 'left')
l3 = Tkinter.Button(app,text = "L3", command = set_l3)
l3.pack(side = 'left')
l4 = Tkinter.Button(app,text = "L4", command = set_l4)
l4.pack(side = 'left')
off_set = Tkinter.Button(app,text = "OFF", command = set_Off)
off_set.pack(side = 'right')
app.mainloop()
return

t = threading.Thread(target = tk_pop)
t.start()
while True:
l_update()

time.sleep(5)

hippy
Posts: 7792
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: reading file from rpi1 with rpi2

Sat Nov 07, 2015 9:07 pm

ritter4242 wrote:The reason I have two raspberry pis is the 'Official touch screen' only allows one screen (locks the HDMI) at the current time, so one raspberry pi will have a tkinter program running on the touch screen, the other will have the variable display, 10.1 inch hdmi.
As I understand it; both the official screen and HDMI are available and accessible at the same time but you will need to code your software to access both, one of those presumable through a frame buffer.

I have no idea how to do that myself, but figuring out how to would probably be a lot easier than using two Pi computers to do the job.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Sat Nov 07, 2015 9:51 pm

I've attached some of the code I used to test my theory *it seems you can't attach .py code... so its below*.
I think you are supposed to enclose code in a code block to avoid everything being left justified. I did, however, copy the Python code to a file and work through what I thought the formatting was , this added effort when things don't copy with their formatting being one of two reasons why I hate languages that abstain from the {} of C. What I get is a program that opens a GUI panel that can choose from five buttons to set one bit of four or clear all bits. Bits are represented as strings of 'off' and 'on'. So that part works for me. The issue is, to start out with, showing how two separate instances of python could communicate using global variables or something like that. In your tests, did you do this using some kind of parallel processing within one python instance or did you do this with two instances under the same user name? That latter bit doesn't work for me, at least so far.

stderr
Posts: 2178
Joined: Sat Dec 01, 2012 11:29 pm

Re: reading file from rpi1 with rpi2

Sat Nov 07, 2015 9:56 pm

hippy wrote:As I understand it; both the official screen and HDMI are available and accessible at the same time but you will need to code your software to access both, one of those presumable through a frame buffer.
Advantage can be gained, I think by simply creating your output on the one device, perhaps in an image format, and then copying that image to the other for display. Since the image is pretty much the lowest common usable format possible, I mean without any further processing, one design could work on many output devices. So whether he uses the 2 RPis or one with two displays, he could be getting the same output.

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Tue Nov 10, 2015 4:37 pm

Hi there everyone, sorry life has been a little...messy lately. Work has been cutting into my coding time. Anywho..

Hippy:
I'm rather new to coding on the raspberry pi, so to describe things a little better, I'm mounting the touchscreen and pi in my car, and using the second pi to display a hud on teleprompter glass (please don't let this degrade down to 'thats not legal'). I'm wanting to split them up for:
1. mounting reasons
2. I believe there will be enough processing going on to kill what I'm planning if I'm doing this only on 1 pi. I'm trying to do a can bus kind of thing, but I'm so unsure of what it is I'm doing.

stderr:
I will try using the blocks next time I put code in here :) I had done a couple tests on my laptop and my pi separatly, I might not have put up the right code... sorry about that. I'm hoping to have some time this evening to rewrite the code on my pi and see what all I can accomplish (accidentally lost access to my config.txt and reformatted the whole damn thing, which in the long run gave me more of what I wanted :) )

As always I'm so grateful for all the help :)

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Thu Nov 26, 2015 4:25 am

alright, my trick i was trying to get to work doesn't.... So, stdrr I will be switching to that message passing trick you mentioned.

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Thu Nov 26, 2015 4:26 am

stderr, Do you know if that message thing will work between pis?

ritter4242
Posts: 11
Joined: Fri Sep 25, 2015 4:07 am

Re: reading file from rpi1 with rpi2

Thu Nov 26, 2015 5:14 am

stderr, looked into that python cookbook recommendation, turns out 11.13 is basically what I was trying to do. sending large arrays across the network would work excellently. I'd be able to send and receive arrays that have the status of each variable. I guess the next question is, how do I connect two RPis together with an ethernet cable and have them able to talk to each other?

Return to “Beginners”