User avatar
Hove
Posts: 1182
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

RPi to RPi TCP connection

Thu Nov 01, 2012 11:13 am

I want to set up a TCP connection between a couple of RPis in Python. It's not the code I'm struggling with, it's the assignment of IP addresses. Each RPi is assigned it's own IP address on boot time depending on the next free address in the LAN they are in. Typically these are 192.168.1.66 / 67. Which RPi gets which address depends on which joins the LAN first (I assume).

One RPi is a server, the other will be a headless client driven by the server via the TCP connection. Hence I'd like my Python to learn it's IP address, and also I'd like the client to be able to discover the Server IP address to enable it to open the TCP connection to the "listening" RPi server.

While I can do that on the command line just by typing ifconfig before launching the server / client software, that enforced server to be started before the client, and I'll need to then supply command line parameter of the server IP address to the client, so the client can open a socket to the server IP address on a well known port.

My question is whether I can do this without having the ifconfig / command line but instead have the server code learn its own IP address in python, and have the client learn its own IP address and either a name or IP address it can use to connect to the server.

Any suggestions hugely appreciated. Note I'm a network expert but a python ignoramus!

Ta,

Hove

P.S. A very secondary question: how do I find out what port numbers are available for my custom TCP connection since that's what the server needs to be listening on, and the client to be connecting to. Ta
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

User avatar
PeterO
Posts: 3615
Joined: Sun Jul 22, 2012 4:14 pm

Re: RPi to RPi TCP connection

Thu Nov 01, 2012 11:19 am

Each pi has a unique MAC address ( http://en.wikipedia.org/wiki/MAC_address ) and you ought to be able to configure your dhcp server (possibly part of your ADSL router if you are in a domestic situation) to bind an IP address to a MAC address. That way each pi will always get the same IP address.
I run a dhcp server on my desktop linux machine and it serves fixed ip address to the pis.
PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

bredman
Posts: 1415
Joined: Tue Jan 17, 2012 2:38 pm

Re: RPi to RPi TCP connection

Thu Nov 01, 2012 11:55 am

Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.

User avatar
PeterO
Posts: 3615
Joined: Sun Jul 22, 2012 4:14 pm

Re: RPi to RPi TCP connection

Thu Nov 01, 2012 12:13 pm

bredman wrote:Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.
Then you have a changed the problem to "How to resolve a hostname to an IP address that is changing."
PeterO
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

User avatar
Hove
Posts: 1182
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
Contact: Website

Re: RPi to RPi TCP connection

Thu Nov 01, 2012 1:42 pm

Thanks,

I was aware of the stable layer 2 MAC address, but had forgotten that it's the dhcp server (which is on my ADSL wireless router) that would do the IP address assignment - thanks for refreshing me.

I'd considered host names also, but then the question becomes how do I assign a stable hostname to each RPi, and how do I then open sockets to a name rather than an IP address. Is this going to be DNS lookup instead? I'm guessing so, but how does each RPi register it's local LAN "name" with the DNS (on my ADSL router?

Ta,

Hove
www.pistuffing.co.uk - Raspberry Pi and other stuffing!

User avatar
PeterO
Posts: 3615
Joined: Sun Jul 22, 2012 4:14 pm

Re: RPi to RPi TCP connection

Thu Nov 01, 2012 2:53 pm

Hove wrote:Thanks,
I'd considered host names also, but then the question becomes how do I assign a stable hostname to each RPi, and how do I then open sockets to a name rather than an IP address. Is this going to be DNS lookup instead? I'm guessing so, but how does each RPi register it's local LAN "name" with the DNS (on my ADSL router?
Ta,
Hove
I have a solution, but it probably won't be suitable for you as you are using your router's dhcp service may not provide the required functionality.

The dhcp daemon I use is on my desktop linux machine and is configured to send a hostname as well as an IP address. A small script is added on the PI which uses the supplied hostname to set the actual hosthame. This way the mac address maps to a fixed IP which maps to a fixed hostname. Then it is a simple matter of having a hosts file (/etc/hosts) on each machine that maps the hostnames to the correct IP address.

PeterO

PS: I'm not sure what will happen if I plug my PIs into a network where the dhcp server doesn't provide a hostname in the address leases.
Discoverer of the PI2 XENON DEATH FLASH!
Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
"The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

MWRMWR
Posts: 1
Joined: Sun Nov 04, 2012 8:01 pm

Re: RPi to RPi TCP connection

Sun Nov 04, 2012 8:40 pm

The dhcp daemon I use is on my desktop linux machine and is configured to send a hostname as well as an IP address. A small script is added on the PI which uses the supplied hostname to set the actual hosthame. This way the mac address maps to a fixed IP which maps to a fixed hostname. Then it is a simple matter of having a hosts file (/etc/hosts) on each machine that maps the hostnames to the correct IP address.
Can you confirm the mechanism you use before I try this out please ?

My initial thoughts are to include one of these possibilities into /etc/dhcpd.conf on the server and figure out how they work:
  • host Blue {
    hardware ethernet 08:00:2b:4c:79:27;
    fixed-address 129.16.1.22;
    }
or
  • send host-name "<hostname>";
and then use the /etc/hosts on the client, ... but I get a bit lost there.

Ideally, I don't want to have to manually synchronise the /etc/hosts file, but anyway,
you are somehow picking up the hostname (Blue) sent out ...?
How do you do that - I assume you aren't doing DNS lookup ?

Hmm, I just stumbled across http://www.thekelleys.org.uk/dnsmasq/doc.html which looks a promissing solution.

Anyway, as you can infer, I don't really have enough knowledge to figure out from your posting how you did this!
Thanks for any clarification or advice.

    User avatar
    Hove
    Posts: 1182
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    Contact: Website

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 7:55 am

    I've accepted a simple though less than perfect solution for now.

    My dhcp server (in my wireless ADSL router) cached IP addresses against MAC addresses, so unless the server had run out of IP addresses, each discovered MAC address gets the same IP address every time.

    The dhcp service does off a subset of the IP pool to be statically configured also, so I could combine this with an /etc/hosts entry, but there's no benefit to this extra work, so I've filed in my "good to know" part of the brain!
    www.pistuffing.co.uk - Raspberry Pi and other stuffing!

    User avatar
    PeterO
    Posts: 3615
    Joined: Sun Jul 22, 2012 4:14 pm

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 9:29 am

    MWRMWR wrote:
    Can you confirm the mechanism you use before I try this out please ?
    I can't check until I get home this evening, but I'll post the script then.
    PeterO
    Discoverer of the PI2 XENON DEATH FLASH!
    Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
    "The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

    User avatar
    rurwin
    Forum Moderator
    Forum Moderator
    Posts: 4209
    Joined: Mon Jan 09, 2012 3:16 pm
    Contact: Website

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 10:57 am

    There is technology to do this, but it's rather more complex than the easy bodges.

    Use RARP or DHCP to fetch a static IP address for each MAC address. Have a single /etc/hosts file with all the names and static IP addresses listed in it. Use the IP address and the hosts file to work out what the local hostname is.

    That's the way it used to be done in SunOS, which shows just how old I am.

    For extra credit, distribute the hosts file using Sun NIS.

    User avatar
    PeterO
    Posts: 3615
    Joined: Sun Jul 22, 2012 4:14 pm

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 12:15 pm

    rurwin wrote:For extra credit, distribute the hosts file using Sun NIS.
    "Sun NIS" Pah, I was still called "Yellow Pages" when I first came across it :-)
    PeterO
    Discoverer of the PI2 XENON DEATH FLASH!
    Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
    "The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

    User avatar
    tedhale
    Posts: 114
    Joined: Thu Sep 20, 2012 4:52 pm
    Location: Williamsburg, VA, USA
    Contact: Website

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 2:04 pm

    the command
    netstat -a | grep LISTEN
    will show the ports that are currently being listened on.

    Just choose a port that is not a commonly used port.
    The file /etc/services lists a lot of the common ports.

    Use something above 10000 and you are usually safe.

    Question - do these systems need to move from one net to another (so they need DHCP)
    or are they staying put on the one network?
    It they don't move around, why not just set them to static IP addresses?
    - Ted B. Hale
    http://raspberrypihobbyist.blogspot.com

    User avatar
    Hove
    Posts: 1182
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    Contact: Website

    Re: RPi to RPi TCP connection

    Mon Nov 05, 2012 2:31 pm

    Hi Ted,

    The turtle will remain in the home network, and yes, I could just set them up with static IP addresses. But actually I don't need to since the dhcp server I use (in my ADSL wireless router) caches MAC addresses and tries to assign the same IP address to the same MAC addresses across timeouts. It's nice to know a better solution is there, but actually, my currently solution is good enought for now.

    Thanks for you help,

    Hove.

    P.S. Thanks for the port advice, that was the one piece of the picture I was missing. Cheers.
    www.pistuffing.co.uk - Raspberry Pi and other stuffing!

    User avatar
    PeterO
    Posts: 3615
    Joined: Sun Jul 22, 2012 4:14 pm

    Re: RPi to RPi TCP connection

    Tue Nov 06, 2012 6:54 am

    My dhcpserver configs for each Pi look like this:

    Code: Select all

    host PiOne {
      hardware ethernet b8:27:eb:65:08:36;
      fixed-address 192.168.1.101;
      option host-name "PiOne";
    }
    
    Each Pi has a script in /etc/dhcp/dhclient-exit-hooks.d/sethostname

    Code: Select all

    #!/bin/bash
    # Filename:     /etc/dhcp/dhclient-exit-hooks.d/sethostname
    # Purpose:      Used to set the hostname of the system
    #               as provided by  DHCP.
    
    if [ "$reason" == BOUND ] && [ -v new_host_name ]
    then
            echo $new_host_name > /etc/hostname
    	hostname $new_host_name
    fi
    
    The /etc/hosts files on all PIs match the entries in the dhcp server config

    Code: Select all

    192.168.1.101	PiOne.local	PiOne
    192.168.1.102	PiTwo.local	PiTwo
    
    HTH
    PeterO
    Discoverer of the PI2 XENON DEATH FLASH!
    Interests: C,Python,PIC,Electronics,Ham Radio (G0DZB),Aeromodelling,1960s British Computers.
    "The primary requirement (as we've always seen in your examples) is that the code is readable. " Dougie Lawson

    geztor
    Posts: 5
    Joined: Tue Nov 06, 2012 1:21 pm

    Re: RPi to RPi TCP connection

    Tue Nov 06, 2012 1:25 pm

    Resolving a persistent name to IP+port is a job for mDNS/DNS-SD (a.k.a. Bonjour). Take a look at the Python bindings for Avahi at http://avahi.org/wiki/Bindings.

    rbiks
    Posts: 16
    Joined: Sun Oct 07, 2012 2:47 pm
    Location: central Germany

    Re: RPi to RPi TCP connection

    Wed Nov 07, 2012 8:29 am

    Hi Hove,

    is you Pi going to be always on? If this is the case I suggest to disable the router's dhcp-server (as PeterO already stated the dhcp functionality on these devices is very limited) and setup a dhcp and dns-server for your local net on your Pi. You will be able to bind mac adresses to always the same ip adress and to use hostnames. I use dnsmasq. You should need less than half an hour to set it up. dnsmasq is also able to map more then one mac to one ip address. Very useful for laptops with wireless and cable network.

    Hope this helps

    rbiks

    User avatar
    Hove
    Posts: 1182
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    Contact: Website

    Re: RPi to RPi TCP connection

    Wed Nov 07, 2012 12:51 pm

    Thanks but the Turtle is just a toy - just a way for me to catch up with where I was 30 years ago with Python, electronics etc. So it's only on for a few hours a day at most and my dhcp works with hostnames, and also ensures that the IP addresses assigned the MAC addresses are the same each time (if possible), so it's providing all I need for my tinkering. Thanks anyway.

    Hove
    www.pistuffing.co.uk - Raspberry Pi and other stuffing!

    User avatar
    wallarug
    Posts: 459
    Joined: Mon May 14, 2012 8:21 am
    Contact: Website

    Re: RPi to RPi TCP connection

    Sun Nov 25, 2012 1:46 am

    I am using python socket module for a TCP client/server set-up.

    I have a question:

    How can I send some sort of message to the client or server that the connection has been lost?
    and then:
    how can I get the connection back up again, knowing that the connection has been lost?

    I can answer the second question if someone can tell me how to return that a connection has been lost.

    TCP server:

    Code: Select all

    import sys
    import socket               # Import socket module
    s = socket.socket()         # Create a socket object
    host = 'localhost' # Get local machine name
    port = 9999                # Reserve a port for your service.
    
     
    print 'Server started!'
    print 'Waiting for clients...'
    try: 
       s.bind((host, port))        # Bind to the port
       s.listen(5)                 # Now wait for client connection.
       c, addr = s.accept()     # Establish connection with client.
       print 'Got connection from', addr
    
    except KeyboardInterrupt:
       print "closing shell..."
       s.close()
       sys.exit("Closing Application...")
    TCP client:

    Code: Select all

    To messy to put here.
    Any help?
    RPi Hardware Guide

    App Store: https://itunes.apple.com/us/app/rpi-hardware-guide/id723108328?ls=1&mt=8
    Play Store: https://play.google.com/store/apps/details?id=org.cmdenterprises.rpihardwareguide

    http://www.youtube.com/user/CMDenterprises

    vmp32k
    Posts: 14
    Joined: Fri Jul 27, 2012 3:05 pm

    Re: RPi to RPi TCP connection

    Sun Nov 25, 2012 8:44 am

    Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)

    edit: this wouldn't require changing the dhcp/dns settings/hostnames and would just work with any existing network (given that it has some kind of autoconfiguration like dhcp so that sending packets back and forth is possible in the first place)

    User avatar
    wallarug
    Posts: 459
    Joined: Mon May 14, 2012 8:21 am
    Contact: Website

    Re: RPi to RPi TCP connection

    Sun Nov 25, 2012 9:17 am

    vmp32k wrote:Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)

    edit: this wouldn't require changing the dhcp/dns settings/hostnames and would just work with any existing network (given that it has some kind of autoconfiguration like dhcp so that sending packets back and forth is possible in the first place)
    I found a way that when the connection goes down (ie. msg = " ") then break the loop into a master loop.
    RPi Hardware Guide

    App Store: https://itunes.apple.com/us/app/rpi-hardware-guide/id723108328?ls=1&mt=8
    Play Store: https://play.google.com/store/apps/details?id=org.cmdenterprises.rpihardwareguide

    http://www.youtube.com/user/CMDenterprises

    User avatar
    rurwin
    Forum Moderator
    Forum Moderator
    Posts: 4209
    Joined: Mon Jan 09, 2012 3:16 pm
    Contact: Website

    Re: RPi to RPi TCP connection

    Sun Nov 25, 2012 10:21 am

    @wallarug

    You pose a question that gets a lot of professionals in trouble. I have used devices costing tens of thousands of pounds that have to be rebooted when the computer talking to them is rebooted.

    There is a solution for single clients and a different one for multiple clients.

    On the client, have a timeout for each response from the server. If the timeout triggers, or if the TCP stack reports the connection has failed, close the connection and open a new one.

    If the server supports multiple clients, then a new connection will be accepted and the old one will just hang around. So you need some way to close that down, but it does not have to be quick. Something like a long timeout waiting for the next request would do.

    If the server supports only a single client, and if only a single client will be trying to connect, then you have to make a bit of magic. This is the failure case that lots of people get wrong. When the client dies and then reconnects, the server wont listen because it already has a connection. Even with TCP KEEP_ALIVEs active, it still takes much too long to realise that the connection is bad.

    So if the server receives a new connection request, accept it and close the existing one.

    If you do ever have two clients trying to connect then it gets crazy for a while until one of them is stopped, but the remaining client then gets a stable connection.

    If there are many clients, but the server can only handle one at a time, then you need some other solution. Probably the best idea would be a fairly quick timeout between requests, forcing the clients to send a request every few seconds or be dropped. After all they are competing for a rare resource, they ought to be efficient about it.

    geztor
    Posts: 5
    Joined: Tue Nov 06, 2012 1:21 pm

    Re: RPi to RPi TCP connection

    Mon Nov 26, 2012 1:34 pm

    vmp32k wrote:Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)
    While it's always entertaining to read people reinventing the wheel, this is exactly what mDNS/DNS-SD does. You give your server a persistent DNS name (e.g., "myPiServer.local"), start a listening socket on some port, then advertise the domain-service-port tuple over mDNS/DNS-SD. Your client will run a discovery for the service you specified, which will return the current IP address and port combination of the server. This is literally half a dozen lines of code with the Python bindings for Avahi that I linked before, and requires no additional configuration.

    Corndork2
    Posts: 9
    Joined: Tue Nov 27, 2012 4:09 pm

    Re: RPi to RPi TCP connection

    Tue Nov 27, 2012 4:43 pm

    bredman wrote:Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.
    This would work if you are running your own DNS server that has table entries for the respective hostnames. They would need to be mapped to IP's as well, which should be static and not dynamic.

    As such, you would probably still need to set statid DHCP leases to ensure that the IP's arent changing.

    User avatar
    Hove
    Posts: 1182
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    Contact: Website

    Re: RPi to RPi TCP connection

    Sun Dec 09, 2012 9:41 am

    Thanks all for your assistance on this - I have my TCP connection working, mostly but...

    if I try to connect to the listening RPi, which I am actively pinging, the remove RPi accepts the connection without a problem.
    However, if I ping the listening RPi, but then stop the ping before attempting to connect to the listening RPi, the listening RPi reboots.

    The listening RPi is headless, although connected via rlogin to the other RPi purely for the sake of running the listening python code.

    My gut feel says this is an IP stack problem / kernel crash, but I have only behavioural evidence to show for that.

    Can anyone suggest diagnostics / debugging tactics (or a suggested cause / fix) that I can try?

    Cheers,

    Hove
    www.pistuffing.co.uk - Raspberry Pi and other stuffing!

    Digital Larry
    Posts: 70
    Joined: Tue Jul 24, 2012 9:10 pm
    Location: Silicon Valley, CA

    Re: RPi to RPi TCP connection

    Wed Jan 02, 2013 5:07 am

    geztor wrote: While it's always entertaining to read people reinventing the wheel, this is exactly what mDNS/DNS-SD does. You give your server a persistent DNS name (e.g., "myPiServer.local"), start a listening socket on some port, then advertise the domain-service-port tuple over mDNS/DNS-SD. Your client will run a discovery for the service you specified, which will return the current IP address and port combination of the server. This is literally half a dozen lines of code with the Python bindings for Avahi that I linked before, and requires no additional configuration.
    I second the recommendation to use mDNS. We use it on the commercial product I work on and it works great. Shows up automatically using the Safari browser under MacOS or various tools on Linux or Windows. However the supplied link does not work currently - it returns:

    Code: Select all

    Internal Server Error
    
    TracError: IOError: [Errno 2] No such file or directory: '/home/lennart/svn/trac/avahi/VERSION'
    I'd be quite interested in a step-by-step to enable mDNS and hook to it via Python.

    Thanks!

    DL

    Return to “Networking and servers”

    Who is online

    Users browsing this forum: No registered users and 20 guests