RPi to RPi TCP connection


26 posts   Page 1 of 2   1, 2
by Hove » Thu Nov 01, 2012 11:13 am
I want to set up a TCP connection between a couple of RPis in Python. It's not the code I'm struggling with, it's the assignment of IP addresses. Each RPi is assigned it's own IP address on boot time depending on the next free address in the LAN they are in. Typically these are 192.168.1.66 / 67. Which RPi gets which address depends on which joins the LAN first (I assume).

One RPi is a server, the other will be a headless client driven by the server via the TCP connection. Hence I'd like my Python to learn it's IP address, and also I'd like the client to be able to discover the Server IP address to enable it to open the TCP connection to the "listening" RPi server.

While I can do that on the command line just by typing ifconfig before launching the server / client software, that enforced server to be started before the client, and I'll need to then supply command line parameter of the server IP address to the client, so the client can open a socket to the server IP address on a well known port.

My question is whether I can do this without having the ifconfig / command line but instead have the server code learn its own IP address in python, and have the client learn its own IP address and either a name or IP address it can use to connect to the server.

Any suggestions hugely appreciated. Note I'm a network expert but a python ignoramus!

Ta,

Hove

P.S. A very secondary question: how do I find out what port numbers are available for my custom TCP connection since that's what the server needs to be listening on, and the client to be connecting to. Ta
www.pistuffing.co.uk - things to fulfil your Raspberry Pi
User avatar
Posts: 750
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
by PeterO » Thu Nov 01, 2012 11:19 am
Each pi has a unique MAC address ( http://en.wikipedia.org/wiki/MAC_address ) and you ought to be able to configure your dhcp server (possibly part of your ADSL router if you are in a domestic situation) to bind an IP address to a MAC address. That way each pi will always get the same IP address.
I run a dhcp server on my desktop linux machine and it serves fixed ip address to the pis.
PeterO
User avatar
Posts: 1041
Joined: Sun Jul 22, 2012 4:14 pm
by bredman » Thu Nov 01, 2012 11:55 am
Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.
Posts: 1413
Joined: Tue Jan 17, 2012 2:38 pm
by PeterO » Thu Nov 01, 2012 12:13 pm
bredman wrote:Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.

Then you have a changed the problem to "How to resolve a hostname to an IP address that is changing."
PeterO
User avatar
Posts: 1041
Joined: Sun Jul 22, 2012 4:14 pm
by Hove » Thu Nov 01, 2012 1:42 pm
Thanks,

I was aware of the stable layer 2 MAC address, but had forgotten that it's the dhcp server (which is on my ADSL wireless router) that would do the IP address assignment - thanks for refreshing me.

I'd considered host names also, but then the question becomes how do I assign a stable hostname to each RPi, and how do I then open sockets to a name rather than an IP address. Is this going to be DNS lookup instead? I'm guessing so, but how does each RPi register it's local LAN "name" with the DNS (on my ADSL router?

Ta,

Hove
www.pistuffing.co.uk - things to fulfil your Raspberry Pi
User avatar
Posts: 750
Joined: Sun Oct 21, 2012 6:55 pm
Location: Cotswolds, UK
by PeterO » Thu Nov 01, 2012 2:53 pm
Hove wrote:Thanks,
I'd considered host names also, but then the question becomes how do I assign a stable hostname to each RPi, and how do I then open sockets to a name rather than an IP address. Is this going to be DNS lookup instead? I'm guessing so, but how does each RPi register it's local LAN "name" with the DNS (on my ADSL router?
Ta,
Hove

I have a solution, but it probably won't be suitable for you as you are using your router's dhcp service may not provide the required functionality.

The dhcp daemon I use is on my desktop linux machine and is configured to send a hostname as well as an IP address. A small script is added on the PI which uses the supplied hostname to set the actual hosthame. This way the mac address maps to a fixed IP which maps to a fixed hostname. Then it is a simple matter of having a hosts file (/etc/hosts) on each machine that maps the hostnames to the correct IP address.

PeterO

PS: I'm not sure what will happen if I plug my PIs into a network where the dhcp server doesn't provide a hostname in the address leases.
User avatar
Posts: 1041
Joined: Sun Jul 22, 2012 4:14 pm
by MWRMWR » Sun Nov 04, 2012 8:40 pm
The dhcp daemon I use is on my desktop linux machine and is configured to send a hostname as well as an IP address. A small script is added on the PI which uses the supplied hostname to set the actual hosthame. This way the mac address maps to a fixed IP which maps to a fixed hostname. Then it is a simple matter of having a hosts file (/etc/hosts) on each machine that maps the hostnames to the correct IP address.


Can you confirm the mechanism you use before I try this out please ?

My initial thoughts are to include one of these possibilities into /etc/dhcpd.conf on the server and figure out how they work:
    host Blue {
    hardware ethernet 08:00:2b:4c:79:27;
    fixed-address 129.16.1.22;
    }
or
    send host-name "<hostname>";
and then use the /etc/hosts on the client, ... but I get a bit lost there.

Ideally, I don't want to have to manually synchronise the /etc/hosts file, but anyway,
you are somehow picking up the hostname (Blue) sent out ...?
How do you do that - I assume you aren't doing DNS lookup ?

Hmm, I just stumbled across http://www.thekelleys.org.uk/dnsmasq/doc.html which looks a promissing solution.

Anyway, as you can infer, I don't really have enough knowledge to figure out from your posting how you did this!
Thanks for any clarification or advice.
    Posts: 1
    Joined: Sun Nov 04, 2012 8:01 pm
    by Hove » Mon Nov 05, 2012 7:55 am
    I've accepted a simple though less than perfect solution for now.

    My dhcp server (in my wireless ADSL router) cached IP addresses against MAC addresses, so unless the server had run out of IP addresses, each discovered MAC address gets the same IP address every time.

    The dhcp service does off a subset of the IP pool to be statically configured also, so I could combine this with an /etc/hosts entry, but there's no benefit to this extra work, so I've filed in my "good to know" part of the brain!
    www.pistuffing.co.uk - things to fulfil your Raspberry Pi
    User avatar
    Posts: 750
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    by PeterO » Mon Nov 05, 2012 9:29 am
    MWRMWR wrote:
    Can you confirm the mechanism you use before I try this out please ?

    I can't check until I get home this evening, but I'll post the script then.
    PeterO
    User avatar
    Posts: 1041
    Joined: Sun Jul 22, 2012 4:14 pm
    by rurwin » Mon Nov 05, 2012 10:57 am
    There is technology to do this, but it's rather more complex than the easy bodges.

    Use RARP or DHCP to fetch a static IP address for each MAC address. Have a single /etc/hosts file with all the names and static IP addresses listed in it. Use the IP address and the hosts file to work out what the local hostname is.

    That's the way it used to be done in SunOS, which shows just how old I am.

    For extra credit, distribute the hosts file using Sun NIS.
    User avatar
    Forum Moderator
    Forum Moderator
    Posts: 2936
    Joined: Mon Jan 09, 2012 3:16 pm
    by PeterO » Mon Nov 05, 2012 12:15 pm
    rurwin wrote:For extra credit, distribute the hosts file using Sun NIS.

    "Sun NIS" Pah, I was still called "Yellow Pages" when I first came across it :-)
    PeterO
    User avatar
    Posts: 1041
    Joined: Sun Jul 22, 2012 4:14 pm
    by tedhale » Mon Nov 05, 2012 2:04 pm
    the command
    netstat -a | grep LISTEN
    will show the ports that are currently being listened on.

    Just choose a port that is not a commonly used port.
    The file /etc/services lists a lot of the common ports.

    Use something above 10000 and you are usually safe.

    Question - do these systems need to move from one net to another (so they need DHCP)
    or are they staying put on the one network?
    It they don't move around, why not just set them to static IP addresses?
    - Ted B. Hale
    http://raspberrypihobbyist.blogspot.com
    User avatar
    Posts: 112
    Joined: Thu Sep 20, 2012 4:52 pm
    Location: Williamsburg, VA, USA
    by Hove » Mon Nov 05, 2012 2:31 pm
    Hi Ted,

    The turtle will remain in the home network, and yes, I could just set them up with static IP addresses. But actually I don't need to since the dhcp server I use (in my ADSL wireless router) caches MAC addresses and tries to assign the same IP address to the same MAC addresses across timeouts. It's nice to know a better solution is there, but actually, my currently solution is good enought for now.

    Thanks for you help,

    Hove.

    P.S. Thanks for the port advice, that was the one piece of the picture I was missing. Cheers.
    www.pistuffing.co.uk - things to fulfil your Raspberry Pi
    User avatar
    Posts: 750
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    by PeterO » Tue Nov 06, 2012 6:54 am
    My dhcpserver configs for each Pi look like this:
    Code: Select all
    host PiOne {
      hardware ethernet b8:27:eb:65:08:36;
      fixed-address 192.168.1.101;
      option host-name "PiOne";
    }


    Each Pi has a script in /etc/dhcp/dhclient-exit-hooks.d/sethostname
    Code: Select all
    #!/bin/bash
    # Filename:     /etc/dhcp/dhclient-exit-hooks.d/sethostname
    # Purpose:      Used to set the hostname of the system
    #               as provided by  DHCP.

    if [ "$reason" == BOUND ] && [ -v new_host_name ]
    then
            echo $new_host_name > /etc/hostname
       hostname $new_host_name
    fi


    The /etc/hosts files on all PIs match the entries in the dhcp server config
    Code: Select all
    192.168.1.101   PiOne.local   PiOne
    192.168.1.102   PiTwo.local   PiTwo


    HTH
    PeterO
    User avatar
    Posts: 1041
    Joined: Sun Jul 22, 2012 4:14 pm
    by geztor » Tue Nov 06, 2012 1:25 pm
    Resolving a persistent name to IP+port is a job for mDNS/DNS-SD (a.k.a. Bonjour). Take a look at the Python bindings for Avahi at http://avahi.org/wiki/Bindings.
    Posts: 5
    Joined: Tue Nov 06, 2012 1:21 pm
    by rbiks » Wed Nov 07, 2012 8:29 am
    Hi Hove,

    is you Pi going to be always on? If this is the case I suggest to disable the router's dhcp-server (as PeterO already stated the dhcp functionality on these devices is very limited) and setup a dhcp and dns-server for your local net on your Pi. You will be able to bind mac adresses to always the same ip adress and to use hostnames. I use dnsmasq. You should need less than half an hour to set it up. dnsmasq is also able to map more then one mac to one ip address. Very useful for laptops with wireless and cable network.

    Hope this helps

    rbiks
    Posts: 16
    Joined: Sun Oct 07, 2012 2:47 pm
    Location: central Germany
    by Hove » Wed Nov 07, 2012 12:51 pm
    Thanks but the Turtle is just a toy - just a way for me to catch up with where I was 30 years ago with Python, electronics etc. So it's only on for a few hours a day at most and my dhcp works with hostnames, and also ensures that the IP addresses assigned the MAC addresses are the same each time (if possible), so it's providing all I need for my tinkering. Thanks anyway.

    Hove
    www.pistuffing.co.uk - things to fulfil your Raspberry Pi
    User avatar
    Posts: 750
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    by wallarug » Sun Nov 25, 2012 1:46 am
    I am using python socket module for a TCP client/server set-up.

    I have a question:

    How can I send some sort of message to the client or server that the connection has been lost?
    and then:
    how can I get the connection back up again, knowing that the connection has been lost?

    I can answer the second question if someone can tell me how to return that a connection has been lost.

    TCP server:
    Code: Select all
    import sys
    import socket               # Import socket module
    s = socket.socket()         # Create a socket object
    host = 'localhost' # Get local machine name
    port = 9999                # Reserve a port for your service.

     
    print 'Server started!'
    print 'Waiting for clients...'
    try:
       s.bind((host, port))        # Bind to the port
       s.listen(5)                 # Now wait for client connection.
       c, addr = s.accept()     # Establish connection with client.
       print 'Got connection from', addr

    except KeyboardInterrupt:
       print "closing shell..."
       s.close()
       sys.exit("Closing Application...")


    TCP client:
    Code: Select all
    To messy to put here.


    Any help?
    RPi Hardware Guide

    App Store: https://itunes.apple.com/us/app/rpi-hardware-guide/id723108328?ls=1&mt=8
    Play Store: https://play.google.com/store/apps/details?id=org.cmdenterprises.rpihardwareguide

    http://www.youtube.com/user/CMDenterprises
    User avatar
    Posts: 457
    Joined: Mon May 14, 2012 8:21 am
    by vmp32k » Sun Nov 25, 2012 8:44 am
    Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)

    edit: this wouldn't require changing the dhcp/dns settings/hostnames and would just work with any existing network (given that it has some kind of autoconfiguration like dhcp so that sending packets back and forth is possible in the first place)
    Posts: 14
    Joined: Fri Jul 27, 2012 3:05 pm
    by wallarug » Sun Nov 25, 2012 9:17 am
    vmp32k wrote:Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)

    edit: this wouldn't require changing the dhcp/dns settings/hostnames and would just work with any existing network (given that it has some kind of autoconfiguration like dhcp so that sending packets back and forth is possible in the first place)


    I found a way that when the connection goes down (ie. msg = " ") then break the loop into a master loop.
    RPi Hardware Guide

    App Store: https://itunes.apple.com/us/app/rpi-hardware-guide/id723108328?ls=1&mt=8
    Play Store: https://play.google.com/store/apps/details?id=org.cmdenterprises.rpihardwareguide

    http://www.youtube.com/user/CMDenterprises
    User avatar
    Posts: 457
    Joined: Mon May 14, 2012 8:21 am
    by rurwin » Sun Nov 25, 2012 10:21 am
    @wallarug

    You pose a question that gets a lot of professionals in trouble. I have used devices costing tens of thousands of pounds that have to be rebooted when the computer talking to them is rebooted.

    There is a solution for single clients and a different one for multiple clients.

    On the client, have a timeout for each response from the server. If the timeout triggers, or if the TCP stack reports the connection has failed, close the connection and open a new one.

    If the server supports multiple clients, then a new connection will be accepted and the old one will just hang around. So you need some way to close that down, but it does not have to be quick. Something like a long timeout waiting for the next request would do.

    If the server supports only a single client, and if only a single client will be trying to connect, then you have to make a bit of magic. This is the failure case that lots of people get wrong. When the client dies and then reconnects, the server wont listen because it already has a connection. Even with TCP KEEP_ALIVEs active, it still takes much too long to realise that the connection is bad.

    So if the server receives a new connection request, accept it and close the existing one.

    If you do ever have two clients trying to connect then it gets crazy for a while until one of them is stopped, but the remaining client then gets a stable connection.

    If there are many clients, but the server can only handle one at a time, then you need some other solution. Probably the best idea would be a fairly quick timeout between requests, forcing the clients to send a request every few seconds or be dropped. After all they are competing for a rare resource, they ought to be efficient about it.
    User avatar
    Forum Moderator
    Forum Moderator
    Posts: 2936
    Joined: Mon Jan 09, 2012 3:16 pm
    by geztor » Mon Nov 26, 2012 1:34 pm
    vmp32k wrote:Another approach could be (depending on your python knowledge) using broadcasts. Have the server or client send out broadcasts, that the other Pi listens for, containing their IP address.
    Like some kind of a "peer discovery" feature. :)


    While it's always entertaining to read people reinventing the wheel, this is exactly what mDNS/DNS-SD does. You give your server a persistent DNS name (e.g., "myPiServer.local"), start a listening socket on some port, then advertise the domain-service-port tuple over mDNS/DNS-SD. Your client will run a discovery for the service you specified, which will return the current IP address and port combination of the server. This is literally half a dozen lines of code with the Python bindings for Avahi that I linked before, and requires no additional configuration.
    Posts: 5
    Joined: Tue Nov 06, 2012 1:21 pm
    by Corndork2 » Tue Nov 27, 2012 4:43 pm
    bredman wrote:Using IP addresses is very restrictive. It would be much better to use hostnames to identify the RPi's.


    This would work if you are running your own DNS server that has table entries for the respective hostnames. They would need to be mapped to IP's as well, which should be static and not dynamic.

    As such, you would probably still need to set statid DHCP leases to ensure that the IP's arent changing.
    Posts: 9
    Joined: Tue Nov 27, 2012 4:09 pm
    by Hove » Sun Dec 09, 2012 9:41 am
    Thanks all for your assistance on this - I have my TCP connection working, mostly but...

    if I try to connect to the listening RPi, which I am actively pinging, the remove RPi accepts the connection without a problem.
    However, if I ping the listening RPi, but then stop the ping before attempting to connect to the listening RPi, the listening RPi reboots.

    The listening RPi is headless, although connected via rlogin to the other RPi purely for the sake of running the listening python code.

    My gut feel says this is an IP stack problem / kernel crash, but I have only behavioural evidence to show for that.

    Can anyone suggest diagnostics / debugging tactics (or a suggested cause / fix) that I can try?

    Cheers,

    Hove
    www.pistuffing.co.uk - things to fulfil your Raspberry Pi
    User avatar
    Posts: 750
    Joined: Sun Oct 21, 2012 6:55 pm
    Location: Cotswolds, UK
    by Digital Larry » Wed Jan 02, 2013 5:07 am
    geztor wrote:While it's always entertaining to read people reinventing the wheel, this is exactly what mDNS/DNS-SD does. You give your server a persistent DNS name (e.g., "myPiServer.local"), start a listening socket on some port, then advertise the domain-service-port tuple over mDNS/DNS-SD. Your client will run a discovery for the service you specified, which will return the current IP address and port combination of the server. This is literally half a dozen lines of code with the Python bindings for Avahi that I linked before, and requires no additional configuration.


    I second the recommendation to use mDNS. We use it on the commercial product I work on and it works great. Shows up automatically using the Safari browser under MacOS or various tools on Linux or Windows. However the supplied link does not work currently - it returns:

    Code: Select all
    Internal Server Error

    TracError: IOError: [Errno 2] No such file or directory: '/home/lennart/svn/trac/avahi/VERSION'


    I'd be quite interested in a step-by-step to enable mDNS and hook to it via Python.

    Thanks!

    DL
    Posts: 62
    Joined: Tue Jul 24, 2012 9:10 pm
    Location: Silicon Valley, CA