Unusual Network Activity


23 posts
by themacboy » Wed Nov 27, 2013 10:02 pm
Guys,

This is a bit of a weird issue that I am not sure how to solve. I have just purchased 4 Rasberrys for some monitoring (they are connected to some TVs, and will be streaming some webpages with stats on them)

I run a windows network primarily. All the users connect to a terminal server for all work tasks. When I plug one Pi into the network, no issues. As soon as I plug in the second one, I get some weird network issues. Users start to lose connectivity to the remote desktop session.

I have tried just running one for a full day without issues, but as soon as I plug in the second one, the issues start. I have checked the cables and today and it all looks fine. I have plugged in a Mac to the same network port the second Pi is on, and have had this running for the last hour with no issue. I think having 2 Pi's on my network is causing some network clash.

I have set static IPs on the 2 devices, but I still have an issue. Is there some other network settings I could try?

Any help would be appreciated.

Cheers
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by DougieLawson » Thu Nov 28, 2013 1:06 am
Have you assigned static or DHCP addresses?

If static are they all unique and not overlapping with any DHCP addresses?

If DHCP are they all assigned by ONE DHCP server and not overlapping with any static addresses.

On my network x.y.z.1 is the gateway/router.
X.y.z.2 up to x.y.z.29 are static servers
X.y.z.30 to x.y.z.100 are my DHCP pool
X.y.z.100 to x.y.z.229 are unused.
X.y.z.230 to x.y.z.254 are network kit like access points and printers and my NAS.

No overlaps.

X.y.8.z addresses are used for OpenVPN.
Hacker on ZX80 and Microtan65
Unemployed mainframe database specialist.
Linux hacker since 1995.
RPi owner since 2012.
Twitter: @DougieLawson

Gaffer tape is "The Force", it has a dark side and a light side and it holds the Universe together.
User avatar
Posts: 3563
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
by FLYFISH TECHNOLOGIES » Thu Nov 28, 2013 1:11 am
Hi,

themacboy wrote:Any help would be appreciated.

Try to obtain some details about the network traffic. You could run any network analyzer (eg. tcpdump or wireshark) and monitor packets. This could be a bit time-consuming task, but you'll be able to precisely find out what is going on...


Best wishes, Ivan Zilic.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by DougieLawson » Thu Nov 28, 2013 1:59 am
FLYFISH TECHNOLOGIES wrote:Hi,

themacboy wrote:Any help would be appreciated.

Try to obtain some details about the network traffic. You could run any network analyzer (eg. tcpdump or wireshark) and monitor packets. This could be a bit time-consuming task, but you'll be able to precisely find out what is going on...


Best wishes, Ivan Zilic.


Hey that was going to be my next tactic after checking the basics. I'm hoping its a simple addressing conflict and not something that needs packet inspection.
Hacker on ZX80 and Microtan65
Unemployed mainframe database specialist.
Linux hacker since 1995.
RPi owner since 2012.
Twitter: @DougieLawson

Gaffer tape is "The Force", it has a dark side and a light side and it holds the Universe together.
User avatar
Posts: 3563
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
by FLYFISH TECHNOLOGIES » Thu Nov 28, 2013 2:28 am
Hi Dougie,

DougieLawson wrote:Hey that was going to be my next tactic after checking the basics. I'm hoping its a simple addressing conflict and not something that needs packet inspection.

Sorry .... shell I delete my previous message ? :lol:

Pal took some time to compose the message, so I'm sure that he tried this and that, probably also swap IPs between two RasPis, etc. Furthermore, I saw a couple of times Windows popping up alert baloon in case of detected IP conflict, so this would probably be seen also here during hours of troubleshooting.
Therefore, I didn't want to guess anything... ;-)


Best wishes, Ivan Zilic.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by themacboy » Thu Nov 28, 2013 10:45 pm
Yesterday I stupidly realised that the Pis are all using the same build and so therefore all have the same hostname. I changed this, and plugged in a second one, but the issue came back :(

I am setting static IPs for these. Here is my /etc/network/interfaces

#auto lo
#iface lo inet loopback
#iface eth0 inet dhcp
#allow-hotplug wlan0
#iface wlan0 inet manual
#wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf
#iface default inet dhcp
#Loopback
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
#Static Address
address 192.168.4.177
#Gatway
gateway 192.168.4.253
netmask 255.255.255.0
#Address range
network 192.168.4.0
broadcast 192.168.4.255

Cheers
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by FLYFISH TECHNOLOGIES » Thu Nov 28, 2013 11:10 pm
Hi,

themacboy wrote:I am setting static IPs for these. Here is my /etc/network/interfaces

You can (should) check settings with ifconfig command. Data listed with this command is applicable.

Try to locate what exactly is going wrong, then you'll quickly know what to change...


Best wishes, Ivan Zilic.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by DougieLawson » Thu Nov 28, 2013 11:13 pm
themacboy wrote:address 192.168.4.177


I hope that's 192.168.4.177 for Pi#1
192.168.4.178 for Pi#2
192.168.4.179 ...

Or something like that.
Hacker on ZX80 and Microtan65
Unemployed mainframe database specialist.
Linux hacker since 1995.
RPi owner since 2012.
Twitter: @DougieLawson

Gaffer tape is "The Force", it has a dark side and a light side and it holds the Universe together.
User avatar
Posts: 3563
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
by FLYFISH TECHNOLOGIES » Thu Nov 28, 2013 11:23 pm
Hi,

DougieLawson wrote:I hope that's 192.168.4.177 for Pi#1
192.168.4.178 for Pi#2


It would surprise me if two RasPis with equal IPs on the network would cause other users on the network to start losing connectivity to Windows remote desktop session...
Anyway, I hope that when the mystery will be solved, explanation will be written here.


Regards, Ivan Zilic.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by themacboy » Thu Nov 28, 2013 11:26 pm
I hope that's 192.168.4.177 for Pi#1
192.168.4.178 for Pi#2
192.168.4.179 ...


Yup that is the plan. At the moment I have 2 Pis with different static IPs, but as soon as I put them on the network I get the issue.

I have just put a different distro in one of them (My Xbian build from home) and all seems OK at the moment. Must be some network config I have missed in Raspbien.

Does this build have some DHCP server that could be competing with my main one and causing the issue? How could I check that?

Cheers
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by Dutch_Master » Fri Nov 29, 2013 12:00 am
The fact your Win-OS users loose their connectivity is an indication the RPi's have the same addresses. Configure all RPi's to obtain a DHCP address. If the connectivity issues disappear, reconfigure the RPi's again with static addresses, but 100 more (so x.y.z.277 instead of x.y.z.177). Check the range of the DHCP server and make sure the fixed addresses are outside that range.
Posts: 251
Joined: Sat Jul 27, 2013 11:36 am
by precious_pony » Fri Nov 29, 2013 1:13 am
Dutch_Master wrote:The fact your Win-OS users loose their connectivity is an indication the RPi's have the same addresses. Configure all RPi's to obtain a DHCP address. If the connectivity issues disappear, reconfigure the RPi's again with static addresses, but 100 more (so x.y.z.277 instead of x.y.z.177). Check the range of the DHCP server and make sure the fixed addresses are outside that range.



Ip(v4) addresses only go up to 255 (so 277 is impossible).

By far the easiest solution is to get the mac address of all the pis, and to set up your router to give each pi a static ip address in a range outside of the normal DHCP range. (set up DHCP to range from x.x.x.50 to x.x.x.230 for example and then associate pi1's mac address with x.x.x.231, pi2 to 232, pi3 to 233, pi4 to 234).

All this can be done without configuring the pis themselves to use static ip addresses. It is the job of the router to sort these things out.
Posts: 28
Joined: Tue May 22, 2012 11:09 pm
by FLYFISH TECHNOLOGIES » Fri Nov 29, 2013 1:15 am
Hi,

themacboy wrote:I have just put a different distro in one of them

I'm not a big fan of this approach to solve issues...

Can you find a pattern, when the problem occurs ? Is the problem visible exactly when two RasPis are on the network... Therefore, if you have just the first one attached or just the second one, then everything works well ?
What happens when you change RasPis IPs to some other value (to eg. .33 and .34) ?
What is your DHCP pool (IP range) on this network ?

themacboy wrote:Does this build have some DHCP server that could be competing with my main one and causing the issue?

When all other computers are already "on the network", adding RasPi with DHCP server enabled to this network has no influence, because everybody already has its IP, so nobody asks for it (ok, true there is a renewal case, but I'd not consider this as an applicable case which would cause problems each time you connect RasPis to the network.)


Best regards, Ivan Zilic.
Last edited by FLYFISH TECHNOLOGIES on Fri Nov 29, 2013 1:17 am, edited 1 time in total.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by DougieLawson » Fri Nov 29, 2013 1:17 am
Dutch_Master wrote:addresses, but 100 more (so x.y.z.277 instead of x.y.z.177).


Hmmm, it's a little difficult with dotted-decimal addresses. Each segment can only have the range 0-255. x.y.z.277 isn't a valid address. There's no harm in using adjacent addresses just as long as everyone get a unique address. DHCP from a clean start hands the addresses out in sequence.

You could change the subnet mask to 255.255.nnn.0 (nnn is one value 254,252,248,240,224,192,128) from 255.255.255.0 but that starts to get complex. It would be easier to go to a 255.255.0.0 subnet from the 172.16.0.1 to 172.31.255.254 private network range.

If your DHCP server allows it you could assign a fixed address to each MAC address.

Code: Select all
        host pi1 {
                   hardware ethernet aa:bb:cc:dd:ee:ff; # <==  change me to the MAC seen on ifconfig
                    fixed-address 192.168.4.178;  # <== change me to a fixed address from your DHCP pool
           }
Hacker on ZX80 and Microtan65
Unemployed mainframe database specialist.
Linux hacker since 1995.
RPi owner since 2012.
Twitter: @DougieLawson

Gaffer tape is "The Force", it has a dark side and a light side and it holds the Universe together.
User avatar
Posts: 3563
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
by DougieLawson » Fri Nov 29, 2013 1:37 am
FLYFISH TECHNOLOGIES wrote:When all other computers are already "on the network", adding RasPi with DHCP server enabled to this network has no influence, because everybody already has its IP, so nobody asks for it (ok, true there is a renewal case, but I'd not consider this as an applicable case which would cause problems each time you connect RasPis to the network.)


The problem comes when a new machine or expired lease (unrecognised MAC) connects or an unexpired lease reconnects . There's then a race condition as both DHCP servers will get the "Hello my MAC is aa:bb:cc:dd:ee:ff and I'd like an IP address". Who wins, who loses and gets a bad ARP table? The other race is "Hello my current IP address is 192.168.x.y please can I renew." one DHCP server will say yes, the other will say no, the connecting machine may have a different IP address to the one in the ARP table to one of the DHCP servers. If the DHCP server with the bad ARP table is also the Internet gateway then packets will go down a black hole.

This is where IPv6 has a much better design and the route advertiser only has to tell the new machine my prefix is [2001:xxxx:yyyy:zzzz::/64]. In Linux we work out the bottom 64-bits of the address by taking 48-bits of MAC address, setting bit seven to 1 and adding ff:fe in the middle. So MAC b8:27:eb:00:06:9e becomes IPv6 2001:xxxx:yyyy:zzzz:llll:ba27:ebff:fe00:069e/64.

Windows (being windows) doesn't do that neat work with the MAC address, it uses a random number first time and stores it in the registry so that it consistently get the same address.

My LAN runs a mix of IPv4 and IPv6 with a 6to4 tunnel.
Hacker on ZX80 and Microtan65
Unemployed mainframe database specialist.
Linux hacker since 1995.
RPi owner since 2012.
Twitter: @DougieLawson

Gaffer tape is "The Force", it has a dark side and a light side and it holds the Universe together.
User avatar
Posts: 3563
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
by FLYFISH TECHNOLOGIES » Fri Nov 29, 2013 2:03 am
Hi,

DougieLawson wrote:
FLYFISH TECHNOLOGIES wrote:When all other computers are already "on the network", adding RasPi with DHCP server enabled to this network has no influence, because everybody already has its IP, so nobody asks for it (ok, true there is a renewal case, but I'd not consider this as an applicable case which would cause problems each time you connect RasPis to the network.)


The problem comes when a new machine or expired lease (unrecognised MAC) connects or an unexpired lease reconnects.

Thanks for longer explanation.
As you can see in my message, the point is my doubt that adding RasPi with enabled DHCP server would cause described trouble. I underlined the preconditions that:
- network is "stable" (there is no new computer asking for its IP) and
- the problem occurs quickly each time you plug RasPi to this network.

We can discuss what are proper settings (or "strategy") when you have more than one DHCP server on the network, but this thread is about locating the described issue... which, according to the description, I don't believe can be caused by active DHCP server on RasPi...


Best wishes, Ivan Zilic.
Running out of GPIO pins?
Solution:
http://www.flyfish-tech.com/FF32
User avatar
Posts: 925
Joined: Thu Oct 03, 2013 7:48 am
Location: Ljubljana, Slovenia
by themacboy » Fri Nov 29, 2013 4:33 am
Wow... So much good info here. Thanks for the prompt feedback guys, I will keep working on the problem, and update this thread soon!
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by rpdom » Fri Nov 29, 2013 6:33 am
Just a thought, possibly not relevant - especially with static addresses - do the two Pis have different hostnames? Or are they both called "raspberrypi"?

It might confuse other systems if they are both announcing the same hostname.

I have renamed mine to "raspi", "raspi2" and "raspi3".
Posts: 2513
Joined: Sun May 06, 2012 5:17 am
Location: Essex, UK
by cyrano » Fri Nov 29, 2013 11:02 am
I seem to remember Windows TS using multicast. The RPi doesn't support multicast. Could that trigger other users losing connectivity to the TS?
User avatar
Posts: 371
Joined: Wed Dec 05, 2012 11:48 pm
Location: Belgium
by emgi » Fri Nov 29, 2013 12:43 pm
FLYFISH TECHNOLOGIES wrote:We can discuss what are proper settings (or "strategy") when you have more than one DHCP server on the network, but this thread is about locating the described issue... which, according to the description, I don't believe can be caused by active DHCP server on RasPi...


Best wishes, Ivan Zilic.


This is a statement that I fully agree with. DHCP or misconfigured static IP addresses can NOT cause this type of phenomenon.
IMHO we are missing information about how the network actually looks and hence it becomes a guessing game.

Some questions from my side if I may:
- Everyone here seems to assume the terminal server is on the same subnet but is that really the case?

- Also interesting to know would be what kind of network equipment is used and how many nodes there are. Please provide brand & model; a drawing would be even better. Also add the total number of machines/ip addresses in use.

- Finally, I would like to know for sure if the issue occurs when connecting a specific Pi to the network or -as seems to be the case- any two out of several devices? Is connecting the Pi all it takes to introduce the problem or is it actually running/doing something like connecting to the Internet for example?

/emgi
User avatar
Posts: 56
Joined: Thu Nov 07, 2013 4:08 pm
Location: NL
by themacboy » Mon Dec 02, 2013 1:55 am
- Everyone here seems to assume the terminal server is on the same subnet but is that really the case?


Yup, it is a very small network that I manage (only 40 staff here) Everything is on 1 subnet.

Also interesting to know would be what kind of network equipment is used


I have a mix of Netgear (for servers, printers and some PCs) and Cisco (for phones and PCs). Unfortunately the last IT guy here didnt really leave me any documentation (we have all heard that story before right! :) ) I am trying to piece together the network quirks!

Finally, I would like to know for sure if the issue occurs when connecting a specific Pi to the network or -as seems to be the case- any two out of several devices? Is connecting the Pi all it takes to introduce the problem or is it actually running/doing something like connecting to the Internet for example?


It is when 2 or more are connected. It doesnt happen straight away, and I dont need to be running anything in particular for the issue to come about.
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by themacboy » Thu Dec 05, 2013 9:01 am
Ok so this is a little embarrassing!

Today I realised I had set a static IP for on of the Pi's to one of my Windows Servers (the main terminal server!)

DOH!!! Once I fixed that the problem went away.

Thanks for all the helpful comments, turns out I am a bit of a network nitwit :D
Posts: 6
Joined: Wed Nov 27, 2013 9:54 pm
by RaTTuS » Thu Dec 05, 2013 9:34 am
^ :-p
TBH setting each RPI to have a static address on each card is prone to problems
having more than one dhcp server on a network is going to give you headaces

having a dhcp server is good and this is where you should setup static IP's if you want them [this means the RPi can go anywhere and still get a good IP]
1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX - Prosliver FTW
"That's not right, the badgers have moved the goalposts."
User avatar
Posts: 4156
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK