Excessive broadband usage

[foredeck]

Odd thing this; I got RPi working a while back (End of Nov) and was happy enough with it to leave it turned on and connected.

A few days later (1st week of Dec) I got a 50% broadband usage warning from my ISP; this was unusual as the billing month runs 24-23 of the month, we usually get a warning of 50% used about the 20th so to get one less than 1/2-way through the month was odd.

A day later I got a 90% warning, which I've *never* had before.

Checking the ISP reports, all the traffic was listed as 'Web Browsing' which, of course, just means HTTP/HTTPS rather than anything intelligent.

I checked with the family but none of us had been doing anything unusual - no iPlayer usage to speak of, no torrents etc, so my only suspect was the RPi. Being short of time I just unplugged it and the ISP support team were able to confirm that traffic had dropped right back, so I'm pretty sure that's what it was.

Question is; does anyone have a clue about what was likely going on?

Using standard Raspbian distro and the only thing that might have been running was Scratch as #1 son (age 8) had been playing with it, although not to achieve anything much...

I haven't had a chance to turn it back on since, family and work commitments mean I've not had the time as yet. I'll be disconnecting Ethernet before starting it again, but was wondering what I should be checking for and, if anyone knows what's been going on, how to stop it.

Comments and suggestions appreciated.

Thanks

[hippy]

No idea what the problem or cause is but you should be able to calculate what amount of traffic must have been generated to cause the ISP's warning, determine your normal traffic, subtract that and you can estimate what amount of traffic the Pi would have been generating. That might give some clue.

If you have a hub which has network utilisation LEDs on the front ( some 3com hubs do - http://www.the-printer-man.com/ebay/3com-3c16751a-1.jpg - and maybe others ) you could perhaps put that between your Pi and the hub to the outside world.

If nothing else watching the activity LED will show if it's going crazy all the time or how frequently.

[foredeck]

Good suggestion about the hub, I may try that.

Usage is normally ~5-8GB/month with peak of no more than 1.5 in any particular day (that would be if we're iPlayering something).

Traffic limit at the time was 10GB/month; RPi munched 8+GB in about 3 days - I should probably take a look at the contents of the SD card and see if there's anything over-size there.

Thanks

[hippy]

Another thought ... Is the excessive traffic upstream or downstream - Could it be there's some open port that someone 'probing around' has chanced upon and started hammering ?

[gragib]

Which OS are you running on your Raspberry Pi, and are you running any server or other apps on it? In other words, what are you doing with your Raspberry Pi?

That amount of data usage suggests something nefarious.

[pluggy]

You could knock up a script that runs from cron and see exactly how much is coming into and leaving the Pi. I have 3 Pis permanently running and connected to the internet, and I have a monitoring system that among a lot of other things interrogates the router every minute and logs how much traffic is coming and going. It drops off to close to zero when everyone is asleep but the 3 Pis are still running.

Take a look : http://pluggy.is-a-geek.com/index.html

I'd be checking your Windows boxes for malware, its a common cause of sudden internet bandwidth increase. 90%+ of the spam you get in your inbox is sent from compromised Windows machines. You could be part of a botnet belonging to the Russian mafia. (I wouldn't sneer, its very common).

http://en.wikipedia.org/wiki/Botnet

Code: Select all

ifconfig

Code: Select all

eth0      Link encap:Ethernet  HWaddr b8:27:eb:06:4e:88  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

wlan0     Link encap:Ethernet  HWaddr 00:0f:12:19:0e:94  
          inet addr:192.168.1.248  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1062505 errors:0 dropped:1420443 overruns:0 frame:0
          TX packets:230560 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:115700832 (110.3 MiB)  TX bytes:164650588 (157.0 MiB)

The last line show how much has gone in and come out since the Pi was started. Its been up just short of 24 hours and I know that most of that is moving text data across the local network. If you aren't doing stuff across the LAN, its coming and going to the internet.

If you're using ethernet, it will be on the eth0 entry and wlan0 probably won't be there.

[foredeck]

Answers to several questions posted:

Traffic is Downstream - I hadn't thought of exploit possibilities, but being Linux haven't (yet) taken any precautions against nefarious access.

OS is standard/latest Raspbian, vanilla with no server apps or anything; as I said, I'd only just got it up and running. At present we're mucking about with Scratch and trying to get #1 son interested in programming concepts (at 8 he's a little young for straight Python!) but I haven't set up anything else.

Does Raspbian run an FTP server by default, or anything similar?

Windows boxen are all properly secured; 2 different corporate laptops in the house, along with 2 home ones running McAfee/Norton/MSFT and firewalls etc. Plus (again as stated) the excessive traffic stopped literally as I pulled the plug on RPi - I was on the phone to the ISP when I did it and they confirmed traffic stopped in real-time, ergo it was the Pi.

I'll look at the monitoring script suggested when I get a chance and implement before I reconnect to the internet, thanks.

[aaa801]

wait, did you use the raspbian image, or the raspbian instlaller?

[efflandt]

See what netstat -ltn shows for listening ports on the Pi and netstat -tn for active or recent connections (should not be much of anything if not actively doing something with it).

This would give a clue which programs are listening (6010 must be for ssh ForwardX11)

Code: Select all

efflandt@raspberrypi ~ $ sudo netstat -ltnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2116/sshd       
tcp        0      0 0.0.0.0:631             0.0.0.0:*               LISTEN      2115/cupsd      
tcp        0      0 127.0.0.1:6010          0.0.0.0:*               LISTEN      5255/0          
efflandt@raspberrypi ~ $ echo $DISPLAY
localhost:10.0

Check port forwarding on your router too and make sure that nothing unintended has ports forwarded to it or was accidentally set as DMZ.

Note that if something is generating a lot of traffic, the 32-bit numbers in ifconfig can roll over to zero at around 4.3 GB. So you might get false low numbers if something uses more traffic than that.

[Jim Manley]

I'm actually surprised something like this hasn't come up more often, but make sure that you have changed the Pi password and turn off SSH via sudo raspi-config before you reconnect your Pi to where it can be reached from the Internet. While you should be protected via network address translation (NAT) and a firewall in your router, there may be a misconfiguration that's allowing someone to access your Pi. If it's running with the default pi user ID and password then it's a cinch for them to own your Pi via sudo.

I would guess that by now the botnet owners have added the default Pi network signatures and accounts to their scripts to look for open systems just for the hack of it. There's probably not much valuable stuff on the vast majority of Pi systems so far, due to the hobbyist nature of their typical use - one of the beneficial side-effects of its low-end performance as a primary computing platform. However, a compromised system on your LAN could potentially be used to attack other systems there, particularly if you have any default accounts running on them, or accounts with weak passwords, etc.

You might also want to run top to see what processes are running on your Pi, as anything using that kind of bandwidth would also be consuming CPU resources to a large fraction. Of course, you'd have to reconnect the Pi to the Internet, but doing so briefly for observation won't result in too much more loss of account usage capacity.

The tools discussed on the following page may be useful for monitoring and analyzing network traffic on your local connections to the Internet:

http://www.debianhelp.co.uk/networktools2.htm

[jamesh]

Do you use a wireless router? Has someone hacked your password and is using your broadband over wireless? Should appear in the logs.

[harroxelas]

Here are a few suggestion of programs/commands you can run on your Raspberry Pi in order to monitor your bandwidth:

1) netstat -tunp

Lists tcp/udp connections by ip addresses, ports and programs

2) iftop -i [interface] (wlan0 or eth0 for example)

Listens to all network traffic on a particular interface

3) nethogs [interface] (wlan0 or eth0 for example)

Shows traffic per process. Great tool.
Nethogs is available on arch linux arm repos, dunno about raspbian or other distros.

Hope it helps.

[Lob0426]

If you suspect that someone is hammering your RasPi, you should first change your password. Then I would install "fail2ban". It "bans" anyone, who fails the authentication several times, temporarily. You can edit its settings. That should slow them up a bit.

If you have a wireless router I would definitely change its password, if you have not done it anytime recently. Unfortunately there are people that would rather use your bandwidth than theirs!

[jojopi]

If you suspect that someone is hammering your RasPi, you should first change your password. Then I would install "fail2ban".

That is not logical. If the Pi could be reached from the internet, and the password was weak, then it is now too late to change it. The card should be re-imaged.

Software to "slow them up a bit" is security theatre. Deny access or use a strong password.

To avoid giving too much credibility to the mass hysteria in this thread, it must be pointed out that there is no actual evidence of any wrongdoing, nor that the router did allow inbound access.

[cyrano]

Using standard Raspbian distro and the only thing that might have been running was Scratch as #1 son (age 8) had been playing with it, although not to achieve anything much...

I don't know Scratch, but isn't there a "load internet page" or other http function in it? Any chance your son accidentally created a function that was left running? Even 404 errors can add up if left running 24/7.

And that's outgoing traffic, so it wouldn't even need port forwarding from your router.

[Fludizz]

Security of the Pi is exactly the reason why I switched from OpenELEC to RaspBMC on my 24/7 Pi. I have a dual-stack ISP, and have both IPv4 and IPv6 running.

I discovered OpenELEC enabled IPv6 by default but does not have any netfilter/ip6tables modules present in it's kernel. As per design, OpenELEC is very limited in modifications. This means the Pi was internet reachable on it's IPv6 address using both SSH and XBMC web and there was nothing I could do to prevent that.

In RaspBMC I was able to apply firewall rules to block access from the internet to my Pi. Only my local subnets are now allowed to connect to the Pi and SSH is set up to only accept pubkey authentication. Good luck getting in now!

Yes, security is a big issue with the Pi... Bigger then most people realize, mostly because of the well-known default passwords which are not being changed. Anyone with access to your network (be it through your router or via wifi) can compromise your Pi if you don't change passwords. I have network graphs for my entire network and my Pi's are not showing any excess traffic when left idle, the suggestion that your Pi might be compromised can be a very valid explanation.

[obcd]

Was the Pi connected to a router or is the ISP modem having a buildin router?
If that's the case, are some of the router settings changed to allow incoming traffic to reach one of the local machines?
If the Pi is behind a router, and no ports are set open on that router to it, I doubt the traffic comes or is initiated from the outsite.
If your router is having logging options, it might be worth to check what traffic is coming from the Pi.
I think it's possible to add an iptables script to the pi so that it generates it's own loggings. I am no expert in this. You could log all attempts to reach an ip outside the local network.
Simply running ifconfig on the Pi and checking the number of RX and TX packets also might help figuring out if the Pi is generating a lot of traffic on the ethernet.
I love all this conspiracy theories with the russian maffia and cia, but last time I checked my contacts there, they said they had no interest in the Pi at the moment since nobody was doing his online banking with it.

[Fludizz]

The router-bit is valid regarding IPv4, NAT adds some end-user protection as long as you don't open the wrong ports to the wrong machines
IPv6, by design, provides end-user to end-user connectivity and requires firewalling at every end-station (Or you need to put a statefull IPv6 capable firewall on the network boundary).

The biggest concern for the pi is not the phishing/banking/CIA/FBI/Mafia/Russians/whatever crap. Things like botnets, DDoS, network penetration (Hopping through an IPv6 enabled Pi to a private IPv4 network) and malware distribution seem more likely to be done using compromised Pi's

For the record, I've been using IPv6 for years. Until this point in time I have not seen any attacks via IPv6 to my systems.

[jj462]

I know this is bit old now but I have encountered a similar problem. I have a RPI4 with raspian lite os. I am in the process on setting it up as a Home Assistant hub. I have the RPI4 connected to my router, which is an HTC 5g hub, by Ethernet. I have left the RPI4 running and noticed the data usage was quite high, about 15g in 7 days. I found about 14g had been used by the RPI4. The hub gives me a breakdown of by whom and how much is being used. I have looked at the Home Assistant app a few times but do not have any devices setup with it yet So I don’t think much should have gone through it. Any ideas.

If there are any intrusions would going through a VPN help?