Networking - Raw Ethernet programming

[JohnAdriaan]

Hi everybody,

This is gonna be a long one (or not - I hope)!
TL;DR: How do I 'quarantine' an active, usable Ethernet port from the TCP/IP stack?

I've been programming TCP/IP sockets for twenty years. ARP, DHCP, ICMP, UDP, TCP, DNS - I understand it all. My favourite diagnostic tool is Ethereal (WireShark) - but it has one huge limitation. Ever since broadcast hubs went the way of the Dodo and were replaced by smart switches, you don't see everything passing by: if you're not involved in the conversation, you miss things. You either pay $20K for an Ethernet diagnostic device - or you program a Raspberry Pi to do it for you!

Take an ordinary Raspberry Pi, and plug in a USB-to-Ethernet adapter. Connect the on-board Ethernet to the WAN modem, and the USB adapter to the LAN router, and you have a device that could, conceivably, with the right software, watch every packet that passes through. All it'd have to do is transmit every packet it received to the opposite side so that communications wouldn't be disrupted (with a slight latency) - but you could also log packets of interest in real time. And since there's WiFi, you could also serve up a Web page to configure and monitor the traffic in real time, again without disrupting the traffic.

Moreover, the two endpoints wouldn't know that the forwarding was taking place. I call it "Pi in the Middle", or PitM (like cryptography's Man in the Middle - only benigner). But it's not working.

Specifically, as soon as I plug the USB-to-Ethernet adapter in, it gets an interface name of "eth1" (not mentioned anywhere in /etc/network/interfaces or dhcpcd.conf), and then DHCPs an Ethernet address. Assigning a static address is beside the point - I don't want ANY address while it is being used for PitM. I want the Interface to be Up so that I can open it, and read and write to it with a Raw Socket - but I don't want the Interface to do any of the 'normal' TCP/IP things.

Ideally, I'd want PitM to "strip off" the TCP/IP-ness of the port when it was configured as one of the two ports (ioctl?) - and restore it when it was released again. For example (but not correct): use SIOCSIFFLAGS to change IFF_RUNNABLE. But if necessary I'd accept a systemic change that would 'quarantine' the ports from the TCP/IP stack.

Any suggestions? Thanks,

John Adriaan

[PeterO]

TL:DR

Are these any help ? https://gist.github.com/austinmarton
https://gist.github.com/austinmarton/2862515
https://gist.github.com/austinmarton/1922600

PeterO

[DougieLawson]

Those examples from Peter are UDP/IP not TCP/IP. But you've not stated what layer three protocol you'd like to use.

[PeterO]

Those examples from Peter are UDP/IP not TCP/IP. But you've not stated what layer three protocol you'd like to use.

As far as I can see it only assumes they are UDP packets because it looks for the header where it expects to see it.
I can't see any code that checks the protocol number.

PeterO

[JohnAdriaan]

https://gist.github.com/austinmarton/2862515
https://gist.github.com/austinmarton/1922600

Peter,
The code given in those links are for the receiving and transmitting of raw Ethernet packets - code that I've written many times before.

My question is about how to configure Raspbian (using code or config files) so that the Ethernet ports that I'm using for that raw receive and transmit aren't also used for normal IP traffic. I can't "ifconfig down" them - then they're not available for use by my code. I need to surgically exclude them from the TCP/IP stack.

Those examples from Peter are UDP/IP not TCP/IP. But you've not stated what layer three protocol you'd like to use.

Dougie,
The statements about "what layer three protocol [...] to use" and "UDP vs TCP" miss the point. I'm asking about (for example) monitoring a conversation, and you're asking which language I want: English, Spanish or Russian. I don't want any language - I want the raw audio. They could be speaking Martian for all I care.

The raw Ethernet frames received can definitely be analysed and selectively saved for logging and troubleshooting purposes once they've been captured. I haven't got that far. The moment I plug one of the ports that I'll use for monitoring into the router, Raspbian uses DHCP to assign an address to the port. That's what I want to stop.

Before I plug any USB-to-Ethernet converter into the Pi, none of the configuration files even mention eth1. After I plug the USB-to-Ethernet adapter in, still none of the configuration files mention it - and yet eth1 gets an IP address.

[DougieLawson]

If you want raw frames then install tcpdump it reads traffic at layer 2.

[pi-anazazi]

...I use some raspis with 2 (two) USB-ethernet adapters in bridged mode running Wireshark (monitoring internet traffic of a PC as "pi-in-the-middle) using the built-in RJ45 to access the (headless) raspi via VNC.

Bridging the two USB-devices (as sudo):

Code: Select all

ifconfig eth1 -arp promisc 0.0.0.0 up 
ifconfig eth2 -arp promisc 0.0.0.0 up 
brctl addbr br0 
brctl addif br0 eth1 
brctl addif br0 eth2 
ifconfig br0 -arp promisc 0.0.0.0 up

One cable from switch to USB-RJ45 no.1, second cable from USB-RJ45 no.2 to the PC you want to monitor. Can do short-term surveillance (watch what is contacted while opening a browser etc.) or long-term surveillance (watch what your OS/apps try to contact whole day long...).

[JohnAdriaan]

...I use some raspis with 2 (two) USB-ethernet adapters in bridged mode running Wireshark (monitoring internet traffic of a PC as "pi-in-the-middle) using the built-in RJ45 to access the (headless) raspi via VNC.
quote]

Anazazi,

That's absolutely perfect: thank you very much! Exactly what I need. I'll use WiFi as the VNC/ssh port, so I'll only need one USB-ethernet adapter. It's the "-arp" option to ifconfig that I wasn't aware of.

Thanks again,

John

[pi-anazazi]

...raspis can be nice tools

I never tried a raspi 3... If I bridge the WIFI with the USB-RJ45, do I get a wifi sniffer?

[hippy]

My question is about how to configure Raspbian (using code or config files) so that the Ethernet ports that I'm using for that raw receive and transmit aren't also used for normal IP traffic.

If I undesrtand correctly: You want to be able to divert all the packets to your own program and prevent the protocol stack within the Pi passing those packets on to what would normally handle those -

Code: Select all

                            Tap    Break              .---
Ethernet  _    .--------.                 .----------/
=========|_|---| Buffer |----.-----/  /---| Dispatch |----
               `--------'    |            `----------\
                             |                        `---
                             |   .--------------.
                             `---| Your program |
                                 `--------------'
                                 

Unfortunately I have no idea how you would do that with a Pi or under LInux.

[JohnAdriaan]

If I bridge the WIFI with the USB-RJ45, do I get a wifi sniffer?

Sadly, according to this answer on RaspberryPi StackExchange that's not possible, due to a lack of information required. Also, you'd miss a lot of the WiFi-specific exchanges: only the Ethernet packets would be passed through.

[pi-anazazi]

...I usually have opensuse tumbleweed on the Wireshark raspis.

My Raspbian (stretch light with XFCE desktop) rapsis have a password for root and passwordless sudo disabled. I can't start Wireshark in this environment (installed it so that only root can run it). I tried

- gksudo -> wireshark

- gksudo -> dbus-launch wireshark

as well as from console with sudo or after su, nothing works...

Code: Select all

Client is not authorized to connect to ServerQXcbConnection: Could not connect to display :1.0 Aborted 

[pi-anazazi]

OK, got wireshark running on raspbian stretch light with xfce desktop and did a sniff on the WAN of my router (i.e. the two bridged USB-RJ45 in-line with the RJ45 cable connecting the fiber-to-RJ45 converter of my ISP and the WAN-port of my router).

To my surprise for the first time I saw some traffic originating from the USB-RJ45 connected to the ISPs converter! It was DCHP discover traffic:

With the MAC of the USB-RJ45 it was shouting to my ISP with source IP 0.0.0.0 to destination 255.255.255.255

(source port 68, destination port 67)

Code: Select all

dhcpcd-6.11.5:Linux-4.9.59-v7+:armv7l:BCM2835

...and the hostname for the built-in RJ45, which I use to control the Raspi 3 via VNC.

I never saw any traffic originating from the USB-RJ45s in the bridge in wireshark. Strange?

[pi-anazazi]

...with a second Raspi 3 (cloned SD-card from the other machine-in-the-middle) used on the LAN side of my router (only client in this LAN is a WIN 10 64bit machine on the other side of the Raspi) it's even worse:

The USB-RJ45 successfully requests an IP, get's an offer and acknowledges! Then it looses the IP after some time, requests a new IP and get's an offer, but never ack's.

Never saw this with Wireshark under opensuse tumbleweed for Raspi 3...

[pi-anazazi]

I switched the SD-card back to opensuse and the bootstrap DHCP chatter between the USB-RJ45 and the DHCP server of my router is gone. Raspbian stretch related?

[JohnAdriaan]

Raspbian stretch related?

It certainly sounds like it! Maybe you should raise a bug report?

[pi-anazazi]

...I've never done that and have no idea how to do. Maybe someone from Raspian finds this here and can make something out of it.

The USB-RJ45 are all four tested from this here:

https://www.amazon.de/UGREEN-Ethernet-N ... 00MYT47EA/

The MAC addresses all start with 00:0e:c6 which is AsixElec.

Using a normal Raspi 3 and Raspbian stretch light (updated to the latest) and Wireshark-GTK:

Code: Select all

 Version 2.2.6 (Git Rev Unknown from unknown)

Copyright 1998-2017 Gerald Combs <gerald@wireshark.org> and contributors.
License GPLv2+: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (32-bit) with GTK+ 3.22.11, with Cairo 1.14.8, with Pango 1.40.5, with
libpcap, with POSIX capabilities (Linux), with libnl 3, with GLib 2.50.3, with
zlib 1.2.8, with SMI 0.4.8, with c-ares 1.12.0, with Lua 5.2.4, with GnuTLS
3.5.8, with Gcrypt 1.7.6-beta, with MIT Kerberos, with GeoIP, with nghttp2
1.18.1, with PortAudio V19.6.0-devel, revision
396fe4b6699ae929d3a685b3ef8a7e97396139a4, without AirPcap.

Running on Linux 4.9.59-v7+, with locale en_GB.UTF-8, with libpcap version
1.8.1, with GnuTLS 3.5.8, with Gcrypt 1.7.6-beta, with zlib 1.2.8.

Built using gcc 6.3.0 20170516.

I see no such DHCP bootstrap traffic with the identical hardware, but opensuse Tumbleweed JeOS (with Enlightenment desktop) and Wireshark:

Code: Select all

Version 2.4.2 (v2.4.2)

Copyright 1998-2017 Gerald Combs <gerald@wireshark.org> and contributors.
License GPLv2+: GNU GPL version 2 or later <http://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Compiled (64-bit) with Qt 5.9.1, with libpcap, without POSIX capabilities, with
libnl 3, with GLib 2.54.1, with zlib 1.2.11, with SMI 0.4.8, with c-ares 1.13.0,
with Lua 5.1.5, with GnuTLS 3.6.0, with Gcrypt 1.8.1, with MIT Kerberos, with
GeoIP, without nghttp2, without LZ4, without Snappy, without libxml2, with
QtMultimedia, without AirPcap, without SBC, without SpanDSP.

Running on Linux 4.14.1-1-default, with 861 MB of physical memory, with locale
en_US.UTF-8, with libpcap version 1.8.1, with GnuTLS 3.6.1, with Gcrypt 1.8.1,
with zlib 1.2.11.

Built using gcc 7.2.1 20171005 [gcc-7-branch revision 253439].
 

[rurwin]

If you want an accurate test tap to check traffic, check out the Netgear ProSAFE Plus Switch GS105E. It does port mirroring. Amazon have it for £25.

As a solution to your problem in itself, it isn't as cheap but it doesn't slow down a gigabit link either. The v2 version has a web-browser interface, the v1 may require a Windows PC to configure it.

[pi-anazazi]

Hi rurwin!

Normally I used the raspi with Tumbleweed, so no problem. But I wanted to try Raspbian, as you can simply duplicate the SD-card with piclone, which doesn't work with Tumbleweed.

I normally monitor single clients, which by no means can use the gigabit bandwidth of their network adapters, anyway. And my ISP has no gigabit I just wanted to have an eye on my router/firewall, to see if strange traffic is originating from the device.

Many thanks for the suggestion!

[pi-anazazi]

PS:

Bridge-utils (brctl) are version 1.6-1.5-aarch64 on opensuse. On stretch light it's the latest inthe repo (updated yesterday...).

[pi-anazazi]

Just an update:

did yesterday a apt-get update &apt-get upgrade / dist-apgrade

Code: Select all

Linux rastretch1117 4.9.80-v7+ #1098 SMP Fri Mar 9 19:11:42 GMT 2018 armv7l GNU/Linux 

Started the bridge:

Code: Select all

ifconfig eth1 -arp promisc 0.0.0.0 up 
ifconfig eth2 -arp promisc 0.0.0.0 up 
brctl addbr br0 
brctl addif br0 eth1 
brctl addif br0 eth2 
ifconfig br0 -arp promisc 0.0.0.0 up

Now the two USB-Ethernet adaptors BOTH (although in the bridge) get an IPv4 IP address from the router, as I see in Wireshark listening on the bridge... It's a pain...