Run Python nmap as Root OR let me run full nmap without root

[StaticDet5]

I'm looking to poll my network every minute (through cron), to get the list of MAC addresses associated to my network. I want to use Python and Python-nmap to do this. I've written a quick program, but I've learned that I can only get MAC addresses from nmap when it is run as root.

First, can I somehow tell my OS to allow nmap to be run by a non-root user?
Failing that, can I get my program to run as root, and have that carry over to the nmap requests within the program?
I'm trying to do this "correctly" and not have this thing running with full root privileges.

[bobstro]

Do you have to use nmap? You can could simply send ICMP pings then check the ARP table (e.g. ping x.x.x.x, arp -n), or generate ARP requests directly instead of using nmap. Although I've not used them myself, there are arp libraries for python.

[DougieLawson]

Try
/usr/sbin/arp -a -n
to print the active address resolution protocol table no polling needed.

[RaTTuS]

^ what Dougue says
also look at
ntop
iftop
for other options [different solutions to different problems ]
using cron to poll something every min is usually not what you want

[StaticDet5]

OK, I had some weird results yesterday using arp, but didn't have them this morning. I initially started using arp, but I didn't seem to get all the systems that were on the network, therefore I started looking at nmap.
It seemed like I wasn't getting complete results using arp, unless I ran an nmap scan before, (nmap -sP) over the subnet.

I'm pulling ntop and iftop down now, but they seem like overkill. I'm not interested in usage, only when a MAC associates and disassociates from the network (Specifically the wireless routers and wifi access points). I'm going to build in a flagging system to deal with intermittent signals when folks are still here but have interrupted connectivity.

What's the downside to doing this with cron? I think this is the first time I've heard someone say "Don't use cron for a repetitive task".

Thanks folks!

[DougieLawson]

What's the downside to doing this with cron? I think this is the first time I've heard someone say "Don't use cron for a repetitive task".

It's easy to get in a mess when cron triggers a second/third/fourth instance of your task before the first is complete. So you need to add some locking to prevent a new task starting.

In the general case starting a single task that includes a sleep or idle/no work to do wait makes life easier.

[bobstro]

OK, I had some weird results yesterday using arp, but didn't have them this morning. I initially started using arp, but I didn't seem to get all the systems that were on the network, therefore I started looking at nmap.
It seemed like I wasn't getting complete results using arp, unless I ran an nmap scan before, (nmap -sP) over the subnet.

That is how it should work. ARP entries are cached, and you're viewing that cache with the arp command. Until your Pi has a reason to resolve the Layer 3 IP address to a Layer 2 MAC address, there won't be an entry for an IP address. When you ping an IP address, if there's not already an entry mapping the IP address to a MAC address in your Pi's arp cache, It sends and ARP request ("who has ip X.X.X.X?") and if the device with that IP is alive, it will respond with an ARP reply ("I have X.X.X.X and here's my MAC address"). The result is cached for a period of time, typically a few minutes. You can watch this in action with Wireshark.

You can, of course, just sniff traffic to map IP addresses to MAC addresses, but that assumes you'll see every device communicating. On a switch, you aren't guaranteed to see activity from other devices unless you try to talk to them directly or they send multicast or broadcast traffic that your switch forwards, or you configure a SPAN or monitor port to see all traffic. The ping sweep ensures that your device's ARP cache will have a recent match for every live IP device by communicating directly with each one (unicast). Without that ping sweep or other unicast communication, you'll have to wait until each device sends broadcast or multicast traffic that the switch forwards to you. For long-term passive monitoring, this might be acceptable.

You certainly don't have to use nmap to do a ping sweep. You might want to check out the arpwatch package from the repository, although it too resides in /usr/sbin so will require root. Same for arping. Doing a simple ping sweep and checking the ARP table does not require root. arp -n or ip neighbor commands will work without root. Unfortunately, most modern device ignore ICMP ping broadcasts. Otherwise, you could use ping -b 255.255.255.255 to combine steps.

I'm still coming up to speed with python myself, but you should certainly be able to code a solution to do an ICMP ping request and check the response.

[StaticDet5]

OK, looking at it from a different direction, is there an elegant way to ask my gateway (The Fios Router) for it's ARP table?

Right now, the system that works involves two steps. I manually perform an "nmap -sP 192.168.1.1-254". I give it about 15 seconds, and then run my Python script. The script is coming back pretty quickly, identifying known MAC's, unknown but previously seen MAC's, new MAC's (I have people moving in to the house, so it's interesting to see them getting gadgets online), and other MAC's (Some of the MAC's are returning incomplete).

I know that nmap scans can be a little bulky. I know all the data that I really want resides on the DHCP Server/Gateway (in theory it's not, but I'm happy doing an nmap scan periodically to poll non-DHCP/non-gateway accessing systems.

I recognize my execution may be off, so if folks have a better idea, I'm happy to listen.

[bobstro]

Unless the router provides some sort of service to return its ARP table, probably not. Does it provide a web interface that you could scrape for information?

I'm still coming up to speed with python, but my understanding is that your script would need to run with root permissions to do the ICMP scans directly in python.

You could have your script simply call ping X.X.X.X command individually and then call the arp -n X.X.X.X command for the same address immediately. That way, you'd know the ARP cache entry would be fresh.

If your script ran with root permissions, perhaps as a daemon, life would probably be a lot easier and you could do it all in python. Then you'd just need your user script to check its results. A cheap trick would be to put your RPi on a hub or monitor/SPAN port on a switch between your monitored LAN and router and sniff that traffic. You'd at least see all Internet-bound traffic that way.

Somewhere on these boards I read about someone using this method to detect when the kids come home... or at least their phones do.

[StaticDet5]

That was the post article got me thinking about doing this. Ultimately I'd like the system to detect when someone has arrived at the house, possibly announcing their presence. Everyone has a cellphone, they almost always carry it, and they're constantly on the network when they are here. It will be truly cool if I can get an 802.11 point to listen in promiscuous mode, logging all MAC's that it comes in contact with.

I can ssh into the router, and from there I can't get anything to happen (besides authenticating my creds). I don't even know how to add programs to the box, nothing seems to work (I can list the directory and change directories. No arp, nothing else).

I think right now I'm just going to do fast ping sweeps. I've got a nice little inline tool on the way from Kickstarter. I may wait to use that before I try to optimize the discovery part of the process.

I thought about doing something inline, or messing around with the hub to see if it can set up one of the gigabit ports as a listening port. The problem I'm having is that the documentation Verizon provides with the hub is very rudimentary. I haven't had time to dig around too much (I did finally figure out how to restrict the DHCP range).

I've got my initial proof of code down for MAC scanning of the ARP table on the Pi. Next step is to start planning the persistence portion of the project. One of the things I'd like to be able to do is log presence and attributes, eventually being able to graph these items.

[bobstro]

There's nothing wrong with your ping-then-check-arp cache approach. I'm just tossing out some alternatives here:

If you're looking at a wireless network, there are a lot of variables. I'd suggest simply sniffing the wireless interface using tcpump or tshark/wireshark to see if what you can catch without a lot of work will suffice. (I had to do this on a Linux PC.) You might be able to see each device making broadcast/multicast announcements that will suffice for your needs. Some wireless access points have features to limit what clients can see, others let you see everything, so you probably won't know until you try. I always enjoy firing up wireshark on a plane.

Does the FIOS router support a bridged configuration? If so, you could put a Linux router behind it and just let the FIOS router pass traffic through. That way, you could run whatever scripts you want on the Linux router and this would all be trivial.

Another variation would be to use your RPi configured as an internal DNS server. I've got one configured at my house using dnsmasq acting as a server for both DNS and DHCP. DHCP is configured to point to my router as the default gateway, but my RPi for DNS. I use it for internal name resolution, print serving and the like. Using this, it's trivial to capture IP and MAC info for anything on my home network from DNS or DHCP logs. The possible issue here is if someone is actively trying to avoid you they could simply use an external IP address, but it doesn't sound like this is a likely scenario.

Is this only for use on your network, or do you want something that will work on any wireless network? Are you only worried about detecting visitors using wifi, or do you want to catch all mobile devices? What is the inline tool from kickstarter that you're referring to?

[StaticDet5]

I didn't even think to use the Pi as a DNS or DHCP server... That may be the best solution, combined with occasional nmap scans to verify static clients and find "rogue" systems.

I'm gonna set up wireshark. I need to get more familiar with it. There may be another elegant solution there.

This is strictly for a home network. It's going to sit on my lab bench while I play with it, possibly to be integrated into a bigger project. I initially want to sniff bluetooth signals, but that got very bulky and complex (I'd need bluetooth receivers throughout the house... pass). I probably have all of the gear to sniff all wireless signals, but there's some legal issues there, and I'm just not going to go down that road right now.

I'm going to dig deeper into the Fios Router angle. I've been able to get some detailed manuals from a friend of mine before. I'm waiting to see him again, and I'll hit him up. Someone is doing command line work with those things. The easy solution is for me to use the Router as my sensor, because it (currently) sees all traffic going to the internet (which all of the devices in the house are checking on when they get on to the network).

This is the cool little tool for inline applications:
https://www.kickstarter.com/projects/wa ... nav_search
I can build something similar using a Pi, but I don't know if I could bridge two gigabit ethernet USB adapters. The whole house is now wired for gigabit, and I don't want to kill their usage (the two girls are big WOW players, I'd like to keep them happy while I'm geeking around in the corner).

[bobstro]

To catch "everything", or at least everything that might talk to the Internet, you ideally want to be in the path out to the Internet. If you can set the router to bridge, putting together a Linux PC with dual gigE interfaces for your router/firewall is trivial. According to this DSL reports post, you can switch FIOS Actiontec routers to bridge mode, so this may be your best bet. It sounds as if a Cat. 5 Ethernet feed to completely eliminate the need for a FIOS router is possible if you're data only (no video), but otherwise you start breaking video things that your girls might care about.

I used a Linux box as my router/firewall for years and it's perfect for the late-night geek-fest sessions. I don't think the RPi would handle high throughput, never mind the challenge of getting 2 Gbps USB adapters going, but RPis are perfect little supplementary tools.

That gizmo is interesting, but it loses the stealth factor if you don't have PoE available. Personally, I'd put the $150 towards a switch with SPAN/monitor capabilities, as well as more ports and keep playing with all the capabilities Linux provides.

Until then, an RPi as DNS/DHCP server should work for plenty of experimentation. DEFINITELY spend time with Wireshark! It's an eye opener. You might also look into wireless probe packets to catch devices that aren't associated with your WLAN.

As long as your user query tools don't run as root, I wouldn't worry about a daemon running with root permissions. At least not in a home environment.

Be sure to get your girls off of WOW and into computers! My son was a WOW addict, but I swayed him to the joys of Open Source, and now he's a full-time developer.