mondalaci
Posts: 5
Joined: Fri Jan 03, 2020 6:40 pm

Self-hosted, NAT-friendly IoT fleet management?

Fri Jan 03, 2020 7:52 pm

I have a growing number of Pis behind NAT routers, and I'm looking for a self-hosted solution to manage them.

I want to access the Pis without making network infrastructure changes such as port forwards and VPNs, partly because I don't want to deal with network configuration and partly because I don't have access to the networking gear of every network my Pis will inhabit.

In specific, I'm interested about the presence (online/offline status) of the individual Pis, and want to be able to reach their ports (primarily SSH and VNC) via reverse SSH tunnels. I'm thinking of launching a project to implement a client-server application in node.js. I'd use websockets for client-server communication because of their lightweight and real-time nature. This way, the clients would be able to report their presence, and the server would be able to request tunnels to be opened.

Wondering whether you guys know something like the above, because I don't want to reinvent the wheel.

nomdediot
Posts: 52
Joined: Sat Feb 02, 2013 3:04 pm

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 7:31 am

Already worked on this kind of project, using some https request instead of ws socket, but basically the same idea. We made custom request and response to have information from the système (online, internal statuses, and specific responses to opn ssh tunneling with user and password given in the response).

I m not aware of existing solutions to do it.

Nodejs is great idea.
Last edited by nomdediot on Sat Jan 04, 2020 2:24 pm, edited 1 time in total.

plugwash
Forum Moderator
Forum Moderator
Posts: 3723
Joined: Wed Dec 28, 2011 11:45 pm

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 10:01 am

A VPN doesn't have to involve "network infrastructure changes", you can just run a VPN server on the internet and then have all your Pis connect to it.

User avatar
rpdom
Posts: 18002
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 12:56 pm

plugwash wrote:
Sat Jan 04, 2020 10:01 am
A VPN doesn't have to involve "network infrastructure changes", you can just run a VPN server on the internet and then have all your Pis connect to it.
Exactly. I use TINC for my local VPN. I have a Virtual server I pay about $5 a month to run (which does other things as well). On of my Hi Pi connect to ttinc on that server and I can then connect to that server from anywhere also using tinc and I'm connected to my home subnet.

tinc is slightly different to most VPNs in that every system using it is both server and client. It will route traffic through any connection it can find through the systems that it knows about and will learn about new systems as they are added.
The setup could be a little easier, but I'm running it just fine on a Pi 1B and a couple of VPS.
Unreadable squiggle

mondalaci
Posts: 5
Joined: Fri Jan 03, 2020 6:40 pm

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 9:22 pm

Thanks guys, but in my experience, VPNs are generally a pain to setup, so I'd much rather use a zero effort, no brainer solution like the one I suggested. I'll share my progress here.

User avatar
rpdom
Posts: 18002
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 9:40 pm

No worries, but your method sounds more complicated than the one I use. My configuration files are only a few lines long in most cases, although there are a few of them per system.

I'll admit it did take a little trial and error to get it working correctly, but now that it does it seems to be pretty bullet proof.
Unreadable squiggle

mondalaci
Posts: 5
Joined: Fri Jan 03, 2020 6:40 pm

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 9:54 pm

Would you mind sharing your config, rpdom? I'd like to see how much effort this would take.

User avatar
rpdom
Posts: 18002
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 10:40 pm

I'll have a look and hide any private bits.

For example, the main config file on my "Debian with Raspberry Pi Desktop" laptop is essentially

Code: Select all

# tinc configuration for laptop

Name = laptop
ConnectTo = server1
ConnectTo = server2
Mode = switch
There are also some config files for each of the two servers which are like

Code: Select all

# Host config

Address = xxx.xxx.xxx.xxx

-----BEGIN RSA PUBLIC KEY-----
(not going to show you my public key either)
-----END RSA PUBLIC KEY-----
and a tweak at the end of /etc/dhcpcd.conf to get the interface and routing set correctly

Code: Select all

# For VPN
interface rpdom
nogateway
static ip_address=192.168.42.55/24
static domain_name_servers=192.168.42.30 208.67.222.222 208.67.220.220
There was a little more coding needed on the Pi that bridges it into my home network. I ended up using systemd's networking to set up the bridge (although I can see some legacy code that can probably be got rid of. I'll have to look at that.).
What I really should do is to set the whole thing up again from the beginning, just to make sure I got it right.
Unreadable squiggle

mondalaci
Posts: 5
Joined: Fri Jan 03, 2020 6:40 pm

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 11:17 pm

Thanks for sharing! What I'm afraid of are all the little details that you didn't share. In my experience, it can take an eternity to figure out all the things to make this work really well. Feel free to follow up with the details any time, possibly by linking a GitHub project that contains all the files and setup instructions. Interested in giving this a try!

User avatar
dividuum
Posts: 229
Joined: Sun Jun 16, 2013 1:18 pm
Location: Germany
Contact: Website

Re: Self-hosted, NAT-friendly IoT fleet management?

Sat Jan 04, 2020 11:46 pm

mondalaci wrote:
Fri Jan 03, 2020 7:52 pm
I want to access the Pis without making network infrastructure changes such as port forwards and VPNs, partly because I don't want to deal with network configuration and partly because I don't have access to the networking gear of every network my Pis will inhabit.
Have a look at https://www.zerotier.com/

It create a virtual network adapter, automatically assigns an internal IP that allows you to reach all devices within the same virtual network without any manual setup. It's a self-organising overlay network, uses encryption for that and solves all the NAT traversal issue and other annoyances automatically for you. I toyed around with it a bit and it works pretty well. You can self-host the master server from what I understand but I've only used the provided instance at zerotier.com. Be aware of their licensing model.
info-beamer hosted - A user and programmer friendly digital signage platform for the Pi: https://info-beamer.com/hosted

User avatar
rpdom
Posts: 18002
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Self-hosted, NAT-friendly IoT fleet management?

Sun Jan 05, 2020 9:20 am

mondalaci wrote:
Sat Jan 04, 2020 11:17 pm
Thanks for sharing! What I'm afraid of are all the little details that you didn't share. In my experience, it can take an eternity to figure out all the things to make this work really well. Feel free to follow up with the details any time, possibly by linking a GitHub project that contains all the files and setup instructions. Interested in giving this a try!
I'll have a go at that. What I don't want to do is to stop my current set up while it is in use, so I'll set up a second network of Pis at home and see if I can make step by step instructions of how to do it. When I get some time. I might have to use another external server for a while, but that won't cost too much if I only use it for a short time. I can get one on a cheap hourly rate.
Unreadable squiggle

Return to “Networking and servers”