Connect REST API on Pi to the Cloud

[mesand]

Hi all,

I have a project where I need to control things like pumps and read data from sensors. I currently have a flask webserver set up and can access a website that can control my pi from within my local network.

My objective is to change my flask webserver to a flask REST APi. Then I want to set up my raspberry pi on a cloud/iot platform and control my pi over the cloud via the flask REST api.

Any idea on how to best do this? Ive been researching it a lot and could use some help.

[deepo]

What do you need help with?
Getting outside access into your RPi on your local net?
Creating the API?
Creating an app that uses the API?

/Mogens

[mesand]

I need help with outside access to the RPi. I assume I might also need to create an app that uses the API, i dont know what the difference between an app and a webserver are. I have the API written with flask and can access and control my pi from my local network. I basically want to use the same API but with a login page so people can control the pi from outside the home network.

[deepo]

The problem of accessing your RPi from the internet is usually solved with "port forward" in your router, and dynamic DNS.
Port forward is used to forward requests to a specific port on the outside of your router to your RPi on the inside.
Dynamic DNS is used to map a fully qualified domain name to a changing IP address that most of us are issued from our ISP's.
But there are many ways to go about this, depending on how you're connected to the internet, so you'll need to read up on these subjects.

Next is the web API. I'm not familiar with Flask, but an API is usually a resource that you can manipulate with calls to the API.
A web API is different from a normal web page.
Think of it as the web page uses an API to display status of what the API controls, and allows a user to control what the API controls by displaying e.g. buttons, graphs etc.

/Mogens

[mesand]

great thank you. i am aware that I can access the pi from outside my home network via portforwading and dns. However, I would rather go a different route and connect to a cloud platform to be able to access the pi. what are your thoughts?

[Heater]

mesand,

...i dont know what the difference between an app and a webserver are.

The important difference between a web server and an "apps" or web pages is that one is a server and the others are clients.

They communicate using TCP/IP. That means that the clients connect to the servers and make requests. There can be many clients connecting to the server at the same time. The server does not initiate connection to the clients.

The practical upshot of this is that if you run your Flask web server on a Pi at home, on your local network, you need some things to make it accessible to apps and web browsers out in the world on the internet:

1) You need a "public" IP address for your home LAN that can be reached from users out in the world. Your Internet Service Provider (ISP) may or may not allocate you a public IP address. If yes, you are good to go. If not you have problems.

2) Once it is possible to access your home LAN from the outside, typically that is your internet router, you need to configure it to forward connections to your Flask web server on the Pi. So called "port forwarding". If your router allows this you are good to go. If not you have problems.

3) With all that in place it's generally better to have a domain name registered that can be used by outsiders to resolve a human memorable name to an IP address. You can get yourself a domain name very cheaply from people like godaddy.com. Having a domain name also helps when your public IP address changes, you just point the Domain Name System (DNS) at your new IP address. If you can do all that you are good to go. If not you have problems.

4) You will of course want your system to be secure when opened to the public internet. To that end you will need to tweak your Flask web server to use HTTPS. In order to use HTTPS you will need security certificates related to the domain name you have registered above. You can get those for free from letsencrypt.org. If you can do that you are good to go. If not you have problems.

5) Of course even with HTTPS your Flask web server is in no way secure unless you implement some kind of user login system with user names, passwords etc. If you can do that you are good to go. If not you have problems.

Are you sure you want to do all this?

Or take a short cut. Sign up for dataplicity.com. With their free service you can connect to your Pi from anywhere in the world with a web browser and get a command line shell access to it. Not only that, dataplicity can create a tunnel to allow access to your web server.

[Heater]

Oh crap, after I wrote all that you say this:

However, I would rather go a different route and connect to a cloud platform to be able to access the pi. what are your thoughts?

There are many cloud platforms you can use.

I mentioned dataplicity.com above. Works really well.

Then there is balena.io https://www.balena.io/ which I tried out many years ago when it was resin.io. Also works really well.

Now a days I rent a server instance on Digital Ocean. https://www.digitalocean.com/ It's my own Debain Linux machine in the cloud.

On that Digital Ocean server I run the NATS messaging system so that my Pi at home and elsewhere can exchange messages securely with it. https://nats.io/

I also run my web server, written in node.js or Rust on that Digital Ocean cloud machine. Then I can access my Pi through it from anywhere in the world.

You could move your Flask web server to such a cloud instance.

[deepo]

great thank you. i am aware that I can access the pi from outside my home network via portforwading and dns. However, I would rather go a different route and connect to a cloud platform to be able to access the pi. what are your thoughts?

Why didn't you say so then?

Usually your RPi will contact the could platform, as having the cloud platform connect to your RPI is almost always a royal pain.
MQTT could be a solution I think.

/Mogens

[rpdom]

Now a days I rent a server instance on Digital Ocean. https://www.digitalocean.com/ It's my own Debain Linux machine in the cloud.

On that Digital Ocean server I run the NATS messaging system so that my Pi at home and elsewhere can exchange messages securely with it. https://nats.io/

I have a DO server, as well as a Vultr server. Both run the tinc VPN service which I connect to from a Pi at home. This extends my home subnet to both servers. I can then connect from anywhere else to either/both servers also using tinc and get an IP address on my home LAN. I have actually got it set do that my laptop gets the same 192.168.42.56 IP address when I'm at home or connected to the VPN.

I prefer to run my own VPN rather than relying on a third party connection.

Also, I don't need to do any port forwarding on my home network.

[KMyers]

Depending where you are located you might consider Adafruit IO in the USA, Good basic tutorials and its free unless you upgrade to more feeds etc.

Hi @Heater, miss you on Parallax but I get it.

[mesand]

great thank you for all your suggestions. dataplicity seems like a good route. When you say "tunnel" to my local webserver. Does that involve poking any holes or giving dataplicity access to manipulate my homes local firewall?

[mesand]

it looks like the word they use on dataplicity is "wormhole"

[Heater]

KMyers,

Hi @Heater, miss you on Parallax but I get it.

Why thank you for saying so.

How time flies. I was just this week thinking it's been the best part of a year since I attenteded to the Parallax forum. I was thinking I should pay a visit and ask "Is it Chipmas yet?"

What with spending a lot of time tinkering with Verilog, SpinalHDL, FPGA and a home brew RISC V. And now Rust. Oh and starting a new company and dealing with the consequent legal action by our former employer. It's been a busy year!

Perhaps see you over there soon.

[Heater]

mesand,

When you say "tunnel" to my local webserver. Does that involve poking any holes or giving dataplicity access to manipulate my homes local firewall?

No.

The deal is that you run a Python program as a daemon on your Pi. It connects to the Dataplicity servers using HTTP I believe. Same as any web browser on any computer you have at home.

With that connection Dataplicty can "tunnel" a terminal shell connection from your Pi, through their servers to your browser where ever you happen to be.

Optionally they can tunnel HTTP connections to some special URL from your browser, through their servers, to your Pi at home.

[mesand]

ok great, thank you. I assume it is also possible to be able to include https messaging and ssl certificates for more security?

[Heater]

You don't need to.

The connection between your Pi and Dataplicity is as secure as anything.

The connection from your web browser to Dataplicty uses HTTPS already.

Question might be, how much do your trust a third party like Dataplicity or the many others in the IoT space today?

If you don't trust. Or what you are doing is super critical then you should look to something else.

Of course anything you make for yourself is likely to be less secure!

[mesand]

Yea true, I wont be dealing with sensitive information or anything like that. My main worry is someone being able to hack into my whole network through something like dataplicty.

[Heater]

Like I said. You are going to have to trust somebody.

Or make your own system. Which will likely be less secure.

You could always get a dedicated connection to the internet for your Pi(s). To keep then away from the rest of your home network.

And pay your ISP accordingly.

We use 3/4G mobile connections for that some times.

[mesand]

genuinely, thank you all for the great suggestions you have helped me greatly.

[mesand]

Would you all recommend I use a REST api or a regular local webserver to connect to dataplicity

[Heater]

If you are going to use dataplicity you will be running your web server on your Pi at home. With whatever web pages it serves up as a normal web server does.

The dataplicity "worm hole" makes that web server accessible to the outside world. For example, here is the Pi on my desk just now with a message for you: https://presageful-elephantseal-6438.dataplicity.io/

You will also be serving whatever REST API you need from that same Pi.

Really, there is no difference between a web page server and a REST API server. It's all done via the same HTTP/HTTPS. Except one may respond with HTML that can be used to render a web page in your browser. And the other will likely respond with JSON or XML or whatever that is intended to be consumed by some program other than a web browser.

[KMyers]

Good luck Heater and Happy Holidays. Its the same ol same old there but some P2.s ln wild.
Sorry for interrupting your conversation.

Kem N8SYG

[Heater]

N8SYG,

And happy holiday to you. Good to know the P2 is a thing. Now I gotta check it out!