mikluz
Posts: 17
Joined: Thu Mar 31, 2016 11:52 am
Location: Szczecin, Poland

Dynamic web page - nginx as a server, what CGI to choose?

Wed May 25, 2016 11:15 am

Hi,

I want to publish my sensors' measurement on the web page that will run on my Raspberry Pi 3.
I have nginx already installed, and it's working fine. Standard static index.html site is displayed correctly.

Now, I want to go a step further and make it dynamic site which will take data from the sensors (assuming SQLite or MySQL database used) and display the data on the web page and refresh automatically when data has changed.

I had had some experience with basic C cgi-bin calls, but I remember it was extremely slow! So here come few questions from my side:
1. What is the best choice to use on Raspberry Pi as a CGI? Assuming I don't want to use old cgi-bin library...
1a. I heard of a such things like FastCGI and also other solutions that are not related to CGI techniques at all (like Embedded Javascript ot so on).
2. I want to use as much C code as possible, but I could also accept standard Bash scripting.
3. SQLite or MySQL. Which one is better? I just want to use fast database without any 3rd party user access, so I assume SQLite is the best choice.
4. I run nginx as a web server.
5. What charts API do you recommend. I've found this web page http://thenextweb.com/dd/2015/06/12/20- ... libraries/, so do you recommend Charist.js? Is it quick to learn and use? My constraint is to have all needed libraries/scripts offline accessed, so I don't want to use Google Charts...
6. Are there any tutorials/examples of the suggested CGI techniques (I've found a lot using PERL/PHP, but as I wrote at the beginning I want to limit myself to C/Bash/Javascript)? If so, please post them here.

Complex post with a lots of questions. Hopefully I will find someone helpful to guide me with my problems. Thanks! :)


I don't know if this post is placed in the proper place (which I really apologize, and if admin decides he/she can move my post to a proper place).
Raspberry Pi 3 @ Raspbian

ghans
Posts: 7325
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany

Re: Dynamic web page - nginx as a server, what CGI to choose

Wed May 25, 2016 4:32 pm

If you really want to stay with C at all costs i
suggest to at least look into FastCGI. Your program
practically runs in an endless loop and processes incoming
CGI requests , avoiding repeated startup for each request like
with traditional CGI.

Other languages offer much tighter integration (at least
Apache has mod_php and mod_wsgi).

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org

scotty101
Posts: 2246
Joined: Fri Jun 08, 2012 6:03 pm

Re: Dynamic web page - nginx as a server, what CGI to choose

Wed May 25, 2016 7:51 pm

Do you really want to use CGI? It's a bit 1990s!

There are so many modern frameworks for generating dynamic HTML.
Electronic and Computer Engineer
Pi Interests: Home Automation, IOT, Python and Tkinter

mikluz
Posts: 17
Joined: Thu Mar 31, 2016 11:52 am
Location: Szczecin, Poland

Re: Dynamic web page - nginx as a server, what CGI to choose

Fri May 27, 2016 9:24 pm

I'm open to anything, that is fancy, and easy-to-learn, as I had never been doing anything with PHP for instance...
I know standard old Javascript a bit, and AJAX that comes with it.

I also was told that there is something nice called "Web sockets". I will try to learn about it more. What do you think about that?
Raspberry Pi 3 @ Raspbian

User avatar
r3d4
Posts: 899
Joined: Sat Jul 30, 2011 8:21 am
Location: ./

Re: Dynamic web page - nginx as a server, what CGI to choose

Fri May 27, 2016 9:52 pm

websocketd ? ( github.com/joewalnes/websocketd ) # might be slow? idk.. github.com/joewalnes/websocketd/issues/204
websocketd.com wrote:It's like CGI, twenty years later, for WebSockets

User avatar
Gavinmc42
Posts: 1382
Joined: Wed Aug 28, 2013 3:31 am

Re: Dynamic web page - nginx as a server, what CGI to choose

Fri May 27, 2016 11:49 pm

I use HTML5 with javascript served by busybox-httpd, very basic.
CGI works with it and I mostly use shells scripts, if I need faster stuff, python, micropython, lua, luajit
I like to use interpreted languages because I can modify the code on the fly remotely, don't need to recompile or keep GCC on the Pi's.

For temperature plots I use dygraphs.js, it can read my sensor csv files directly.
With just a csv file saved for each day I don't need a database.
Did a check just yesterday, old Pi B, refreshed the web page, cpu usage jumped up to 1.8%.
Not exactly pushing them hard.

For fast real time data you need to use sockets.
Sockets allow a link to be kept open and only pass the small amount of data need to update the web page in the browser.
Javascript is probably a better choice and maybe use nodejs as the server.
Only need to learn one language then, played with nodejs a few years ago, since then my JS skills are better, might be worth looking at again.

You can write socket servers in a few languages.
Python comes with a SimpleHTTPserver example, micropython has one I used.
I can confirm websocketd works well and is simple to get going.

My next step is HTML5 with WebGL as a way of seeing a years worth of data in 3D.
WebGL should allow zooming and viewing the data much faster than any other way.
Assuming you run the browser on a suitable Desktop, not a Pi:)
This is the reason I keep data in csv files, don't need a database getting in the way.
Got two Pi's been logging every 5 mins 24/7 for a 18 months, still have 98% SD card memory free.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Heater
Posts: 7613
Joined: Tue Jul 17, 2012 3:02 pm

Re: Dynamic web page - nginx as a server, what CGI to choose

Mon May 30, 2016 7:32 pm

I'm not sure what you want to do but at the end of the day it comes down to:

1) A program on the Pi that can read sensors on a regular basis and perhaps store the data away in a database.

2) A web browser, external to the Pi, that can get data from 1) above. And perhaps have updates pushed to the display in real time.

The modern way to do this is to write the whole thing in Javascript, server side and browser side. JS is a natural for the browser, JS runs on the server under node.js just fine.

I have a simple example of such a thing here: https://bitbucket.org/zicog/pigpio2html

Now, the magic of nginx is that it can server a lot of static content, but it is also trivially easy to get it to forward requests to your application specific node.js server. Or many of them if you like.

Heater
Posts: 7613
Joined: Tue Jul 17, 2012 3:02 pm

Re: Dynamic web page - nginx as a server, what CGI to choose

Mon May 30, 2016 7:39 pm

If anybody wants to get into real-time data visualization in 3D you are going to need web sockets, best handled by socketio.js.

Also don't try to use webgl directly. Use three.js or babylon.js. Or if you only want accelerated 2D graphics use ivanko.js

mikluz
Posts: 17
Joined: Thu Mar 31, 2016 11:52 am
Location: Szczecin, Poland

Re: Dynamic web page - nginx as a server, what CGI to choose

Wed Jun 01, 2016 2:52 pm

Thanks Gavinmc42 for your reply :).

Also thanks to others that gave me useful hints.


I will try to use websockets and will try both ways, using precompiled programs that get data from the sensors and store it into a some-kind-of-database (as I want to get to know more about databases). And second way is to use simple scripts (written in Bash :), and maybe in Python - as Python is something that I need to learn).

Thanks again!
Raspberry Pi 3 @ Raspbian

Massi
Posts: 1599
Joined: Fri May 02, 2014 1:52 pm
Location: Italy

Re: Dynamic web page - nginx as a server, what CGI to choose

Wed Jun 01, 2016 3:29 pm

The modern way to do this is to write the whole thing in Javascript
web sockets, best handled by socketio.js
too many absolutes to be a point of view..

why socketio.js should be the best way to handle websockets?
and why node.js should be "modern"?

using websockets can be the "modern" way of doing what the op wants, js is only one of the dozilion ways of doing that

Return to “General programming discussion”

Who is online

Users browsing this forum: No registered users and 4 guests