SunMarc
Posts: 4
Joined: Wed Mar 15, 2017 8:57 am

Simplest temperature data logger

Fri Sep 27, 2019 7:41 pm

Hi,

I want to create a data logger, with charts, but the simplest way (no php and mysql) to monitor 4 temperatures (DS18B20)

The data (date, temperatures) will be saved on a file on SD Card
Then it will be shown as chart on a webpage (in real time if possible)

What would you recommand for this ?
Hicharts, nodejs?
Any example?

Thanks

gordon77
Posts: 4239
Joined: Sun Aug 05, 2012 3:12 pm

Re: Simplest temperature data logger

Fri Sep 27, 2019 7:54 pm

Here's an example using pi and Arduino...

arduino code (you'll need to interface the sensors to analog 1-4, or add code.)

Code: Select all

    Serial.begin( 9600 );}
void loop() {
    int inByte1 = 'a';
    float analog1 = 0;
    float analog2 = 0;
    float analog3 = 0;
    float analog4 = 0;
    if (Serial.available()>0) {
        inByte1 = Serial.read(); 
        if (inByte1 == 'A') {
             analog1 = analogRead(1);
             Serial.print('A');Serial.print (char(44));
             Serial.print(analog1);
             Serial.write("\n");
        }
        if (inByte1 == 'B') {
             analog2 = analogRead(2);
             Serial.print('B');Serial.print (char(44));
             Serial.print(analog2);
             Serial.write("\n");
        }
        if (inByte1 == 'C') {
             analog3 = analogRead(3);
             Serial.print('C');Serial.print (char(44));
             Serial.print(analog3);
             Serial.write("\n");
        }
        if (inByte1 == 'D') {
             analog4 = analogRead(4);
             Serial.print('D');Serial.print (char(44));
             Serial.print(analog4);
             Serial.write("\n");
        }
        if (inByte1 == 'X') {
             analog1 = analogRead(1);
             analog2 = analogRead(2);
             analog3 = analogRead(3);
             analog4 = analogRead(4);
             Serial.print('X');Serial.print (char(44));
             Serial.print(analog1);Serial.print (char(44));Serial.print(analog2);Serial.print (char(44));Serial.print(analog3);Serial.print (char(44));Serial.print(analog4);
             Serial.write("\n");
        }
    }
 } 
and pi code, which stores the data and makes a graph....

Code: Select all

#! /usr/bin/env python3
import serial
import time
import os
import threading
import queue
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from matplotlib import style
import datetime

if os.path.exists('/dev/ttyACM0') == True:
    ser = serial.Serial(port='/dev/ttyACM0',baudrate = 9600,parity=serial.PARITY_NONE,
    stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)
if os.path.exists('/dev/ttyACM1') == True:
     ser = serial.Serial(port='/dev/ttyACM1',baudrate = 9600,parity=serial.PARITY_NONE,
     stopbits=serial.STOPBITS_ONE,bytesize=serial.EIGHTBITS,timeout=1)

if os.path.exists('/run/shm/example.txt'): # note log currently saved to RAM
   os.remove('/run/shm/example.txt')       # and gets deleted as you start the script
   
def thread_read():
   axis = 0
   global ser
   while True:
       a = b = c = d = 0
       time.sleep(1.0)
       # send an X to the arduino to read ALL temps
       ser.write(b"X")
       #read data from arduino
       myData2 = ser.readline()
       myData2 = myData2.decode("utf-8","ignore")
       if len(myData2) > 0 and myData2[0] == "X":
          x, e, f, g, h = myData2.split(',')
          a = float(e)
          b = float(f)
          c = float(g)
          d = float(h)
       if a > 0 and b > 0 and c > 0 and d > 0:
           now = datetime.datetime.now()
           timestamp = now.strftime("%y/%m/%d-%H:%M:%S")
           with open('/run/shm/example.txt', 'a') as f:
               f.write(timestamp + "," + str(axis)+"," + str(a) + "," + str(b) + ","  + str(c) + "," + str(d) + "\n")
       axis +=1

def thread_plot():
       global fig,animate, ax1
       fig = plt.figure()
       ax1 = fig.add_subplot(1,1,1)
       ani = animation.FuncAnimation(fig, animate, interval=1000)
       plt.show()
   
def animate(i):
   if os.path.exists('/run/shm/example.txt'):
    graph_data = open('/run/shm/example.txt','r').read()
    lines = graph_data.split('\n')
    xs = []
    ys = []
    zs = []
    js = []
    ks = []
    for line in lines:
        if len(line) > 1:
            t, x, y, z, j, k = line.split(',')
            xs.append(float(x))
            ys.append(float(y))
            zs.append(float(z))
            js.append(float(j))
            ks.append(float(k))
    ax1.clear()
    plt.xlabel('Time Secs') 
    plt.ylabel('Temp degC')
    plt.xlim(1,100)
    if int(x) > 100:
        plt.xlim(int(x)-100,int(x))
    ax1.plot(xs, ys)
    ax1.plot(xs, zs)
    ax1.plot(xs, js)
    ax1.plot(xs, ks)

read_thread = threading.Thread(target=thread_read)
read_thread.start()

plot_thread = threading.Thread(target=thread_plot)
plot_thread.start()

Attachments
graph.jpg
graph.jpg (108.41 KiB) Viewed 537 times
Last edited by gordon77 on Sun Sep 29, 2019 9:14 am, edited 4 times in total.

Andyroo

Re: Simplest temperature data logger

Fri Sep 27, 2019 7:59 pm

Python for everything?

Python can do the interfacing to the sensor, use Flask and Pygal to draw the graphs and host the web page.

If you do not want to use MySQL or MariaDB then you could use flat files, json or even SQLite to hold the data.

User avatar
DougieLawson
Posts: 36327
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Simplest temperature data logger

Fri Sep 27, 2019 8:00 pm

I've got a server collecting temperature data in an SQLite3 database every three minutes.

So I wrote a C++ program to run frequently that extracts the last 24 hours worth of data and push that out on a MQTT topic.
dbread.cpp

Code: Select all

#include "DBLite.hpp"

int main()
{
        DBLite sensor;
        //cout << "{\"bmp_data\": [" << endl;
        sensor.showTable();
        sensor.closeDB();
        //cout << "]}" << endl;

        return 0;
}
DBLite.hpp

Code: Select all

#include <iostream>
#include <string>
#include <sstream>
#include <cstdlib>
#include <sqlite3.h>
#include "mqttMessage.hpp"

using namespace std;

class DBLite {

    private:
        sqlite3 *db;
        char *zErrMsg;
        int rc;
        char* sql;
        sqlite3_stmt *stmt;

        static int callback(void* NotUsed, int argc, char **argv, char **azColName)
        {
            int hour = 0;
            stringstream sOutput;
            stringstream sColName;
            stringstream sColValue;
            stringstream sRowNum;
            string colName;
            string tempValue;
            string dateTimeValue;
            string hourValue;
            string rowValue;
            int rowNumValue;
            int rowNum;
            for(int i = 0; i < argc; i++)
            {
                sColName << azColName[i];
                sColValue.str(argv[i]);
                sColName >> colName;

                if (colName.compare("rownum") == 0)
                {
                    sColValue >> rowNumValue;
                    rowNum = int((rowNumValue+1)/20);
                    sRowNum << rowNum;
                    rowValue = sRowNum.str();
                }
                if (colName.compare("date_time") == 0)
                {
                    sColValue >> tempValue;
                    sColValue >> dateTimeValue;
                    if (dateTimeValue.compare(2, 4, ":00:") == 0)
                    {
                        hour = 1;
                        hourValue = dateTimeValue.substr(0,2);
                    }
                }

                if (colName.compare("temp") == 0 && hour == 1)
                {
                    sColValue >> tempValue;
                    string sOutputStr;
                    sOutput << "{\"rownum\":" << rowValue;
                    sOutput << ", \"hour\": " << hourValue;
                    sOutput << ", \"temp\": " << tempValue << "}," << endl;
                    hour = 0;
                    sOutputStr += rowValue+":"+hourValue+":"+tempValue+"\n";

                    mqttMessage tempMQTT("tempMQTT", "dbread/out/temp", "10.1.1.11", 1883);
                    tempMQTT.send_message(sOutputStr.c_str());
                    //cout << sOutput.str();
                 }
                 sColName.str(std::string());
                 sColValue.str(std::string());
                 sOutput.str(std::string());
                 sColName.clear();
                 sColValue.clear();
                 sOutput.clear();
             }
             return 0;
         }

         void checkDBErrors()
         {
             if( rc )
             {
                 cout << "DB Error: " << sqlite3_errmsg(db) << endl;
                 closeDB();
             }
         }

         void initDB()
         {
             rc = sqlite3_open("/srv/bmp180/sensordata.db", &db);
             checkDBErrors();
         }

    public:
        DBLite()
        {
            initDB();
        }

        void showTable()
        {
            sql = (char*)"SELECT\
                   row_number() over( order by date_time) rownum,\
                    date_time, temp FROM bmp_data\
                          WHERE date_time BETWEEN datetime('now','-1440 minutes')\
                          AND datetime('now') ;";
            rc = sqlite3_exec(db, sql, callback, NULL, &zErrMsg);
        }

        void closeDB()
        {
            sqlite3_close(db);
        }
};
To graph the data I'm using a pythonn program wiht matplotlib and mpld3 (which writes HTML)

Code: Select all

#!/usr/bin/python3

#!/usr/bin/python3
import numpy as np
import matplotlib.pyplot as plt
import mpld3

import paho.mqtt.client as mqtt
import sys
from time import sleep

def on_connect(mqtts, obj, flags, rc):
  print("Conn rc: "+str(rc))

def on_publish(mqtts, obj, mid):
  print("mid: "+str(mid))

def on_message(mqtts, obj, msg):
  global mqttp, broker, port, x, y

  #print ("Message received on topic: "+str(msg.topic)+" with QoS: "+str(msg.qos)+" and payload: "+str(msg.payload))
  pl = msg.payload.decode('utf-8')
  ii, xxx, yyy = pl.split(':')
  ix = int(ii)
  xx = int(xxx)
  yy = float(yyy)
  x[ix] = xx
  y[ix] = yy

  if ix == 23:
    graph = plt.figure()
    plt.ylabel('Temp')
    plt.xlabel('Time')
    obj = plt.plot(x, y, 'ro-')
    mpld3.save_html(graph,'/home/dougie/public_html/nmplot.html')
    plt.close(graph)

  sleep(1)

def on_subscribe(mqtts, obj, mid, granted_qos):
  print("Subscribed: "+str(mid)+" "+str(granted_qos))

def on_log(mqtts, obj, level, string):
  print(string)

def main():

  global mqttp, broker, port, x, y
  broker = "10.1.1.11"
  port = 1883

  x = [0,0,0,0,0,0,
       0,0,0,0,0,0,
       0,0,0,0,0,0,
       0,0,0,0,0,0]
  y = [0,0,0,0,0,0,
       0,0,0,0,0,0,
       0,0,0,0,0,0,
       0,0,0,0,0,0]

  mqtts = mqtt.Client("dbr_sub")

  mqtts.on_message = on_message
  mqtts.on_connect = on_connect
  mqtts.on_publish = on_publish
  mqtts.on_subscribe = on_subscribe

  mqtts.connect(broker, port, 60)
  mqtts.subscribe("dbread/out/+", 0)

  rc = 0
  while rc == 0:
    rc = mqtts.loop_forever()

  return 0

if __name__ == "__main__":
   sys.exit(main())
The data collection stuff is on Github at: https://github.com/DougieLawson/Raspber ... ter/bmp180 (I'll add those three pieces to the GitHub stuff).
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

User avatar
neilgl
Posts: 1127
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near Aston Martin factory

Re: Simplest temperature data logger

Fri Sep 27, 2019 8:03 pm

I would go for python to read the temperatures, rrdtool to log and create graphs, then apache2 to display them on a web page.

User avatar
jcyr
Posts: 459
Joined: Sun Apr 23, 2017 1:31 pm
Location: Atlanta

Re: Simplest temperature data logger

Sat Sep 28, 2019 3:11 am

C code to read DS18B20 sensor(s) directly with PI pin 7.

Code: Select all

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

const char* therm_file_name = "/sys/bus/w1/devices/28-000008d2287a/w1_slave";
// 28-000008d2287a is the sensors serial #, each sensor's will be different
// needs 'dtoverlay=w1-gpio' in /boot/config.txt to enable 1-wire protocol.

float get_temp(void)
{
    int fd = open(therm_file_name, O_RDONLY);
    char buf[128];
    char* temp;
    if (fd < 0)
        return -1;
    if (read(fd, buf, sizeof(buf)) < 0)
        return -1;
    temp = strchr(buf, 't');
    sscanf(temp, "t=%s", temp);
    close(fd);
    return atof(temp) * .0018 + 32;
}
As for the DB and Web stuff... not my thing.
It's um...uh...well it's kinda like...and it's got a bit of...

jbudd
Posts: 1020
Joined: Mon Dec 16, 2013 10:23 am

Re: Simplest temperature data logger

Sat Sep 28, 2019 9:02 am

Depends what you mean by simplest. Simplest to program or the least programs/applications to run it.

I would say the simplest way to program it would be with Node-Red to interrogate your sensors and store the data in InfluxDb - designed for easy use with time related data. You can easily display a graph of recent readings with the node-red dashboard plugin too. No need to install a web server (I think). You might need a few lines of JavaScript, otherwise no coding required.
More snazzy dynamic graphs can be displayed via Grafana.

tpyo kingg
Posts: 627
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: Simplest temperature data logger

Sat Sep 28, 2019 9:46 am

I'd say python with cron for the temperatures and then Apache2 to access the logs.

My temperature system uses python to read the temperature sensors through cron and add the readings to a plain text log file. That is supplemented with a couple of shell scripts to split the log file in different ways. All three resulting files are plain text served up by standard Apache2, and the normal logrotate utility is configured to deal with the old data. If I were to have it do charts too then I'd probably just throw in GnuPlot or similar. It's hard to beat standard tools like those for simplicity.

For a while, I had mine writing the current time and temperature to an e-paper display. However I repurposed that for a different activity. So now it is just using Apache2.

During the winter I have it in our storage area monitoring plant temperatures. I should probably make another one for elsewhere year round because it has been so useful to have.

User avatar
bensimmo
Posts: 4184
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Simplest temperature data logger

Sat Sep 28, 2019 9:59 am

If you want to keep it drag and drop, you can log and chart DS18B20's in Scratch on the Pi
;-)

geektechstuff.com
Posts: 17
Joined: Sat Mar 02, 2019 8:08 pm
Contact: Website

Re: Simplest temperature data logger

Sat Sep 28, 2019 2:12 pm

SunMarc wrote:
Fri Sep 27, 2019 7:41 pm
Hi,

I want to create a data logger, with charts, but the simplest way (no php and mysql) to monitor 4 temperatures (DS18B20)

The data (date, temperatures) will be saved on a file on SD Card
Then it will be shown as chart on a webpage (in real time if possible)

What would you recommand for this ?
Hicharts, nodejs?
Any example?

Thanks
I did something similar (minus the chart) with Python and Flask here: https://geektechstuff.com/2019/02/05/in ... pi-python/
www.geektechstuff.com

SunMarc
Posts: 4
Joined: Wed Mar 15, 2017 8:57 am

Re: Simplest temperature data logger

Sat Oct 05, 2019 2:18 pm

Thank you !
May be there is a simpler solution, like this one on ESP ? https://circuits4you.com/2019/01/11/esp ... me-graphs/

User avatar
neilgl
Posts: 1127
Joined: Sun Jan 26, 2014 8:36 pm
Location: Near Aston Martin factory

Re: Simplest temperature data logger

Sat Oct 05, 2019 2:46 pm

Ah, we thought you wanted to do it on a raspberry pi ? Anyway, where does that esp8266 version store the data, lets say for the last 2 years?

User avatar
rin67630
Posts: 568
Joined: Fri Mar 04, 2016 10:15 am

Re: Simplest temperature data logger

Sat Oct 05, 2019 4:50 pm

gordon77 wrote:
Fri Sep 27, 2019 7:54 pm
Here's an example using pi and Arduino...
arduino code (you'll need to interface the sensors to analog 1-4, or add code.)
...
You don't even need to program anything on the Pi side. Just open the serial plotter on the Arduino IDE, and enjoy the magic.
But you will not have it on a web page. Just have it on VNC.

To be closer to what the OP wants, the easiest approach would have been Python with DS18B20 and thinger.io libraries to upload the data to a thinger.io dashboard. (Thinger.io being just an example, many other IOT dashboards exist).

I had done it with an ESP32 since I got analog values to process, but a Pi Zero with DS18 on a TWI bus are a great choice.

User avatar
bensimmo
Posts: 4184
Joined: Sun Dec 28, 2014 3:02 pm
Location: East Yorkshire

Re: Simplest temperature data logger

Sat Oct 05, 2019 6:47 pm

For web display, just install Cayenne and it sort it out for you with DSB18's

Return to “General discussion”