User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Trying to Learn Ploticus - need help

Tue Apr 28, 2015 12:50 pm

Hi all,

I have been following a fellow pi users instructions to setup a temperature logger which logs to a sqlite DB (works great), and then use Ploticus to create a graph based on the data logged to the DB.

The code I have been given is as follows:

Code: Select all

sqlite3 -csv temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 y2=4 y3=5 y4=6 xinc="3 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="°C" autodays=yes yrange="0 100" xgrid=yes ygrid=yes name="Value" name2="Panel Out" name3="HWS Out" name4="Loft" xmargin=0  > /dev/null 2>&1

Code: Select all

sudo cp 24h.png /usr/share/nginx/www/
the above code does infact create a png file, but it has no date plotted, just an empty 'template' formatted as per the code I can see.

I am curious if someone knows how to help me point ploticus to look at my temerature.db, and specifically my table called 'PivotTemps10min', and plot the field Demo_Thermometer

I understand about 50% of that ploticus code, but I do not understand how it is reading the data from the DB (i assume it is not, hence the empty output).

Cheers
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:02 pm

Near the beginning is a pipe symbol "|"

Pipes in Unix are cool. They allow you to connect the output of one program to the input of another. In your example code, ploticus is not reading from the database, but it is being fed data from an SQLite query.

So, the first part is this:

Code: Select all

sqlite3 -csv temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-24 hour');"
That should dump a csv file to the screen (aka stdout). If that is empty then there is no data to be plotted.

After the pipe symbol is the ploticus command. One of the parameters is "data=-", which means read from stdin. The pipe symbol basically pipes data from stdout of one program to stdin of the next one.

I suppose if the first part is not producing any data then you need to find out why. Run it on its own and see what you get.

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:08 pm

ame wrote:Near the beginning is a pipe symbol "|"

Pipes in Unix are cool. They allow you to connect the output of one program to the input of another. In your example code, ploticus is not reading from the database, but it is being fed data from an SQLite query.

So, the first part is this:

Code: Select all

sqlite3 -csv temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-24 hour');"
That should dump a csv file to the screen (aka stdout). If that is empty then there is no data to be plotted.

After the pipe symbol is the ploticus command. One of the parameters is "data=-", which means read from stdin. The pipe symbol basically pipes data from stdout of one program to stdin of the next one.

I suppose if the first part is not producing any data then you need to find out why. Run it on its own and see what you get.
Ah ok, I have wondered what the pipe did... makes sense.

So i ran the standalone sqlite query and it returned tons of lines from the database, pasted below is the last dozen lines:

Code: Select all

"2015-04-28 21:00:00",2015-04-28.21:00:00,,,,,,,22.3246
"2015-04-28 21:10:00",2015-04-28.21:10:00,,,,,,,21.0811
"2015-04-28 21:20:00",2015-04-28.21:20:00,,,,,,,22.5621428571429
"2015-04-28 21:30:00",2015-04-28.21:30:00,,,,,,,21.3185
"2015-04-28 21:40:00",2015-04-28.21:40:00,,,,,,,22.5998
"2015-04-28 21:50:00",2015-04-28.21:50:00,,,,,,,21.5684
"2015-04-28 22:00:00",2015-04-28.22:00:00,,,,,,,22.7934
"2015-04-28 22:10:00",2015-04-28.22:10:00,,,,,,,21.7622
"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
"2015-04-28 22:30:00",2015-04-28.22:30:00,,,,,,,20.9748
"2015-04-28 22:40:00",2015-04-28.22:40:00,,,,,,,20.8185
"2015-04-28 22:50:00",2015-04-28.22:50:00,,,,,,,20.6374
"2015-04-28 23:00:00",2015-04-28.23:00:00,,,,,,,20.5623333333333
Note there are about 6 empty columns as I have not connected all sensors to the Pi yet, I am still in development phase so just trialing things before getting technical with 7 thermometers.

So its safe to assume that the problem lies within my ploticus code then?

It is outputting a blank graph (see attached), so do we assume it is successfully running the entire command? i just have something set incorrectly with the data?
24h.png
output with no data
24h.png (1.18 KiB) Viewed 2310 times
is the thin horizontal black line running along the X axis actually plotted data? Or just the graph axis?
Last edited by some_evil on Tue Apr 28, 2015 1:11 pm, edited 1 time in total.
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

User avatar
DougieLawson
Posts: 39302
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:10 pm

Is there a link to the stuff you've been following?

I've got a database of room temp and air pressure (since 10th Feb (minus four hours when I used the sensor for the eclipse)) that might be a candidate for plotting.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:13 pm

DougieLawson wrote:Is there a link to the stuff you've been following?

I've got a database of room temp and air pressure (since 10th Feb (minus four hours when I used the sensor for the eclipse)) that might be a candidate for plotting.
Hi Dougie,

This is the post I stumbled upon, it is very primative in detail, but I contacted the blogger directly and I got an email with all his scripts and setup which I have been hacking/chopping up as I see relevant.
https://irongarment.wordpress.com/2015/ ... e-logging/

If you PM me your email address I can forward this info to you if you are interested?
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:16 pm

some_evil wrote: So i ran the standalone sqlite query and it returned tons of lines from the database, pasted below is the last dozen lines:

Code: Select all

"2015-04-28 21:00:00",2015-04-28.21:00:00,,,,,,,22.3246
"2015-04-28 21:10:00",2015-04-28.21:10:00,,,,,,,21.0811
"2015-04-28 21:20:00",2015-04-28.21:20:00,,,,,,,22.5621428571429
"2015-04-28 21:30:00",2015-04-28.21:30:00,,,,,,,21.3185
"2015-04-28 21:40:00",2015-04-28.21:40:00,,,,,,,22.5998
"2015-04-28 21:50:00",2015-04-28.21:50:00,,,,,,,21.5684
"2015-04-28 22:00:00",2015-04-28.22:00:00,,,,,,,22.7934
"2015-04-28 22:10:00",2015-04-28.22:10:00,,,,,,,21.7622
"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
"2015-04-28 22:30:00",2015-04-28.22:30:00,,,,,,,20.9748
"2015-04-28 22:40:00",2015-04-28.22:40:00,,,,,,,20.8185
"2015-04-28 22:50:00",2015-04-28.22:50:00,,,,,,,20.6374
"2015-04-28 23:00:00",2015-04-28.23:00:00,,,,,,,20.5623333333333
Note there are about 6 empty columns as I have not connected all sensors to the Pi yet, I am still in development phase so just trialing things before getting technical with 7 thermometers.

So its safe to assume that the problem lies within my ploticus code then?

It is outputting a blank graph (see attached), so do we assume it is successfully running the entire command? i just have something set incorrectly with the data?
24h.png
is the thin horizontal black line running along the X axis actually plotted data? Or just the graph axis?
I think you need to move your data to one of the first four positions. The command-line scripts for ploticus will plot four data series. If you want more you need to either write your own script, or run the command-line scripts several times to overlay more lines on the same graph.

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:21 pm

ame wrote:
some_evil wrote:

Code: Select all

"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
"2015-04-28 22:30:00",2015-04-28.22:30:00,,,,,,,20.9748
"2015-04-28 22:40:00",2015-04-28.22:40:00,,,,,,,20.8185
"2015-04-28 22:50:00",2015-04-28.22:50:00,,,,,,,20.6374
"2015-04-28 23:00:00",2015-04-28.23:00:00,,,,,,,20.5623333333333
I think you need to move your data to one of the first four positions. The command-line scripts for ploticus will plot four data series. If you want more you need to either write your own script, or run the command-line scripts several times to overlay more lines on the same graph.
I just edited my code to be this:

Code: Select all

sqlite3 -csv temperature.db "select timeslot, Demo_Thermometer from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 y2=4 y3=5 y4=6 xinc="3 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="°C" autodays=yes yrange="0 100" xgrid=yes ygrid=yes name="Value" xmargin=0  > /dev/null 2>&1
Runnings just the SQL returns lines like this now:

Code: Select all

2015-04-28.22:30:00,20.9748
2015-04-28.22:40:00,20.8185
2015-04-28.22:50:00,20.6374
2015-04-28.23:00:00,20.506
2015-04-28.23:10:00,20.2872
Does the name="Value" set anything important??

Cheers

EDIT: Edited my code
Last edited by some_evil on Tue Apr 28, 2015 1:30 pm, edited 1 time in total.
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:27 pm

Also can someone please tell me how to define where the output png is going to? I would like it to be forced to go to /var/www/images/24hr.png

Cheers
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:31 pm

some_evil wrote:Also can someone please tell me how to define where the output png is going to? I would like it to be forced to go to /var/www/images/24hr.png

Cheers
The -o argument tells ploticus where to put the output. I suppose you could put a full path in there, otherwise it will just go to whatever is the current directory.

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 1:38 pm

some_evil wrote:
ame wrote:
some_evil wrote:

Code: Select all

"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
"2015-04-28 22:30:00",2015-04-28.22:30:00,,,,,,,20.9748
"2015-04-28 22:40:00",2015-04-28.22:40:00,,,,,,,20.8185
"2015-04-28 22:50:00",2015-04-28.22:50:00,,,,,,,20.6374
"2015-04-28 23:00:00",2015-04-28.23:00:00,,,,,,,20.5623333333333
I think you need to move your data to one of the first four positions. The command-line scripts for ploticus will plot four data series. If you want more you need to either write your own script, or run the command-line scripts several times to overlay more lines on the same graph.
I just edited my code to be this:

Code: Select all

sqlite3 -csv temperature.db "select timeslot, Demo_Thermometer from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 y2=4 y3=5 y4=6 xinc="3 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="°C" autodays=yes yrange="0 100" xgrid=yes ygrid=yes name="Value" xmargin=0  > /dev/null 2>&1
Runnings just the SQL returns lines like this now:

Code: Select all

2015-04-28.22:30:00,20.9748
2015-04-28.22:40:00,20.8185
2015-04-28.22:50:00,20.6374
2015-04-28.23:00:00,20.506
2015-04-28.23:10:00,20.2872
Does the name="Value" set anything important??

Cheers

EDIT: Edited my code
These parameters tell ploticus where to pull data out of the csv file
x=2 y=3 y2=4 y3=5 y4=6

x= is the x-axis. In this case it's the second field in the csv
y is the first y data set, it's the third one
y2 is the second y data set, it's in the 4th field.
etc.

Sorry, I should have suggested that you edit y=3 to y=9, to read the 9th field, which contained your data readings. Can you put it back as it was?

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 9:38 pm

ame wrote: These parameters tell ploticus where to pull data out of the csv file
x=2 y=3 y2=4 y3=5 y4=6

x= is the x-axis. In this case it's the second field in the csv
y is the first y data set, it's the third one
y2 is the second y data set, it's in the 4th field.
etc.

Sorry, I should have suggested that you edit y=3 to y=9, to read the 9th field, which contained your data readings. Can you put it back as it was?
Now its starting to make some sense....

I have modified my code to reflect the changes to the SQL, this is my current string:

Code: Select all

sqlite3 -csv temperature.db "select timeslot, Demo_Thermometer from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=1 y=2 xinc="3 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="°C" autodays=yes yrange="10 30" xgrid=yes ygrid=yes name="Onboard Thermometer" xmargin=0  > /dev/null 2>&1
A sample set of data from my SQL is this:

Code: Select all

"2015-04-29 06:30:00",15.4184
"2015-04-29 06:40:00",16.9248
"2015-04-29 06:50:00",18.0998
"2015-04-29 07:00:00",19.3497
"2015-04-29 07:10:00",21.4496
"2015-04-29 07:20:00",22.2426666666667
As you can see I have narrowed it to be the two columns only, and I have reflected that with x=1 y=2.

The graph now has data plotted, but it seems to be some kind of box plot, where it is ploting all the data for an entire day in one location, then the line zips to the next day and draws the max/min there... is there something I can change to make it acknowledge that I have temperatures every 10min???
24h.png
latest graph
24h.png (1.3 KiB) Viewed 2222 times
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Tue Apr 28, 2015 11:54 pm

Ok, I think you're almost there. ploticus needs the datetime strings formatted in a certain way, with a period "." between the date and time parts. Some of your earlier posts show the date formatted like that, but the last one did not.

Bad:
"2015-04-29 06:30:00",15.4184

Good:
"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
You can use field 2 for x and field 9 for y.

Incidentally, are you storing your times in the database as local time or UTC? It doesn't matter at the moment, but I'd recommend UTC in general, and convert to and from local time as required. It saves a lot of headaches.

Finally, have you read the ploticus documentation?
http://ploticus.sourceforge.net/doc/welcome.html

All of the command line options are explained there. There is a lot to learn, but it looks like you have been exposed to most of it already.

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 12:14 am

ame wrote:Ok, I think you're almost there. ploticus needs the datetime strings formatted in a certain way, with a period "." between the date and time parts. Some of your earlier posts show the date formatted like that, but the last one did not.
Bad:
"2015-04-29 06:30:00",15.4184
Good:
"2015-04-28 22:20:00",2015-04-28.22:20:00,,,,,,,21.2123
You can use field 2 for x and field 9 for y.
Excellent, so the code I have been hacking up and trying to utilise is logging two different types of date, I have no idea why, but I now with your comments I know I need to use the other one:
CREATE VIEW [PivotTemps10min] AS
SELECT
timeslot,
strftime('%Y-%m-%d.%H:%M:%S',timeslot,'localtime'),
etc, etc, etc

I see in my DB the two date fields as you pointed out, it is clear now that I need to use the strftime field, not the timeslot field.

Will the above view columns be called timeslot, localtime? how have I defined that second column name?
ame wrote: Incidentally, are you storing your times in the database as local time or UTC? It doesn't matter at the moment, but I'd recommend UTC in general, and convert to and from local time as required. It saves a lot of headaches.
This is the original line that I hacked to send the results to the DB, I am not infront of my PI so i cant give you my code, but essentially this is what I used, I just added several other columns for extra sensors.

Code: Select all

results.append((datetime.utcnow().isoformat(' ')[:19],device,temperature))
So I guess I am using UTC? What are the Pros and Cons of these?
Note my Pi does not have internet connectivity, so I have to manually set the time/date intermittently.
ame wrote: Finally, have you read the ploticus documentation?
http://ploticus.sourceforge.net/doc/welcome.html

All of the command line options are explained there. There is a lot to learn, but it looks like you have been exposed to most of it already.
Yes I have seen that page, but as I have been learning ploticus in reverse (been given code, trying to modify) it is/was hard to know what the components meant, now you have basically told me everything I need to edit, I can search this site with a bit more clarity/direction.

I will let you know how it goes when I get home.

Thanks again.
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 12:36 am

some_evil wrote: Excellent, so the code I have been hacking up and trying to utilise is logging two different types of date, I have no idea why, but I now with your comments I know I need to use the other one:
CREATE VIEW [PivotTemps10min] AS
SELECT
timeslot,
strftime('%Y-%m-%d.%H:%M:%S',timeslot,'localtime'),
etc, etc, etc
This line...
strftime('%Y-%m-%d.%H:%M:%S',timeslot,'localtime'),
...formats the date string with a period between the date and time.
I see in my DB the two date fields as you pointed out, it is clear now that I need to use the strftime field, not the timeslot field.

Will the above view columns be called timeslot, localtime? how have I defined that second column name?
Um. I don't know. Maybe you just refer to it by position? Originally it was the second field, so use x=2 in your ploticus arguments.
ame wrote: Incidentally, are you storing your times in the database as local time or UTC? It doesn't matter at the moment, but I'd recommend UTC in general, and convert to and from local time as required. It saves a lot of headaches.
This is the original line that I hacked to send the results to the DB, I am not infront of my PI so i cant give you my code, but essentially this is what I used, I just added several other columns for extra sensors.

Code: Select all

results.append((datetime.utcnow().isoformat(' ')[:19],device,temperature))
So I guess I am using UTC? What are the Pros and Cons of these?
Note my Pi does not have internet connectivity, so I have to manually set the time/date intermittently.
Ah, if you are setting the time yourself then you are probably looking at your watch and typing local time. If you haven't configured the timezone on your Pi then Linux will interpret that as UTC (or maybe UK time).

This is a huge topic to explain, so I am not going to.

Basically, you need to (sorry, ought to) set your Pi internal clock to UTC, and then let the OS figure out local time for you. If you use UTC then problems such as daylight saving time and other oddities get dealt with.

Use raspi-config to set the timezone properly, and then you should see the database entries are recorded with UTC (which will differ from your local time by about 12 hours because you are in Australia).

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 12:48 am

ame wrote: Basically, you need to (sorry, ought to) set your Pi internal clock to UTC, and then let the OS figure out local time for you. If you use UTC then problems such as daylight saving time and other oddities get dealt with.

Use raspi-config to set the timezone properly, and then you should see the database entries are recorded with UTC (which will differ from your local time by about 12 hours because you are in Australia).
Ok when loading Raspbian initially I selected a timezone, but as the Pi has no internet connectivity I have to set the time with:

Code: Select all

sudo date -s "Jan 5 2015 08:10"
Does this kill the UTC?
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 12:59 am

some_evil wrote:
ame wrote: Basically, you need to (sorry, ought to) set your Pi internal clock to UTC, and then let the OS figure out local time for you. If you use UTC then problems such as daylight saving time and other oddities get dealt with.

Use raspi-config to set the timezone properly, and then you should see the database entries are recorded with UTC (which will differ from your local time by about 12 hours because you are in Australia).
Ok when loading Raspbian initially I selected a timezone, but as the Pi has no internet connectivity I have to set the time with:

Code: Select all

sudo date -s "Jan 5 2015 08:10"
Does this kill the UTC?
Does this time "Jan 5 2015 08:10" represent your local time, or UTC? i.e. do you look at your watch, or phone, or clock on the wall and type that time in as is?

After setting the clock, type

Code: Select all

date -u
to see UTC time.

And also

Code: Select all

date
to see local time.

If you have everything set up properly then they should be different.

I recommend that you hook your Pi up to the network somehow. It will make a lot of things easier in the future. If you can't, then consider getting hold of an RTC module such as this one:
http://www.dx.com/p/ds3231-raspberry-pi ... UAslvCudv0

It will keep time if the the power fails, and can be used to set the time automatically when your Pi reboots.

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 1:10 am

ame wrote: Does this time "Jan 5 2015 08:10" represent your local time, or UTC? i.e. do you look at your watch, or phone, or clock on the wall and type that time in as is?
I will look into this, but I definitely look at my Watch and set the time. I will use the date -u to see if it differs.
ame wrote: I recommend that you hook your Pi up to the network somehow. It will make a lot of things easier in the future. If you can't, then consider getting hold of an RTC module such as this one:
http://www.dx.com/p/ds3231-raspberry-pi ... UAslvCudv0
I do have one of these sitting on my desk ready to go, Its just another thing on my 'to-do' list. I will have to find some walk through/instruction on how to set this up.
(note for myself, found a setup guide - https://nicegear.co.nz/blog/using-an-i2 ... pberry-pi/, take note of the comment "To enable it, type sudo raspi-config, then select option 8 (Advanced Options), then select A7 I2C, then select Yes to enable. After that, you can follow the instructions and the clock works great then..")

unfortunately my Pi is connected to my home network, but my home has no internet available (long story). Thats why I have to set the time manually.

My pi is serving a website which will deliver these PNG files to all devices on my home network.
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 9:07 am

Hi all,

I have had great success, I can manually input the below code into my command line and it outputs the PNG exactly how I would like:

Code: Select all

sqlite3 -csv temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 y2=4 y3=5 y4=6 y5=7 y6=8 y7=9 xinc="2 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="Temp (°C)" autodays=yes yrange="15 25" xgrid=yes ygrid=yes name="Name1" name2="Study" name3="Three" name4="Fourrr" xmargin=0  > /dev/null 2>&1
So what i have done is saved the below (verbatim) into a file called 'make_graph_24h' and it resides in my /home/pi directory.

Code: Select all

sleep 20
sqlite3 -csv temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-24 hour');" | ploticus -png -o 24h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 y2=4 y3=5 y4=6 y5=7 y6=8 y7=9 xinc="2 hour" legendfmt=singleline xstubfmt="hh" title="Temperature over the last 24 hours" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="Temp (°C)" autodays=yes yrange="15 25" xgrid=yes ygrid=yes name="Name1" name2="Study" name3="Three" name4="Fourrr" xmargin=0  > /dev/null 2>&1
sudo cp 24h.png /usr/share/nginx/www/
sudo cp 24h.png /var/www/image
Then in my pi crontab I have added the below line in an attempt to automate this task every 1 minute (this will change once I have it running), but nothing seems to happen/output with this cron job. Have I set it up incorrectly?

* * * * * make_graph_24h

I have only ever gotten .py files to execute from cron, never a file which I hope just executes lines of input.

Any tips?
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 9:41 am

You need to include the path.

./make_graph_24h

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 11:19 am

ame wrote:You need to include the path.

./make_graph_24h
Hi ame,

I have modified my crontab to include the dot backslash, still no output... I'm not sure what else I have to look at...
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Wed Apr 29, 2015 12:24 pm

To test the file was actually firing as per my crontab setup I added a line to the bottom of the make_graph_24 file that was :
Mkdir graph24

And sure enough the file was created, so I know the cron is starting this file... I have no idea why I can copy and paste to the command line, but can't run via cron...
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Fri May 01, 2015 2:38 am

Any progress? I was hoping someone would step in with some hints and tips as I don't profess to know everything.

However, with cron it's always paths, followed by permissions.

You have to be mindful of which user is executing the cronjob, and what is on the path. When cron starts a process to execute the job it runs as the user whose crontab it is, with that user's permissions. The environment the process has is very limited and may not include a path, so, often you have to put explicit paths everywhere. Or arrange things so that no path is needed (i.e. don't use subdirectories).

Getting the script to do something extra as a diagnosis is good (like making a directory, for example). That particular test tells you which directory the script is executing in. Maybe you need to split the job into two parts- one to extract from SQLite, and one to call ploticus. It's probably a path thing there (it's always a path thing).

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Fri May 01, 2015 2:48 am

ame wrote:Any progress? I was hoping someone would step in with some hints and tips as I don't profess to know everything.

However, with cron it's always paths, followed by permissions.

You have to be mindful of which user is executing the cronjob, and what is on the path. When cron starts a process to execute the job it runs as the user whose crontab it is, with that user's permissions. The environment the process has is very limited and may not include a path, so, often you have to put explicit paths everywhere. Or arrange things so that no path is needed (i.e. don't use subdirectories).

Getting the script to do something extra as a diagnosis is good (like making a directory, for example). That particular test tells you which directory the script is executing in. Maybe you need to split the job into two parts- one to extract from SQLite, and one to call ploticus. It's probably a path thing there (it's always a path thing).
Hi Ame,

Your comment is very timely, yes I have only just had a positive outcome.

Basically as you stated, I just had to go through and ensure I had explicit paths on everything (/home/pi/) and its not in front of me ATM, but I believe I also had to sudo my cron task. Thanks for following up though.

I do have a formatting question I hope you can help me with. I will use the only example graph I have on this PC, but see below the image is using the hour of the day along the x axis, I am hoping to display something like HH:MM instead, as I am making the graph plot the last one hour of data logged. Is this a simple change?
24h.png
24hr graph
24h.png (1.3 KiB) Viewed 2007 times
sqlite3 -csv /home/pi/temperature.db "select * from PivotTemps10min where timeslot > datetime('now','-1 hour');" | ploticus -png -o /home/pi/1h.png -prefab chron data=- delim=comma header=no gapmissing=yes unittype=datetime mode=line datefmt="yyyy-mm-dd" x=2 y=3 xinc="10 minute" legendfmt=singleline xstubfmt="hh" title="Temperature over the last one hour" xlbl="Time" xlbldet="adjust=0,-0.15" ylbl="Temp (°C)" autodays=yes yrange="15 25" xgrid=yes ygrid=yes name="Demo Thermom" xmargin=0

PS please note the line on the graph, this is not the current output, its all I have on me at the moment.

thanks again.
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

ame
Posts: 3172
Joined: Sat Aug 18, 2012 1:21 am
Location: New Zealand

Re: Trying to Learn Ploticus - need help

Fri May 01, 2015 3:02 am

To format the times as you like look at the xstubfmt argument.

As you can see in your example it's xstubfmt="hh", which prints only a 2-digit hour. Check the documentation for other variants, but I think "hh:mm" might work (or capital M to disambiguate from "month").

User avatar
some_evil
Posts: 205
Joined: Thu Dec 18, 2014 3:16 am
Location: Albury, Australia

Re: Trying to Learn Ploticus - need help

Fri May 01, 2015 4:16 am

Excellent, now im getting a feel for things I have found that chron (chronological data) is the type of data I am plotting, so this page makes a whole lot more sense to me now - http://ploticus.sourceforge.net/doc/prefab_chron.html

I will let you know how it turns out.

Cheers
PiZeroW - May 2017
Pi 3 - Oct 2016
PiZero - June 2016
Pi 2 - Jan 2016
Pi B+ - Dec 25 2014

Return to “Graphics, sound and multimedia”