Houseboat home automation / surveillance system

[Lomax]

Note: Subject changed from "Anyone here with experience of the Monarco Hat?"

I'm very interested in getting a Monarco Hat for a project I'm working on - it ticks off so many of my requirements with a single add-on that it's almost too good to be true. I need

• 12V power input
• 1-Wire interface
• RS-485
• 24V tolerant I/O
• RTC
• Hardware watchdog

All of which the Monarco Hat provides. But I have some doubts... I am concerned that this is a product which has been designed primarily to be used with their own software, called "REX Controls" I think, and that accessing things like GPIO pins, PWM output, OS level RS-485/1-Wire comms, simply won't work at all., or be very cludgy. I have no intention of switching to REX Controls, but I really could do with the I/O capabilities of the Monarco. Does anyone here know if my concern is justified? If so, any suggestion for similar interfaces which operate more "transparently"?

[Lomax]

*bump*

Surely many Raspberrians must have dealt with interfacing a Pi to an (electrically) noisy, >5V environment? In cars, for example? Even simple automation would be best done with some kind of buffering/ESD protection on the GPIO pins. Or is it just that everyone uses wireless tech for this, so never worry about ESD? Personally, I'm a wires kinda guy, so I do need to worry...

[Lomax]

Never mind, I've just ordered a Monarco Hat, so I can figure these things out for myself.

[CarlRJ]

Surely many Raspberrians must have dealt with interfacing a Pi to an (electrically) noisy, >5V environment? In cars, for example?

Plenty of people have done Pi-in-car projects, some of them have posted here looking for help or detailing their results. I've never seen any of them mention the HAT you're asking about. Folks frequently don't spend extra money to add interfaces they don't need (1-wire, RS485), and there are simpler alternatives to handle the voltage differences, and the board looks to be about four times the cost of a Pi3B itself. You yourself describe a bunch of things that would tend to steer experimenters away from the device, so I'm not surprised it hasn't been discussed here.

[Lomax]

Plenty of people have done Pi-in-car projects, some of them have posted here looking for help or detailing their results. I've never seen any of them mention the HAT you're asking about. Folks frequently don't spend extra money to add interfaces they don't need (1-wire, RS485), and there are simpler alternatives to handle the voltage differences, and the board looks to be about four times the cost of a Pi3B itself. You yourself describe a bunch of things that would tend to steer experimenters away from the device, so I'm not surprised it hasn't been discussed here.

Fair enough, and thank you for giving some input! I have looked around but can't find anything that looks more promising than the Monarco - especially when considering I have a need for pretty much every single functionality it offers. I'm building a home automation/security/monitoring system for a houseboat, and 1-Wire will be used for weather and temperature data (weather is important for obvious reasons, and boy are there a lot of interesting temperatures to monitor on a boat), RS-485 to interface with the solar charge controllers and main battery bank charger/inverters, as well as potentially my own homebrew 12V Modbus LED dimmers. Voltage tolerance is important, and so is ESD/isolation (think old engine alternator, nearby lightning strikes, etc). 12V -> 5V DC/DC is a given. RTC as well (though I am trying to get the Pi to set time/date from GPS data as well). And since remote access is one of the main reasons I'm building this thing, a good hardware watchdog that can bring the Pi back up after a hang is definitely nice to have. Now if you add all those things together, and consider the time it would take to cobble together this functionality from individual parts, the £150 price tag of the Monarco suddenly doesn't look all that bad. If it can be made to work with my chosen software without too much hassle. If.

[CarlRJ]

Given the bit about it being on a boat, with unreliable power, I would strongly suggest setting it up as a system that is run from either an actual UPS, or a scratch-built UPS equivalent, with the Pi drawing from large batteries, which are recharged as necessary from solar/alternator/whatever. Pi's like clean filtered power (your HAT presumably handles some of that, sounds like it was designed for use on shop floors and such), and they like continuous power - they're meant to either run 24/7 or only shut down in an orderly pre-meditated way (one of the problems to overcome in car systems). (BTW, the MoPower UPS HAT is probably not exactly what you want, but is worth looking at for ideas.)

As to the Monarco HAT: for every bit of hardware discussed here, there has to be someone who is/was the first person to try it out for personal use - you can be that person for this HAT and report to the group whether it turns out to be suitable for your project, as well as reporting on your Pi-in-a-boat project. I'm curious to hear how it goes.

[Lomax]

Given the bit about it being on a boat, with unreliable power, I would strongly suggest setting it up as a system that is run from either an actual UPS, or a scratch-built UPS equivalent, with the Pi drawing from large batteries, which are recharged as necessary from solar/alternator/whatever.

I'm waaay ahead of ya

08-hooked_up.JPG (124.82 KiB) Viewed 4275 times

Oh yeah, there's supposed to be a lid over the fuse holder - but the damn thing won't fit because the blade fuses I bought are 1mm too tall! Requires careful manouvering with conductive tools until resolved...

Domoticz Screenshot_2017-08-17_21-45-32.png (95.49 KiB) Viewed 4164 times

Some of my Domoticz sensors

Node-Red Screenshot_2017-08-17_23-47-15.png (61.14 KiB) Viewed 4182 times

Node red flow for SMS query interface and WWAN router telemetry

Pi's like clean filtered power (your HAT presumably handles some of that, sounds like it was designed for use on shop floors and such), and they like continuous power - they're meant to either run 24/7 or only shut down in an orderly pre-meditated way (one of the problems to overcome in car systems). (BTW, the MoPower UPS HAT is probably not exactly what you want, but is worth looking at for ideas.)

Agreed. There's an OpenUPS board under that perforated aluminium shield, backed up by a 17Ah SLA battery, which can keep the whole rig going for about 24 hours (current draw just under 500mA @ 12V). As you can see, the Pi in the photo is still "naked" (though it does control the case fan via hardware PWM) - I've started from the bottom and built up, so first priority was to sort out a reliable (and clean) power source. It feeds off the main bank, which in turn is charged by solar panels, shore power and (when it's running) the engine alternator. Second step was network connectivity, and I managed to get hold of a pretty nice "vehicular router" from Sierra Wireless for this (hint: I did not pay full price). It does a lot of of cool stuff, and will notify me via SMS and email when certain events occur, regardless of the Pi's status. Only now have I got to the point where I'm ready to start interfacing the Pi and the network with other stuff. I've got a Dallas wind speed/direction & external temperature 1-wire weather station, a Panasonic TGP500 SIP phone, a couple of CSI cameras (which will be connected to an IVPort camera multiplexer) and a few other bits & bobs, such as PIR, fire, flooding and gas sensors (which will all operate stand alone and only report to the Pi), security keypad, sirens, strobes... It's a big project.

As to the Monarco HAT: for every bit of hardware discussed here, there has to be someone who is/was the first person to try it out for personal use - you can be that person for this HAT and report to the group whether it turns out to be suitable for your project, as well as reporting on your Pi-in-a-boat project. I'm curious to hear how it goes.

Thanks, I really do appreciate your response! There is literally no-one around me who understands what I'm trying to do or why it's exciting, so the Interwebs is my only chance of friendly geeky dialog. I will be writing about this project, here and on the Domoticz forum - in fact I've been taking photos as it's gone along specifically with a write-up in mind...

[Lomax]

A few more photos...

03-empty.JPG (83.43 KiB) Viewed 4268 times

The enclosure when (nearly) empty.

01-external.JPG (57.11 KiB) Viewed 4267 times

A fairly inconspicuous looking enclosure apart from the fan grille - this is intentional.

02-external.JPG (57.94 KiB) Viewed 4268 times

Bottom air intake grille.

[Lomax]

The cable entry system alone will be worth its own write up:

07-cable_entry.JPG (66.56 KiB) Viewed 4256 times

Pricey, but worth every penny! And yes, I'm still waiting for two (back-ordered) cable glands, both 4x3mm - the power lines are just borrowing a (too large) 4x4mm one in this photo.

[Lomax]

Rough cost break-down, so far (off the top of my head):

Raspberry Pi 2B: £35
16Gb high endurance MicroSD: £15
OpenUPS card: £100
35x25x15cm Enclosure: £30
17Ah SL battery: £30
Sunon MagLev fan: £15
Netgear 5-port switch: £25
IVPort camera multiplexer: £70
2x CSI cameras: £50
2x Petit Studio FPC <> HDMI £30
Monarco Hat :£150
Blue Sea blade fuse holder: £30
Combined LTE/GPS antenna: £20
Phoenix Contact cable entry: £30
200 hrs+ work: priceless

Add aluminium sheeting, cables, crimp terminals, etc, ect, etc, and we're easily talking about £600. And I'm not even going to tell you how much I paid for that AirLink router...

Note to burglars: if you ever break into my boat, STEAL THE SURVEILLANCE SYSTEM. It's probably the most value/kg you can grab - though I will be coming after you.

[Lomax]

It almost seems a shame to shut all those shiny LEDs inside - and I do worry about the unnecessary power draw. But awww, it's so pretty!

09-leds_internal.JPG (78.94 KiB) Viewed 4192 times

A row of four 1mm dia holes in the door line up with the status LEDs on the AirLink, so power, activity, WWAN strength and availability can be seen without having to open it (yeah, 2nd from the left is 0.2mm too high; I am unhappy about this).

10-leds_external.JPG (26.05 KiB) Viewed 4192 times

[scruss]

The only other mention of this product was when they developer showed up (briefly) to announce it: Monarco HAT - analog and digital I/O, RS-485, 1-Wire, RTC... - Raspberry Pi Forums

[Lomax]

The only other mention of this product was when they developer showed up (briefly) to announce it: Monarco HAT - analog and digital I/O, RS-485, 1-Wire, RTC... - Raspberry Pi Forums

Thanks! Shame they are not being more active here. I have been in touch with them though, to get some clarification on how to interface with the, erm, interface. The on-board ARM Cortex M3 handles the pin I/O over SPI and there is a Node.js library available which polls it at a settable frequency (defaults to every 70ms). I'm guessing that creating custom Node-RED nodes which implement this library should be fairly easy, but it will be interesting to see what it means in terms of CPU and power usage.

Based on their response, and the rather excellent hardware manual, I've come up with the following merged "pinmapping":

Mother_pinmap.png (53.09 KiB) Viewed 4117 times

Doesn't look like any major conflicts; the only question mark is pin 16 (GPIO23) which is needed by IVPort but is also available as an optional pass-through pin on the Monarco. I'm hoping the Monarco can simply be told to ignore it, though I'm unsure if this would mean losing one Monarco I/O. As a side note, while I'm currently using pin 32 (GPIO12) as a hardware PWM output from the Pi to control the case fan speed, this means I've had to sacrifice on-board audio, which seems a little "expensive" - particularly since I've recently started looking at TTS output from Node-RED, which resulted in geekbumps. I think I might have to re-think case fan speed control so the on-board audio can be used for notifications.

[Lomax]

Also: the deeper I get into Node-RED, the less need I see for Domoticz. Now looking at dashboards and data logging. Anyone else here feel the same way?

[jsobota]

Hi all,
I'm a bit late for the party, sorry for that. Thanks @Lomax for this excellent thread. Love your project regardless of what hardware and software you decide to use in the end.

Apart from SPI-0, UART, I2C-0 and I2C-1, the only GPIO pins occupied by the Monarco HAT are GPIO20, GPIO21 and GPIO26. You are free to use the remaining GPIO pins of the Raspberry Pi, including GPIO05 [29], GPIO06 [31], GPIO23 [16] and GPIO24 [18]. Standard Monarco HAT firmware keeps these four GPIOs floating. They are intended for hypothetical future use or custom firmware functions.

Thanks for your compliments. We do our best to make our documentation worth reading and I praise you for doing so. You already mentioned the full technical specification of the Monarco HAT and there is also some more Monarco HAT documentation on GitHub.

As for the audio and GPIO12, I believe you can use one digital output of the Monarco HAT for fan speed control, freeing the onboard audio.

The only other mention of this product was when they developer showed up (briefly) to announce it: Monarco HAT - analog and digital I/O, RS-485, 1-Wire, RTC... - Raspberry Pi Forums

You are right. So far there was very little response from users of this forum, resulting from the fact that a typical forum user here is quite different from a user the Monarco HAT was designed for. Comments of @carIRJ moreorless prove it (which by no means makes them less valuable or relevant). Thanks to @Lomas this is no longer true and I'm ready to increase my activity.

Jaroslav

[Lomax]

Many thanks Jaroslav, great to see you here! If the many hours I've spent looking for a sturdy I/O interface for the Pi is anything to go by you have a pretty unique product - don't be shy about it! Good to have confirmation re. the "optional" pin usage on the Monarco; sounds like what I want to do is achievable without having to jury-rig any pin conversions - great! As you can see in the photos, space is pretty tight in every axis apart from depth (which will be needed by the Monarco and IVPort hats), and I have tried to position the Pi so that the Monarco I/O connector still has some wiggle room. While it is fairly laborious to do so, the whole thing has been designed so that the enclosure can be disconnected and removed for servicing, and the detatchable I/O connector that comes with the Monarco fits right in here.

I've come to realise though, that I will need to move the Pi 1.5cm to the left, to make room for the four(!) Petit Studio FPC > HDMI adapter boards to which the cameras will be connected (stacked two high). I had considered bodging together some kind of direct cable to FPC connections, but HDMI wins on simplicity and sturdiness, even if it's bulkier. Besides, ready-made HDMI cables are cheap and plentiful, in whatever length required; buying a 30m reel of shielded 8-pair cable would cost over £200, and I'd probably never use more than half of it. And while it is true that you could hook up CSI cameras over a 6-pair cable, going with HDMI cables for the cameras will give me an extra four I/O lines per camera, which can be used for sensors placed close to the cameras, or controlling camera lights or servos. I may revise this decision when it comes to actually routing the cables around the boat though; HDMI connectors are pretty chunky...

Regarding the case fan PWM control, this is clearly overkill, particularly since I have only defined three fan "speeds"; off, low and high, with "high" bascially being 100% duty cycle. Having tried to implement PID controlled fan speed I found the constantly changing pitch of the fan to be annoying; instead my "low" level has been set where the fan is inaudible yet provides meaningful cooling, and "high" only kicks in when really needed - and even then it remains surprisingly subtle. Current temperature breakpoints are

Code: Select all

off_low=44
low_off=39
low_high=54
high_low=49

and I've only passed 54 degrees with the case in full sun and the Pi running `stress -c 4`. Two regular pins and a 555 should be able to provide the same functionality without having to sacrifice a precious PWM channel, either on the Pi itself or on the Monarco*. Another thing on my list is implementing fan control based on temperature readings from the OpenUPS board and the AirLink WWAN router - at the moment the fan speed is only controlled by the Pi CPU temperature; it would be a shame not to run it should any of the other devices overheat without the Pi doing the same.

Right at this moment though, I have distracted myself with getting stills captures from (one) camera working in Node-RED, and having them sent out as email attachements:

Screenshot_2017-08-19_15-50-06.png (21.83 KiB) Viewed 3986 times

It's almost magical.

*) Edit: Took a while for Jaroslav's point about using a digital pin on the Monarco for this to sink in; any one of the four digital out pins it provides can do PWM, up to 100kHz, without taxing the Pi's CPU. This is nice, though I still might opt for a solution using the Pi's "native" pins; those four output pins are better suited for interfacing with the world outside the enclosure.

[Lomax]

I suddenly remembered I had some bits of fibre-optic filaments lying around, and found a piece which was a perfect fit in the 1mm holes I had drilled in the door (so that the WWAN router's status LEDs can be seen without opening it). Having seen the huge improvement in clarity provided by just a rough snap-off piece, I just had to do it, even though polishing fibre-optics is a chore (it's glass after all). By scoring and snapping I produced four pieces which were ~0.5mm longer than the required length (which I measured by pushing filament into the holes with the door closed):

optic_01_pieces.JPG (62.62 KiB) Viewed 3883 times

I made a simple jig by drilling four holes into a piece of 5mm acrylic, and sanded it down with the filaments inserted until I had the right length - first 180 grit, then 800, then 1200.

optic_02_jig.JPG (67.42 KiB) Viewed 3883 times

CA seemed like the best method for affixing them in the holes, and to keep the CA from seeping through capillarily and ruin the front, I squidged on a piece of 3M's marvellous double sided tape. My thinking was that it would not only form a tight seal - which it did - but that it would be squishy enough to keep the tops of the filaments just shy of the door surface - which it also did. How I've lived without this stuff before is beyond me.

optic_03_tape.JPG (35.72 KiB) Viewed 3883 times

[Lomax]

Once glued into place (and I gave the CA a good hour to set), I painted the inside matte black around the filaments, to improve separation and get rid of some very slight ghosting through the door itself:

optic_05_inside.JPG (47.36 KiB) Viewed 3883 times

After the paint had thoroughly dried I ran over the back of the filaments again with a block of acrylic dressed with 1200 grit sanding paper, just to get rid of any CA frosting and traces of paint. Once I closed the door and removed the 3M tape the improvement was quite stunning - this is definitely a recipe I will be using again:

optic_04_result.JPG (34.5 KiB) Viewed 3883 times

Those four little lights will provide all the vitals at a glance, and thanks to the addition of fibre-optic light guides they're now bright enough to be visible in daylight. If only I can remember what all the combinations mean. Ah yes, there's a manual:

Raven_LEDs.png (56.68 KiB) Viewed 3866 times

Now back to beer

[jsobota]

Hi Lomax,
I'm just wondering about your mention of multiple cameras. Are you planning to use multiple Raspberry Pi? AFAIK there is only one CSI port on the Pi to connect the RPi Camera...

In any case, thanks for sharing your amazing project!

Best regards,
Jaroslav

[Lomax]

Hi Jaroslav!

That's right, and I've spent a long time figuring out how to achieve what I want. In a normal domestic surveillance system you have so many options, anything from PoE cameras, to WiFi cameras, to USB webcams connected to more powerful computer(s), to individual Pi Zeroes with one CSI camera each. My situation is quite different in a number of ways; I have limited power (solar panels), limited bandwidth (mobile network), limited space and limited CPU cycles. And the boat is made from steel, so WiFi, Z-Wave and other wireless technologies may not work well. Everything has to be small, power efficient, CPU efficient, low bandwidth and connected by wires. The whole system needs to idle at very low power consumption (way under 10W). Etc etc etc.

So for example, in a house I would maybe have run individual Pi's with a camera each, all running Motion and all streaming HD video on demand over WiFi and the Internet. On the boat I only want one Pi that is "always on", and it should be idling at very low CPU usage. All cameras should sleep unless requested (with motion detection done by PIRs and accelerometers instead of Motion), and only still images are captured, with very little data being streamed out over the internet connection.

But how? With only one CSI connector it seems impossible.

Well, there's a company in Turkey who have developed an ingenious board which is perfect for my needs. It's called the IVPort and it splits the single CSI connector into four which can be switched between (multiplexed) using a few GPIO pins. When channels are off, the cameras can be unpowered (settable by a jumper on the board), so only one camera at a time will be "on" (and even that can be turned off of course). This means there will need to be a short delay between turning a camera on and capturing an image, to give the camera time to adjust the exposure, but in tests it seems a third of a second is plenty, which I am more than happy with.

So, in Node-RED, I will have certain sensors and events trigger still capture from certain cameras (which camera gets used is set simply by switching two GPIOs, 00,01,10,11). A loop ensures that a sequence is taken for a pre-set time (say one picture every five seconds for one minute), and those images are immediately passed on over the VPN (if available) to my cloud server, and via email to myself. This loop can include any number of cameras, so nothing is missed. If additional light (visible or IR) is needed, Node-RED will switch it on for me (the light dimmers will be on a Modbus network). I even have (currently dormant) plans to make the mast-top camera(s) titlable with a servo. And I can also set Node-RED to send me a still capture from each camera on a schedule, so I'd get a daily visual status update, or email it and request a capture from certain camera(s).

This is the plan, and I already have the first camera working exactly like this, but I'm waiting for the delivery of the IVPort I ordered (only two ports, to start with), so haven't been able to test that part yet. Pretty sure it will work though!

Edit: In case anyone is wondering about the power consumption; I currently have the enclosure powered from my bench supply here, set at 12.5V (current breakpoint to switch to UPS power is 12.4V in, though I might lower that to 12.2V later). Total current draw with the SLA battery on float charge is under 500mA, so about 6W. That's with the following equipment installed:

• Sierra Wireless RV50 3G/4G router (always on)
• Netgear GS-105 5-port gigabit switch (always on)
• Raspberry Pi 2 Model B (always on)
• OpenUPS BMS board (always on)
• 12V -> 5V DC/DC converter (always on)
• One CSI connected camera (normally off)
• One 8x8cm axial fan (normally off)

The Pi is running Raspbian with Domoticz, Mosquitto and Node-RED, as well as a number of minor daemons (upsd, sshd, gpsd, rsyslogd, ntpd, iptables, nginx). CPU usage idles below 0.4%. It pushes updates via email and SMS when certain events occur and it also listens to incoming SMS and email messages and responds with requested information (it checks its email account every two minutes). It is also connected to a cloud server over a VPN (endpoint on the RV50), but at present I'm not streaming any data over it.

[jsobota]

Simply ingenious! Thanks for sharing all the information.

Best regards,
Jaroslav

[Lomax]

As some of you may have read in another thread, I am facing three "show-stopper" issues with this project; 1: VPN connectivity, 2: setting of accurate date/time and 3: anomalous UPS behaviour. Some progress tonight on problem #2: how to get (and set) correct date and time.

I didn't have to write my own NMEA-0183 parser in the end, since I found a very nice NPM package that does this: nmea-0183. Struggled for a while with how to use it in Node-RED without having to package it up as a node, and eventually came across node-red-contrib-npm by James Thomas at IBM, which makes it very easy to include node.js helper methods in a Node-RED flow. I now have a simple flow which listens for incoming NMEA sentences on UDP 10110 and spits them back out as MQTT messages:

Screenshot_2017-08-28_20-28-46.png (15.89 KiB) Viewed 3542 times

The resulting messages look like this:

Code: Select all

{ "topic": "raven/gps/GPGGA/fix", "timestamp": 1503947329529, "payload": { "value": 1 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/satellites", "timestamp": 1503947329529, "payload": { "value": 8 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/hdop", "timestamp": 1503947329529, "payload": { "value": 1 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/aboveGeoid", "timestamp": 1503947329529, "payload": { "value": 47 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/time", "timestamp": 1503947329529, "payload": { "value": "190946.0" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/latitude", "timestamp": 1503947329529, "payload": { "value": "12.34567890" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/longitude", "timestamp": 1503947329529, "payload": { "value": "0.12345678" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPGGA/altitude", "timestamp": 1503947329529, "payload": { "value": 14.7 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/valid", "timestamp": 1503947329558, "payload": { "value": "A" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/mode", "timestamp": 1503947329558, "payload": { "value": "E" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/variation", "timestamp": 1503947329558, "payload": { "value": 0 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/time", "timestamp": 1503947329558, "payload": { "value": "190946.0" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/latitude", "timestamp": 1503947329558, "payload": { "value": "12.34567890" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/longitude", "timestamp": 1503947329558, "payload": { "value": "0.12345678" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/speed", "timestamp": 1503947329558, "payload": { "value": 0 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/course", "timestamp": 1503947329558, "payload": { "value": 83.3 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPRMC/date", "timestamp": 1503947329558, "payload": { "value": "280817" }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPVTG/course", "timestamp": 1503947329584, "payload": { "value": 83.3 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPVTG/knots", "timestamp": 1503947329584, "payload": { "value": 0 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPVTG/kph", "timestamp": 1503947329584, "payload": { "value": 0 }, "_msgid": "bd1b0634.b9f118" }
{ "topic": "raven/gps/GPVTG/mode", "timestamp": 1503947329584, "payload": { "value": "A" }, "_msgid": "bd1b0634.b9f118" }

As you can see, there's quite a lot of overlap between the three sentences provided by the RV50 - I will probably trim this down considerably to only send the values I'm really interested in, but that means abandoning the NMEA-0183 sentence format. The next step is to compare the date and time in the GPS data to the current system date/time and adjust it if it's off. Any suggestions for the best way to do this? I don't really want to give my "nodered" user the right to execute "date -s", but what other options are there?

[Lomax]

While decoding the NMEA sentences proved a doddle, thanks to the node.js library I found, parsing GPS time into a usable format proved much more "interesting". The time and date strings provided in the GPRMC sentence are in the format "DDMMYY" and "HHMMSS.S" respectively, which I needed to turn into, for example, a Javascript UTC epoch. Here's what I've come up with, after a couple hours of wrestling:

Code: Select all

if(msg.payload["id"] == "GPRMC") {
    var date = msg.payload["date"].match(/[0-9]{2}/g);
    var time = msg.payload["time"].match(/[0-9]{2}/g);
    var epoch = Date.parse("20" + date.reverse().join("-") + "T" + time.join(":") + "Z");
    msgs[2].push({
        "topic":"mother/epoch",
        "payload":epoch
    });
}

Phew! Finally I have a number that I can compare with the output of Date.now() to see if the clock needs adjusting. At first I had the idea that I would try to get the time zone as well - based on the actual location coordinates, via some kind of time zone shape database - but while this is doable to some extent, and would surely be worth a few geekpoints, it felt like complete overkill; I'll just have to remember to manually change the system time zone when/if I cross the channel Or maybe I can work out a way to get it from the mobile network later... In any case, it's probably easiest to just do everything in UTC, and only use TZ data for presentational purposes.

The two regular expressions take the "date" and "time" values from the NMEA data and split them into digit pairs, so I get two arrays with e.g. [29,08,17] and [23,46,03] respectively (note that the values are strings, which is important later). The eagle eyed among you might notice that I'm discarding the fractional second; not only do I not need sub-second precision, the RV50 doesn't even provide it, with the fraction always being returned as ".0", and dropping this extraneous digit made things a lot simpler.

These two arrays are then used to assemble a valid ISO 8601 date string, which is fed to Javascript's Date.parse function, which returns the date as a Javascript UTC epoch (the number of milliseconds since Jan 1st 1970, without leap seconds). Note that I'm reversing the date array and adding a hard-coded "20" - yes, girls and boys, this is exactly how "millennium bugs" come into being (or even "century bugs" in this case; far worse). What can I say; if I'm still alive in 83 years and you happen to be affected by this bug - sue me!

Note also the trailing "Z" which is added to designate UTC. Provided that the clock on the Pi has the correct time this should return the same value as Date.now(), making comparison trivial. Or so I hope; I have to admit this whole business with UTC and offsets has made me a little dizzy.

The calculated epoch is then finally returned on the third output of the "decide" node, where it can be passed on to the next step: comparing GPS time with machine time, and correcting the latter if it's off by a second or more. That will have to wait until tomorrow though.

[davidcoton]

In any case, it's probably easiest to just do everything in UTC, and only use TZ data for presentational purposes.

Yes, that is (almost?) always the preferred way to do things. Log timestamps should also be UTC to avoid "nasties" with Summer Time and changing timezones. If you need to preserve the local time of an event, save the TZ as a separate item in the log entry.

[Lomax]

Yes, that is (almost?) always the preferred way to do things. Log timestamps should also be UTC to avoid "nasties" with Summer Time and changing timezones. If you need to preserve the local time of an event, save the TZ as a separate item in the log entry.

Thanks, yes, that is all good advice! Though I didn't mean I wanted to offset the timestamps, but rather having access to the TZ when executing "date -s", so the system timezone would update "automagically". Speaking of which, I've given my "nodered" the right to set the system date/time by adding

Code: Select all

nodered ALL=(ALL) NOPASSWD: /bin/date

To /etc/suoders. While I'm not keen on punching holes in the rights assignment like this, it already needs

Code: Select all

nodered ALL=(ALL) NOPASSWD: /usr/bin/python

for Node-RED to function properly with the Pi, which seems far worse to me!

In other developments, I got the IVPort in the mail today; just the two port version to start with, since that's what they had in stock (and it's a wee bit cheaper less expensive than the 4-port version. Haven't had time to test anything - and I only have one CSI camera to hand - but if I can get things to work with this board, then the 4-port is a great future "upgrade". I also got the last two cable glands for the fancy-schmanzy cable entry system (see picture earlier in this thread), 2x 2x5mm, which should be perfect for ethernet cables:

R0012211.JPG (127.05 KiB) Viewed 3367 times

Bit busy with some other (paid) work at the moment, but I have also managed to produce a partial first "dashboard" for Node-RED. I have to say the more I use it the more impressed I am - and the clearer it becomes that I no longer have any need for Domoticz. Yes, goodbye old friend; after half a decade's trusty service, you are being usurped by a "newcomer". Seduced by its stunning looks, and thrilled by the minimalistic power of MQTT, I have busied myself with getting all the metrics I used to monitor in Domoticz onto a Node-RED dash. Although I still have the UPS data to deal with (needs to be polled with "upsc" and parsed), it's already looking fairly impressive:

Screenshot_2017-08-29_22-30-42.png (82.01 KiB) Viewed 3367 times

Once that migration is complete, I can get rid of Domoticz along with a whole bunch of supporting scripts and services, which should free up a fair amount of system resources (though it's not exactly as if the Pi is buckling under the load, yet). Then I want to get the IVPort working, and see if I can pull in scheduled snaps from each camera on the dash. Based on what I've been able to achieve in just a few hours I expect it will be a doddle. But lurking ahead I can see a far more daunting task: I will need to implement some form of database persistence for all this data! This seems to me the only omission in Node-RED; the fact that it knows nothing about the past. Restart the service, and even the graphs are emptied. I understand the concept of MQTT and that messages only live once, but I crave historical data to feed on! Having only made a few tentative web searches, and a tiny bit of reading, this looks like it may be quite complicated to achieve. Domoticz certainly has one up on Node-RED here; it logs everything without even having to be configured for it, and does it well, with automatic sparse data and min/max/avg calculation. If anyone has any pointers on how to achieve something similar in Node-RED I would love to hear about it!