torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Raspberry Pi / Davis Vantage Pro2 weather station

Wed Dec 26, 2012 11:33 pm

I've finally found time to "port" the Python code from the BeagleBone project (last summer) to the Raspberry Pi. It works quite well, "talking" to the Davis Vantage Pro2 weather station via the expansion header at the back of the weather station. No USB stuff or Davis interface dongles involved, just using the /dev/ttyAMA0 serial device of the Raspberry Pi P1 header, communicating directly with the Davis device.

The result can be found here:
http://meteo.annoyingdesigns.com/

...and a rather detailed writeup will be availabe soon, too.

User avatar
freedomotic
Posts: 154
Joined: Sat Apr 21, 2012 3:59 pm
Location: Italy
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Fri Dec 28, 2012 10:32 am

Hi,
very interesting. Are you using raspbian or debian squeezy distro?
Regards
Freedomotic Open IoT Framework
http://freedomotic.com

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Fri Dec 28, 2012 11:04 am

freedomotic wrote:Hi,
very interesting. Are you using raspbian or debian squeezy distro?
Regards
Running with the default RPi stuff - seems to be pretty stable, too:

PRETTY_NAME="Debian GNU/Linux wheezy/sid"
NAME="Debian GNU/Linux"
ID=debian

blueberry314
Posts: 2
Joined: Sat Dec 29, 2012 6:19 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sat Dec 29, 2012 6:21 pm

Hi. I am really really interested in the configuration (as I just purchased a Vantage and a RPi). Can you please detail the connection and teh software configuration? A SD image would be probably too much trouble....)

Thank you

Bb314

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 12:51 pm

The wiring is the easy part. Three wires required. Both devices expect 3.3V.

Davis <---> RPi
GND <---> GND (P1 pin 6 - labelled GROUND)
RXD0 <---> TXD (P1 pin 8 - labelled GPIO14)
TXD0 <---> RXD (P1 pin 10 - labelled GPIO15)

For a test setup, you'll be good to go with ordinary "female" jumper wires. For a permanent setup, you'll probably want a better connector for the weather station hookup. I found these two parts to be extremely useful:

Samtec ESQT-110-02-G-D-760 ("flexible elevated socket strip"), available from Digi-Key Corp. (http://www.digi-key.com). This part "elevates" the console connector, making it easier to attach the next part, namely:

FCI 89947-720LF ("Connector, receptable, 2mm, 20way"), available from Farnell (http://www.farnell.com). Commonly referred to as a "flat cable connector, 20-pin" - it fits nicely onto the Samtec connector strip mentioned above without further ado. (Doesn't fit directly onto the console connector without a little bit of fitting (i.e. removing a small plastic tab). And ... please: use a small vise to attach the connector to the cable (even pressure distribution). Using any kind of pliers is guaranteed to NOT work. :?

You may want some ribbon cable as well, Digi-Key part # AE20B-5-ND fits nicely.

Working with the software, getting all the good stuff out of the console, takes a bit more time. But it's great fun and provides a good learning experience.

Notes:
- Python will need the python-serial package:

Code: Select all

[sudo] apt-get install python-serial 
- once the python-serial package is installed, it imports as "serial" - import serial
- com settings: 19200 bps, 8 data bits, no parity bit, 1 stop bit (19200, 8, N, 1)
- RPi serial device on /dev/ttyAMA0 (that's a zero at the end)
- all commands to the Davis console in UPPERCASE LETTERS
- all commands to be terminated with a single LF *or* CR (not both)
- console needs a few wakeup LFs
- interesting test commands: TEST, VER, NVER, STRMON, STRMOFF, BARCAL, LOOP 1
- you'd most likely like to "kill" all other serial output on the RPi ttyAMA0 device (kernel debug output, serial-line login prompt): /etc/inittab (one occurence) and /etc/cmdline.txt (two occurences) are the places to look
- Davis Instruments has published a serial communications reference manual. Grab a copy: http://www.davisnet.com/support/weather ... dllsdk.asp

Now, with the above setup as a starting point, you'll get going in no time.

Code: Select all

import serial, string, time, struct
# Serial port settings
WXBAUDRATE = 19200
WXBYTESIZE = serial.EIGHTBITS
WXPARITY = serial.PARITY_NONE
WXSTOPBITS = serial.STOPBITS_ONE
WXXONOFF = False
WXTIMEOUT = 3
WXDELAY = 0.2  # WX station communication delay

wx = serial.Serial(WXPORT, WXBAUDRATE, WXBYTESIZE, WXPARITY, WXSTOPBITS, WXTIMEOUT, WXXONOFF)
time.sleep(WXDELAY)

You'd probably like to have a function which takes care of writing stuff to the console:

Code: Select all

def wxWrite(s, termChar='\n'):
    """write s to wx, terminate string with termChar, then DELAY execution"""
    """NOTE: requires global wx to already be open"""
    if(wx != None):
        s = s + termChar
        wx.write(s)
        if(s == 'VER\n'):
            time.sleep(WXDELAY * 3) # VER seems to need more time?
        else:
            time.sleep(WXDELAY)
To read from the console, after first issuing a command which produces some output, just use:

Code: Select all

i = wx.inWaiting() # number of chars waiting in the RX buffer
s = wx.read(i) # read i chars from RX buffer
Reading the Davis LOOP packet, you'll probably want to decode it using the struct.unpack_from function.

Example:

Code: Select all

# assuming s holds entire LOOP packet 
barometer_INHG = round(struct.unpack_from('H', s, 5)[0] / 1000.0, 2)
GOOD LUCK and HAVE FUN!

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 4:16 pm

Checking the Davis documentation, they've updated their documentation and now also describe the use of the LOOP2 packet. That's interesting, it contains even more wind data (10-minute gust velocity, 10-min average wind speed, 2-min average wind speed) but it lacks the forecast icon value. It works, too, just read up on the details in the PDF document supplied by Davis.

Good luck!

blueberry314
Posts: 2
Joined: Sat Dec 29, 2012 6:19 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 4:37 pm

Fantastic. That will give me a bit of work over the holidays. Can you post the same data at your annoyingdesigns.com site? I assume the wxforum.net guys will go gaga over this (cheaper than weatherlinkip solution or modifying a router/having a dedicated computer by far).

Did you implement scripts for weather underground too?

Blueberry314

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 4:52 pm

The plan is to make a rather detailed writeup once everything is complete (it mostly is...), including all details from part numbers at Farnell to most of the Python code. All I need now is the time to put together all the documentation in a good-looking format. That's the boring part, I guess.

I considered the WeatherlinkIP but decided against one once I realized that it requires users to upload weather data to a Davis-owned & Davis-operated server (I bought a weather station from them, and the data coming out of it definitely belong to me!). That's my motivation for writing the code, and now it works fairly well. I just included support for the LOOP2 packet format, which I first read about this afternoon. Now I'm working with gnuplot to produce a decent-looking wind rose/wind plot.

As for Weather Underground, I haven't started looking into that part yet. I guess that won't be too hard, just a matter of finding enough time for it all. It's on the TO DO-list (ever growing...) for 2013.

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 4:59 pm

torkelmj wrote:I've finally found time to "port" the Python code from the BeagleBone project (last summer) to the Raspberry Pi. It works quite well, "talking" to the Davis Vantage Pro2 weather station via the expansion header at the back of the weather station.
I'm afraid that you'll probably find that this won't work with the latest (v3.00) console firmware, which is in all the current Vantage console (ie Vue/VP2/Envoy) production (and which sadly cannot be downgraded, as far as we know).

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 5:05 pm

Now, why is that?

It works great on the 1.90 firmware. The latest Davis documentation for the serial protocol (dated August, 2012) doesn't indicate any upcoming design changes. ...and was there ever a version 2.xx firmware around?

(Edit: firmware v. 1.90 still the only version available here: http://www.davisnet.com/support/weather ... _frmwr.asp )

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 5:14 pm

torkelmj wrote:Now, why is that?

It works great on the 1.90 firmware. The latest Davis documentation for the serial protocol (dated August, 2012) doesn't indicate any upcoming design changes. ...and was there ever a version 2.xx firmware around?
In short, Davis have now phased in 'green dot' consoles which are all loaded with v3.00 firmware. These now require compatible green dot Davis loggers. (The green dot is just shorthand for a physical green dot on the outer packaging which is a compatibility marker.) A non-Davis interface is no longer accepted as compatible. Almost all of the current station supply we're seeing is now fully green dot (though one or two dealers might still have old stock).

You'll need to look at a weather forum for a fuller picture, eg http://www.wxforum.net .

To answer your edit: v3.00 F/W is not (currently at least) compatible with pre-green-dot console hardware and so is not available as a download. But, I reiterate, all new station supply is v3.00 F/W. The highest previous VP2 F/W was indeed v1.90 as you correctly say, though Vue F/W went to v2.14 from memory.

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 5:53 pm

Very interesting. Indeed!

Especially the fact that Davis doesn't mention anything regarding an interface specification "lockdown" in their own documentation. Shutting off access to weather data in their weather stations will effectively make the unit completely useless ... even with existing Davis add-ons (which make use of the same serial communications protocol and commands as described in their documentation).

Yup. Here it is - including some of the frustrations caused:
http://www.wxforum.net/index.php?topic=17198.0

But I'm still not getting whether this applies to the entire rear interface - the wxforum posts I find all talk about "data loggers". Do these loggers use the serial interface or the SPI? Also, the Davis documentation v. 2.5 dates from about the same time as when the design change took place - and specifically states "version 1.90 firmware and later".

I'll fire off an email to Davis Instruments, just for fun.

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 6:29 pm

By all means email Davis if you wish though I'm not sure how detailed any reply might be. But we're Davis dealers (and 'johnd' on wxforum) and I can tell you most of what you might wish to know - that Davis are willing to make public at this stage at any rate.

All of the API commands will still work fine AFAIK with v3.00 F/W, the only stipulation is that you must use a green dot logger as the interface.

Davis documentation is notoriously slow to be updated and so it's completely unsurprising that it's a few months out of date. In fact if you look at the rev 2.52 serial tech ref closely you'll see that there is a whole new (and important) subsection listed in the Contents which isn't actually there as yet.

If there's anyone close to Cambridge UK who might be interested in following this up any further then I can make available a rev 3.00 Vue console for testing at our premises in Littleport (or potentially Ely). Long shot I know, but the offer's there. I'd do it myself but just zero time to spare at present. I'm away from base at present so contact me by email if interested at gprodata<at>gmail.com.

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 6:52 pm

Davis Instruments obviously needs to be made aware of terms such as "backwards compatibility" and "documentation". If nobody tells them, they'll never know there's a market demand for these things. My console. My data. Don't lock it down, it's a waste of time - sooner or later there will be a workaround.

I've only got the v. 2.5.0 of the serial protocol reference here - current version from the Davis website - published last July. Apart from the "web download protocol" I don't really miss anything in there.

As a sidenote, even the 2009 protocol specification (version 2.3, which I originally based my Python software on), says:
This document explains the serial data protocol (...). This requires a WeatherLink for Vantage Pro data logger and connector. It is not possible to communicate with the console without it.
Well. So much for the official Davis Instruments documentation. "Not possible" --- well. I'd say it isn't very hard. :?

The 2012 version only states:
This requires a WeatherLink with Standard Data Logger.
Let's see how things develop and what answer (if any) comes back from Davis Instruments. Thanks for pointing out this issue. I'm certainly most interested in any updates regarding the version 3 "lockdown" of the console and will keep an eye on the wxforum thread as well.

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 10:06 pm

torkelmj wrote:Davis Instruments obviously needs to be made aware of terms such as "backwards compatibility" ...
Like I can install iOS6 on my original iPad you mean? In the real world, compatibility of long-lived product lines often gets broken of course, usually for some sound reason. Why do you jump to the conclusion that the same is not true here?
Apart from the "web download protocol" I don't really miss anything in there."
That's a key part for many of us, in fact,
...It is not possible to communicate with the console without it.
You have to consider the audience that this was originally written for and when it was written. For the vast majority of users, it is still essentially true - not in absolute principle I agree, but in practice it is. Most developers reading this document will have some commercial application in mind and the cost of the official logger isn't usually much of an issue, but they need to be reminded that having the logger is, in say 98% of cases, the simplest way of making progress with their coding.
Let's see how things develop and what answer (if any) comes back from Davis Instruments.
Yes indeed, but I wouldn't hold my breath thinking that there's going to be a change in policy any time soon. As far as I can see, Davis are committed to making this change stick and we'll see what other benefits might ensue from the hardware revision during 2013 ;)

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Sun Dec 30, 2012 10:47 pm

I bet your iPad doesn't produce too much meaningful data on its own. At least mine doesn't. And I don't expect it to output too much interesting stuff. And ... reading the product details of the VP2 on the davisnet.com website, it still seems to be a mighty fine weather station. That's what it is. I can't imagine that FW v. 3 changes *that* much of its functionality. As such I really can't see *any* plausible reason for ditching backwards compatibility, if that's what they've done. Surely, they are well aware of the market appreciation and many different uses of their devices, be it with original or 3rd-pary add-ons.

The point here is that the Davis Vantage Pro(2) console is known - among other things - for its data output capabilities. Davis changes that overnight, effectively leaving a lot of original and 3rd-party stuff useless. No prior notification. OK, they'll be able to hide behind the "we never said it would work without a Davis data logger" clause in the serial protocol reference. But *is* that really in their own interest?

Davis should certainly get some feedback on this issue. AND appreciate it. They already have a reputation for building quite OK stuff. They should know better than ruining that reputation.

Locking down data in proprietary formats or attempting to hide data behind some obscure mechanisms is neither in customers' interest nor in the long-time interest of Davis Industries, especially if the primary intention is to sell more overpriced stuff just to enable people to access their very own data. Oh, that's so 1990-ish. :( (I bet it's a matter of weeks until we'll see a detailed HOWTO on how to get past any security-by-obscurity mechanisms anyway).

Now, if turns out that the Davis devices with the new firmware version won't give out their data for free, I'm done with Davis Instruments. Unless that's a design choice made by a mistake. Let's hope so, it would be really sad to see these fine devices have to rely on Davis-only stuff ($$$) to communicate with the world around them.

Hm? Did I hear "1-wire" and "put together your own sensor package"? ;)

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Dec 31, 2012 10:15 am

torkelmj wrote: ...The point here is that the Davis Vantage Pro(2) console is known - among other things - for its data output capabilities. Davis changes that overnight, effectively leaving a lot of original and 3rd-party stuff useless....
I suspect this thread is going well beyond relevance to the RPi forum here, but I'll just pick up this particular point because I think it's a gross misrepresentation of what's actually happened. Neither the data format nor the API has changed in any respect as far as I can see - all programs that used to work will continue to work with the new firmware. The single proviso is that you must use the official interface.

In fact, up until a year or so ago, no-one in the wider world was aware that there was any option but to use the official interface. Then one or two users discovered for themselves that there is an undocumented alternative interface option. I can understand why you're upset, but it's well-known in developer-land that if you choose to use an undocumented feature, which the manufacturer then changes - as is their right, then there aren't too many grounds for complaint.

Beyond this, I think I covered all the other points in my previous post and won't repeat them.

FWIW I suspect that someone will post a workaround sooner rather than later - we'll see.

Finally, let me add that I think the Pi could be a very useful addition to a Davis AWS, but just not in the specific role that you're proposing here (for new-build stations at least).

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Dec 31, 2012 1:04 pm

Very valid points.

STILL - your bring up another interesting issue. Why on earth would Davis Instruments restrict access to the console, forcing the use of their own dongles only - effectively crippling a popular product? Unfortunately, there's only one answer to that question: $$$. They're biting their own tail with this (assumed) policy, making a desperate (and useless) move to market their overpriced WeatherLink stuff.

I'll do fine with my 1.90 consoles. But I won't buy, install or recommend any Davis devices until this issue has been resolved. Thanks for pointing it out - and have a very happy new year!

prodata
Posts: 160
Joined: Tue Jan 24, 2012 5:53 pm
Contact: Website

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Dec 31, 2012 2:04 pm

torkelmj wrote:Unfortunately, there's only one answer to that question: $$$.
This is probably the nub of why a few hobbyists are upset. (Remember that great majority of Davis customers - probably >95% - are either business users in one shape or form or weather hobbyists with relatively little interest in computer hardware, both categories of whom are generally happy with the official interface.)

But two points on cost. First, the official logger is really not so costly - about GBP85+VAT in the bundle that's usually bought in the UK (and bearing in mind that a VP2 station will typically cost in the range GBP500-1000 depending on spec).

Second, I'd argue that you have to look at the overall profitability of the stations to Davis. Davis will be looking at net profit across the range of station sales and they will have a % figure that keeps them happy after all costs, including support, have been deducted. (I've no idea what that % might be.) You may well be right that there is a high margin on the logger unit. But that's presumably because they have made a decision, possibly for marketing reasons, to accept a lower margin on the stations themselves. IF (and it's a big if, over which I have major doubts) Davis have made the v3.00 F/W changes solely as an anti-piracy measure then the only other step they could have taken would be to have dropped the price of the logger and increased the price of the stations. Maybe that's what they should have done, if I accept your premise for a moment?
...have a very happy new year!
And to you.

And please remember that the only point I'm picking up on here is the issue of whether the official interface can be circumvented. I think that the general principle of using a Pi to handle the data from a Davis AWS is an excellent one (which is the prime reason that I have a Pi :) )

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Tue Jan 01, 2013 8:05 am

But two points on cost. First, the official logger is really not so costly - about GBP85+VAT in the bundle that's usually bought in the UK (and bearing in mind that a VP2 station will typically cost in the range GBP500-1000 depending on spec).
I'd say that's rather costly for a device which (a) does very little, (b) shouldn't be required in the first place and (c) contains parts for ... GBP10? I don't mind paying what it takes for a proper weather station, and if Davis is so desperate for some extra $, they could as well include the interface on the main board of the console. Which, to a certain extent, is what they did on the 1.90 firmware versions - except for the logger part.
But that's presumably because they have made a decision, possibly for marketing reasons, to accept a lower margin on the stations themselves.
Being well acquainted in Davis-land, I bet you've had several looks at the inside of the VP2. Not too many expensive parts in there. I'd say they have a quite acceptable profit margin on their weather station range. That is, if the price I paid for one over here is representative for the general price level of Davis products.
IF (and it's a big if, over which I have major doubts) Davis have made the v3.00 F/W changes solely as an anti-piracy measure then the only other step they could have taken would be to have dropped the price of the logger and increased the price of the stations. Maybe that's what they should have done, if I accept your premise for a moment?
Just for the record: there's no piracy involved in reading data directly from the VP2.
And, we're talking firmware. That's software, meaning that what they have done can easily be undone - if they so desire. If Davis is desperate for $, why won't they rather make their products as attractive as possible - instead of wasting R&D efforts crippling them with this "upgrade"?
And please remember that the only point I'm picking up on here is the issue of whether the official interface can be circumvented.
It can. It will. It should. Just a matter of time... :geek:

jhony251
Posts: 1
Joined: Thu Feb 28, 2013 9:42 am

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Thu Feb 28, 2013 9:51 am

Hi Torkelmj

I am very interested in your develop.
Now i am trying to get a very cheap way to send to a server the information generated by a vantage pro 2.

Could you get contact with me please by PM to send you my email?

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Thu Mar 14, 2013 11:29 pm

There are quite a few details available here:
http://meteo.annoyingdesigns.com/WOSPi.pdf

Actually, it's a complete guide to everything you need to get going.

...and the time budget, if a Davis Vantage Pro2 Plus is already installed:
- 1 hr of online shopping for required hardware
- 1 hr for assembling connectors, etc.
- 1 to 2 hrs for software setup

Enjoy.

User avatar
hamashi2
Posts: 6
Joined: Mon Jul 01, 2013 10:10 am

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Jul 01, 2013 10:28 am

Hi,
Exactly what data does s hold? Yeah, weather data obviously, but I mean if you print s out as a string, what does it look like? I'm supposed to do pretty much this same thing, but I don't have the weatherstation. I've looked for the documentation, but either It doesn't have raw example of what does loop 1 return as reply.
torkelmj wrote:

Code: Select all

# assuming s holds entire LOOP packet 
barometer_INHG = round(struct.unpack_from('H', s, 5)[0] / 1000.0, 2)

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Jul 01, 2013 11:09 am

As a string ... it doesn't look pretty.

torkelmj
Posts: 25
Joined: Wed Dec 26, 2012 11:10 pm

Re: Raspberry Pi / Davis Vantage Pro2 weather station

Mon Jul 01, 2013 12:46 pm

BTW, here is a writeup of stuff which may be of use to those troubled by firmware version 3.xx:

http://meteo.annoyingdesigns.com/DavisSPI.pdf

Return to “Automation, sensing and robotics”