How to read weather station sensor over 433MHz?

[Bosse_B]

I want to read the output (temp + humidity) of a couple of weather station sensors which use the 433 MHz frequency.
I have a receiver like this:

433MHzRx.png (84.88 KiB) Viewed 7397 times

The sensors are from a weather station (no brand except for the chain tha sell them) and they can be bought separately at a low cost.
The label states that they use 433 MHz. They can be set for 3 "channels", I think that is probably just an address in the transmitted packet.

I have searched the forum for information on the 433MHz receiver usage but not much useful found, mostly discussions on why it is not working and the link targets in these posts are very often non-existing...
Too old maybe?

What I would need is some advice, links etc to working examples and methods to decode the 433MHz signals from the weather sensors into readable figures of temperature and humidity.
Thanks.

[drgeoff]

Which is your problem To get the raw data stream into the RPi or to decode that into readable figures? If the latter then AFAIK there is no standard encoding used by all makers of sensors and stations.

[Bosse_B]

Which is your problem To get the raw data stream into the RPi or to decode that into readable figures? If the latter then AFAIK there is no standard encoding used by all makers of sensors and stations.

Getting the raw output into the Pi is just a matter of hooking the receiver output to a suitable pin on the Pi (with level shifting from 5 to 3.3 V of course.

But I think the real problem is:

1) Detect that data is coming in.
When there is no carrier the AGC circuitry probably increases gain so there is random output.

2) Decode the incoming data stream into the actual data packets.
Maybe the data is Manchester encoded or else there is some other system used....

3) Detect the data inside the packets (figuring out how the packets are constructed)

I hope there are some published ways to do these tasks...

A few years back I was experimenting with using the RPi to control mains switches, normally controlled from a remote control keypad.
Then I was using the transmitter unit and it was pretty straightforward to do with software from the web.
Never finished it though...

This current attempt deals with the opposite direction and also the task of figuring out how the data are actually represented in the packets from the sensor.

[Bosse_B]

I have hooked up the receiver on a breadboard now and connected an oscilloscope to its output.
The situation is as follows:
Whenever nothing is transmitting the receiver outputs a lot of noise.
When I activate the Tx button on the temp/humid sensor the data on the receiver output stabilizes into a bit stream.
Inside of this I have this as an example:

433MHzTempSensor.png (4.38 KiB) Viewed 7321 times

Total transmission length for one packet seems to be 117.6 ms.
Content?
1) There are interruptions of about 8.4 ms in the data that appears to be a sync
2) Between the syncs a data pattern repeats and looks almost identical all the time
3) There are pulses of width 480 us
4) The delay between pulses (front-to-front is either 2450 us or 4360 us

An interpretation of this could be 0=short delay, 1 = big delay

The sensor has an address switch that can be set to 1-2-3 and if I do this and repeat the readings I found that there are two bits that consistently change: Bit # 15-16 are 00 for setting 1, 01 for setting 2 and 10 for setting 3, seems pretty straight-forward.
But all in all I find the pattern consists of 37 bits, a pretty strange number...
The data shown on the weather station when testing is 22.6 C and 48%

Has someone here any idea what this could mean?

[Bosse_B]

Bump, double post edited

[PhatFil]

Ive used a github library for the arduino by a chap called Luc Small when using the same rf modules to pull temp and relay status from the brewfridge stc1000+ alt firmware.

rather than reinvent the wheel? and save all the headaches of the bit bashing, it may be expedient to spend a few £s on an arduino to slave to your pi.. (an afternoon modding example code and interfacing a serial or other connection between pi and arduino OR a Month of bit bashing???)
Your choice

https://github.com/lucsmall/WH2-Weather ... or-Arduino

[Bosse_B]

I planned on doing the detective work on the Pi but then porting the results over to Arduino using the ESP8266 WiFi module as target.
The ESP is already present serving wired sensors (3 of them) but I cannot get access to install more sensor wires anymore, the construction is now closed up too much.
I will look at the library you mentioned.
Thanks.

[btidey]

I think you will need to vary the temp and humidity a bit around the sensor to see how the data changes

Maybe something like first part is a fixed address followed by select address bits and then data/

Taking the bit encoding you suggest then from bit 17 onwards the nibbles are

0E631

If the first 3 were temp and next to were humidity that might have given 230 x 0.1C and 49;close to what display says but not quite right.

That is why you need to get more results.

[hippy]

I think you will need to vary the temp and humidity a bit around the sensor to see how the data changes

It's probably the only way to do it.

First write some code which can capture translate the live digital pulse packet into a binary line sequence of 0's and 1's. Then start noting which bits change and which don't as temperature and humidity change.

That should hopefully give some idea of where the bits for particular data lies. Don't go too over the top because too many readings may simply show all or most bits being used. Noting which bits change from one packet to the next can help.

Within that stream there will likely be some synchronisation or integrity bits, sequence numbers or checksum bits which change as the data does. There may even be battery voltage or a low battery indication included.

The good news is that it's likely a simpler encoding rather than anything complicated or encrypted. Numbers are likely sequential bits and simply scaled, pure binary of perhaps BCD. Note that endianness isn't certain but can probably be determined from which bits change as readings rise or fall.

The bad news is it's still a long slog unless you get lucky and are able to quickly spot where the data lies. On the other hand it gets easier to figure out the more you do figure out. You don't have to figure out checksum algorithms to start with. See where the data is and figure out the rest later.

Write the data packets you are capturing to a file. Then you can read those with programs which can perform the decoding you have determined and that can help show if you have got those right - a sequential 21C, 22C, 21C is probably close, 21C, 93C, 37C most likely isn't ! What's not right may lead you to what is.

With the right set of packets, a good idea of what they represent, you might even be able to craft a program which can try different decodings to figure out what the actual decoding is.

[Bosse_B]

OK,
that is what I figured too.
It was really tedious work capturing the packets with the oscilloscope and then manually inspecting the waveforms for 1 or 0 values.
I made 13 such deconings without being wiser except for the channel indicator bits I found inside the packet.
If I had designed the packet I would probably have had the channel number up front, but now it is bits 15-16...

Question:
In order to snatch the incoming data stream it seems like I need to sample at least at 100 us intervals.
Can this at all be done under Linux or should I switch to a stand-alone microcontroller?
Is there some kind of "recording" program on RPi that I can get via apt and use to record the incoming data to file?

Q2:
Given that the operating system is running, what is the sample rate one can expect out of a program running in a console on the Pi (with Raspbian)?
I have a hunch the OS will switch in regularly and screw up the timing...

PS:
The sensor is now back up in the attic and to read its data I need to check for times when consistent data (not noise) is present.
I think it transmits rather seldom in a burst...

[Raspberry Paul]

Have a watch of this video - https://www.youtube.com/watch?v=L0fSEbGEY-Q
How to Hack your 433 MHz Devices with a Raspberry and a RTL-SDR Dongle

A £6 RTL-SDR Dongle might be money well spent.

[Bosse_B]

Links in the description for the RTL-SDR Dongle did not work.
Seems to be a hard found item, at least for me in Sweden.
Is it not possible to use rtl_433 without an SDR dongle, just using a 433 MHz receiver hooked to a GPIO pin on the RPi?

[hippy]

In order to snatch the incoming data stream it seems like I need to sample at least at 100 us intervals.

Probably better to measure the time of the lows, or rising edge to rising edge, and report those on each rising edge.

Can this at all be done under Linux or should I switch to a stand-alone microcontroller?

I believe it should be possible; see below.

Is there some kind of "recording" program on RPi that I can get via apt and use to record the incoming data to file?

Possibly. But it might be better to develop your own code to do that as it's going to be fundamental to actually using these modules.

The pulses are half a millisecond, the gaps are multiple milliseconds, so they should be detectable even with a simple Python program. I would start with something like ...

Code: Select all

def LowTime(pin):
  do while Input(pin) == 1:
    pass
  elapsedTime = 0
  do while Input(pin) == 0:
    elapsedTime = elapsedTime + 1
  return elapsedTime
 
def Capture(pin):
  do while True:
    s = ""
    do while len(s) < 60: 
      t = LowTime(pin)
      s = s + " " + str(t)
    print( s )

Capture( PIN )

That should hopefully give you something like "101 50 20 22 20 49..." which allows you to determine the value for the synch gap and for 0 and 1 gaps. Those are just made-up numbers.

Then capture packets which start with the sync gap ...

Code: Select all

def Capture(pin):
  do while LowTime(pin) < SYNCH_TIME::
    pass
  do while True:
    t = LowTime(pin)
    do while t < SYNCH_TIME:
      s = s + " " + str(t)
      t = LowTime(pin)
    print( s )

Then change the "s=s+str(t)" into something which produces "0" or "1" ...

Code: Select all

      if t > K : s = s + "1"
      else     : s = s + "0" 

That might not work at all, but it's how I'd approach it. Those time consuming string 's' manipulations could be improved upon by using a binary integer.

[Bosse_B]

I still have the receiver connected to my oscilloscope and I found that off the air there are seemingly other transmitter signals...
It is hard to snatch because there is no way I have found to trigger on a reception because the receiver is outputting constant noise when there are no signal. I tried filtering the output somewhat but it does not help much so I ended up running in capture mode and trying to hit stop when I see signals on the screen.
This is two examples of what I got:

433MHz_dualtrace.png (8.69 KiB) Viewed 7110 times

Note that the time scale is different between the two traces and that the noise level is higher on one.
Probably I am seeing another sensor in the heighborhood...
The bottom trace I believe is from my own sensor in the attic...

Looks like an upwards battle doing this, might as well give up. (Especially since I am Python illiterate...)

[hippy]

I still have the receiver connected to my oscilloscope and I found that off the air there are seemingly other transmitter signals...
It is hard to snatch because there is no way I have found to trigger on a reception because the receiver is outputting constant noise when there are no signal.

Yes; the receiver will just output noise when there is no signal as its AGC ramps up, and can also pick-up other transmitter signals and interfering signals.

You have to do the filtering and discrimination on the data as it arrives and determine what is valid and what is not. It's not like a wired digital signal where one can expect it to be as expected and easily processed.

Where I live there is a burst of 433MHz interference every few milliseconds which I have not tracked down. I suspect it's a 'keep alive' signal from a neighbour's TV / RF remote or alarm system. That has stopped my DIY RF decoding on a microcontroller from working. Other commercial RF products communicate just fine and cut through that interference so I guess they are using different, more reliable and robust, means of filtering and decoding.

One of the advantages of using a microcontroller to process the incoming datastream and deliver received packets or post-filtered data to a Pi or other master controller is that can dedicate itself to the task of filtering the data-stream saving the master from doing that. A microcontroller will usually be better than a Pi or other computer because it will be more deterministic.

As you are finding; 433MHz RF can be a lot more complicated and difficult than it first seems.

[Bosse_B]

As you are finding; 433MHz RF can be a lot more complicated and difficult than it first seems.

Right you are!
I also tested my car key to see if it was indeed a 433 MHz device, and it was.
But to my surprise the protocol for the key seems extremely simple!
No wonder car thieves can easily record keys in parking lots and then open the cars.
One would have thought tah car key codes would be much more complex given the value of a car...

Here is my key (from top: Open, Close, Trunk):

433MHz-Keyfob.png (10.08 KiB) Viewed 7073 times

(I have pasted together 3 separate screens into this composite)

EDIT:
And by the way, concerning car keys:
My keys do not work at all if I park beside a construction site with cranes and offloading trucks using remote control from the driver handling the goods.
Apparently the truck remote is also on 433 MHz or nearby and totally jams my car key. Luckily I have an older car with a real lock that can be used to open the door and start the engine!

[drgeoff]

Car systems tend to use codes which change in a predetermined (but supposedly "secret") "random" sequence every time a button is pushed. The receiver has a "window" of allowable codes extending from the last valid one it received.

Try taking your wireless key out of range from the car and pressing a button lots of times. Come back to the car and you may find that it no longer responds to the remote. Your handbook should tell you how to resync - typically some variation of using the physical key and then using the remote.

[hippy]

No wonder car thieves can easily record keys in parking lots and then open the cars.

"Replay attack" is what we are relying on for using weather station or alarm sensors in our projects though usually in terms of receiving rather than sending - when that can be made to work.

Consistency of data bits is what lets us use those. We can simply ignore any sequence codes sent which are meant to provide security. Of course, if data is also encrypted by the sequence code, one has to also figure that encryption out to get at the raw data, even if we don't care what the sequence code itself is.

Car lock receivers should, as drgeoff notes, respect those sequence codes so a simple replay attack should not be possible. The trick for the criminals is to know what sequence code order a particular vehicle uses. Once a code has been captured, subsequent codes can be determined from that. They fake the next key press rather than the one just seen.

So in your scope capture; you could generate the first two, someone else can capture those and predict the third if they know the code sequence algorithm.