NMEA MULTIPLEX

[Pavo_46]

Hi there!!
This is my first time on a forum so bare with me. I am putting together a project using a raspberry pi and python to collect ASCII data from a GPS and echo sounder, to collect geolocated depths. Now i have hit a serious brick wall, i have managed to write the following script which (altering the baud rates) works for both devices, BUT heres the problem, how do I run both devices at the same time and output a single file? Or how do I automatically flip/flop between the two devices and collect one round of data at a time? (this is what I got so far)

Code: Select all

from __future__ import print_function
import serial, io

addr  = '/dev/ttyUSB0'  # serial port to read data from
baud  = 9600            # baud rate for serial port
fname = 'log.dat'   # log file to save data in
fmode = 'a'             # log file mode = append

with serial.Serial(addr,baud) as pt, open(fname,fmode) as outf:
    spb = io.TextIOWrapper(io.BufferedRWPair(pt,pt,1),
        encoding='ascii', errors='ignore', newline='\r',line_buffering=True)
    spb.readline()  # throw away first line; likely to start mid-sentence (incomplete)
    while (1):
        x = spb.readline()  # read one line of text from serial port
        print (x,end='')    # echo line of text on-screen
        outf.write(x)       # write line of text to file
        outf.flush()        # make sure it actually gets written out

Thanks for any help,
Paul

[topguy]

Are there going to be two different devices, ttyUSB0 and ttyUSB1 when both devices are connected to the Pi at once ?
How often is the "echo sounder" sending out data ?

A GPS usually just send out data once every second. So what you could do is:
* open both devices for reading.
* start reading lines from GPS but use a timeout of lets say 250ms.
* when readline times out with no data you can assume that the GPS is pausing for its next batch of nmea lines.
* then start reading from the "echo sounder" also with a suitable timeout,
* after all the available data is read and you get a new timeout then switch back to GPS.

Another possible solution which is more advanced but more elegant:
* You start two threads where each thread is responsible for reading from one of the devices.
* The trick is then to figure out how to synchronize writing to the log file.
* I c/c++ i would probably use a global file descriptor and a semaphore to control that the two threads wasnt trying to write to the log file at the same time. I'm no Python expert so I dont feel up to recomending a proper method for Python.

[Pavo_46]

thanks for the reply!!

the echo sounder and gps will be connected to the Pi at the same time using usb-RS232 adapters. echo sounder is running at 1 Hz and with a baud of 115200, where as the GPS has a baud of 9600

[ame]

thanks for the reply!!

the echo sounder and gps will be connected to the Pi at the same time using usb-RS232 adapters. echo sounder is running at 1 Hz and with a baud of 115200, where as the GPS has a baud of 9600

You're not really multiplexing. You have two devices connected to two ports. Just open a different file handle for each one:

Code: Select all

addr0  = '/dev/ttyUSB0'  # serial port to read data from GPS 
baud0  = 9600
addr1  = '/dev/ttyUSB1'  # serial port to read data from echosounder
baud1  = 115200  

In your program, open both ports, read data from each, write to file.

[Pavo_46]

hi there,
latest attempt (code below)

Code: Select all

from __future__ import print_function
import serial, io

addr0 = '/dev/ttyUSB0'
addr1 = '/dev/ttyUSB1'
baud0 = 9600
baud1 = 115200
fname = 'RaspSurvey-Output.txt'
fmode = 'a'

with serial.Serial(addr0,baud0,addr1,baud1) as pt, open(fname,fmode) as outf:
    spb = io.TextIOWrapper(io.BufferedRWPair(pt,pt,1),
        encoding='ascii', errors='ignore', newline='\r',line_buffering=True)
    spb.readline()  # throw away first line; likely to start mid-sentence (incomplete)
    while (1):
        x = spb.readline()  # read one line of text from serial port
        print (x,end='')    # echo line of text on-screen
        outf.write(x)       # write line of text to file
        outf.flush()        # make sure it actually gets written out

when run in the console i receive the following error message:

Code: Select all

>>> 

Traceback (most recent call last):
  File "/home/pi/Desktop/dissertation_files/RaspSurvey.py", line 11, in <module>
    with serial.Serial(addr0,baud0,addr1,baud1) as pt, open(fname,fmode) as outf:
  File "/usr/lib/python2.7/dist-packages/serial/serialutil.py", line 249, in __init__
    self.bytesize = bytesize
  File "/usr/lib/python2.7/dist-packages/serial/serialutil.py", line 331, in setByteSize
    if bytesize not in self.BYTESIZES: raise ValueError("Not a valid byte size: %r" % (bytesize,))
ValueError: Not a valid byte size: '/dev/ttyUSB1'

anyone have an idea of this?
thanks in advance

[scotty101]

You can't use serial.Serial in that way. Try something like this.

Code: Select all

gps = serial.Serial(port1,baud1)
echo = serial.Serial(port2,baud2)

[Pavo_46]

this has improved the issue

i used the following code

[code]from __future__ import print_function
import serial, io

addr0 = '/dev/ttyUSB0'
addr1 = '/dev/ttyUSB1'
baud0 = 9600
baud1 = 115200
GPS = serial.Serial(addr0,baud0)
ECHO = serial.Serial(addr1,baud1)
fname = 'RaspSurvey-Output.txt'
fmode = 'a'

with GPS and ECHO as pt, open(fname,fmode) as outf:
spb = io.TextIOWrapper(io.BufferedRWPair(pt,pt,1),
encoding='ascii', errors='ignore', newline='\r',line_buffering=True)
spb.readline() # throw away first line; likely to start mid-sentence (incomplete)
while (1):
x = spb.readline() # read one line of text from serial port
print (x,end='') # echo line of text on-screen
outf.write(x) # write line of text to file
outf.flush() # make sure it actually gets written out[/code]



this ran fine in the terminal, but only the echo sounder data was displayed and written to the output file, the gps data is being ignored, any help?

[code]>>>

$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10
$PRVAT,00.233,M,,*10[/code]

[paulie]

I have an Adafruit ultimate GPS connected to the serial UART.
I use the gpsd package to collect the data, which allows multiple programs to share the data.
I believe it suggests the GPS tools package, which allows you to check your GPS is working.
Viewing the raw data will show you have the baud rate correct, and if you have a position fix.
When you have both devices sending valid data independently, you can then link them into your code.

[Pavo_46]

I have an Adafruit ultimate GPS connected to the serial UART.
I use the gpsd package to collect the data, which allows multiple programs to share the data.
I believe it suggests the GPS tools package, which allows you to check your GPS is working.
Viewing the raw data will show you have the baud rate correct, and if you have a position fix.
When you have both devices sending valid data independently, you can then link them into your code.

So what are you suggesting?

[DougieLawson]

thanks for the reply!!

the echo sounder and gps will be connected to the Pi at the same time using usb-RS232 adapters. echo sounder is running at 1 Hz and with a baud of 115200, where as the GPS has a baud of 9600

What happens when you reboot and /dev/ttyUSB0 gets switched with /dev/ttyUSB1 (because of the way the USB devices get enumerated during boot up)?

Can you post the output from a usb-devices command?

[paulie]

So what are you suggesting?

Here is the website for gpsd

http://www.catb.org/gpsd/

gpsd is available in Raspbian.
Good instructions can be found here:

http://blog.retep.org/2012/06/18/gettin ... pberry-pi/

HTH

[ame]

this has improved the issue

i used the following code
<snip>

I think your 'with' statement syntax is not correct, and the whole thing is too terse for you to understand.

I don't think you should use gpsd yet, until you have something working. I also wouldn't worry about the order in which the serial ports are created until then (but it's worth knowing it's an issue you might have to deal with). Also, don't bother with threads.

There are several ways to solve your problem.

1. Have two programs. One for each device, writing to a common file. UNIX's file locking will prevent them writing at the same time.
2. Have one program which opens and listens to two serial ports. After getting valid data from each port, write two lines to the log file.
3. Replace one of the serial ports with gpsd.
4. ...

I suggest option 2, and that you get rid of 'with', and just open the two serial ports with their own file handle. Then, have an infinite loop that tries to read a line from the file handle. If a line is received, parse it. If it's bad data, or not the data you want, then discard it. Otherwise, write the parsed data to the log file.

To stop your program getting stuck waiting for data, use select() to see which serial port has data waiting, but even that is not needed the first time around, since you can be sure your devices will produce new data every second.

[Pavo_46]

I think your 'with' statement syntax is not correct, and the whole thing is too terse for you to understand.

I don't think you should use gpsd yet, until you have something working. I also wouldn't worry about the order in which the serial ports are created until then (but it's worth knowing it's an issue you might have to deal with). Also, don't bother with threads.

There are several ways to solve your problem.

1. Have two programs. One for each device, writing to a common file. UNIX's file locking will prevent them writing at the same time.
2. Have one program which opens and listens to two serial ports. After getting valid data from each port, write two lines to the log file.
3. Replace one of the serial ports with gpsd.
4. ...

I suggest option 2, and that you get rid of 'with', and just open the two serial ports with their own file handle. Then, have an infinite loop that tries to read a line from the file handle. If a line is received, parse it. If it's bad data, or not the data you want, then discard it. Otherwise, write the parsed data to the log file.

To stop your program getting stuck waiting for data, use select() to see which serial port has data waiting, but even that is not needed the first time around, since you can be sure your devices will produce new data every second.

This sounds great and is exactly what I am aiming for, but having serious difficulty working out how to do this, mainly writing this loop as so far only worked with the with/while loop

[ame]

All right, here is a very basic framework to illustrate the idea. It doesn't use anything fancy, and assumes there are no errors. In fact, error checking is what makes your software great, so that it either does the Right Thing in case of an error, or at least doesn't crash.

Code: Select all

#!/usr/bin/env python

import serial

# Specify serial ports
addr0 = '/dev/ttyUSB0'
addr1 = '/dev/ttyUSB1'
baud0 = 4800 #9600
baud1 = 4800 #115200

# Open serial ports
GPS = serial.Serial(addr0,baud0,timeout=1)
ECHO = serial.Serial(addr1,baud1,timeout=1)

while True:
        gps_data=GPS.readline()
        echo_data=ECHO.readline()

        if gps_data[0]=="$":
                print "GPS:  %s"%gps_data

        if echo_data[0]=="$":
                print "ECHO: %s"%echo_data

I tested this as I happened to have two USB GPS receivers. As you can see, it doesn't really do anything except open the ports (with a 1 second timeout) and continually read lines and print them.

To deal with a partial read (either junk character at the beginning, or halfway through an NMEA sentence) I do a simple check that the string begins with "$". All NMEA strings begin with "$" and end with a checksum. If you like, you can write a function that takes the string and verifies that the checksum is correct.

Presumably you wish to log the position from the GPS and the depth from the echo-sounder. Once you can get the strings into your program it's very easy to pull-apart the $GPRMC sentence from the GPS and the $PRVAT sentence from the echo-sounder. You can either store the whole sentence, or log lat/lon/depth (and date/time) to your logfile.

[Pavo_46]

All right, here is a very basic framework to illustrate the idea. It doesn't use anything fancy, and assumes there are no errors. In fact, error checking is what makes your software great, so that it either does the Right Thing in case of an error, or at least doesn't crash.

Code: Select all

#!/usr/bin/env python

import serial

# Specify serial ports
addr0 = '/dev/ttyUSB0'
addr1 = '/dev/ttyUSB1'
baud0 = 4800 #9600
baud1 = 4800 #115200

# Open serial ports
GPS = serial.Serial(addr0,baud0,timeout=1)
ECHO = serial.Serial(addr1,baud1,timeout=1)

while True:
        gps_data=GPS.readline()
        echo_data=ECHO.readline()

        if gps_data[0]=="$":
                print "GPS:  %s"%gps_data

        if echo_data[0]=="$":
                print "ECHO: %s"%echo_data

I tested this as I happened to have two USB GPS receivers. As you can see, it doesn't really do anything except open the ports (with a 1 second timeout) and continually read lines and print them.

To deal with a partial read (either junk character at the beginning, or halfway through an NMEA sentence) I do a simple check that the string begins with "$". All NMEA strings begin with "$" and end with a checksum. If you like, you can write a function that takes the string and verifies that the checksum is correct.

Presumably you wish to log the position from the GPS and the depth from the echo-sounder. Once you can get the strings into your program it's very easy to pull-apart the $GPRMC sentence from the GPS and the $PRVAT sentence from the echo-sounder. You can either store the whole sentence, or log lat/lon/depth (and date/time) to your logfile.

Adding a output file in append mode and using the f.write and f.flush functions with your help has got me to my goal, thanks you very much!

[ame]

You're welcome. Glad to hear it's working.