Ublox Neo 6M GPS module not changing coords.

[Dranoweb]

Hi,

I'm currently building a GPS logging geiger counter, using a raspberry pi zero w and a Ublox Neo 6M GPS module.
Connected via serial, not USB

Interfacing is happening fine, but I have a weird issue that I cannot find any reference to anywhere.

My waypoints are "clustering", and by that I mean that when I log points, they don't change locations from the first fix of that session
even when i move kilometers from that point, and it seeing up to 9 satellites at times.

The memory battery on the board is shot and I have spares to change that over later.

I'm using pynmea2 to glean the gps data from the NMEA serial data at 9600 baud.
Originally I had though that using the latitude output gave me a decimal degrees output

however when I saw this anomaly I realized that maybe I was just getting the degrees component as a decimal, and not the rest of the location.
I would have to move a very long way to see the location change

So I used the dm_to_sd function of pynmea2 to translate it to a float for decimal degrees.
on the surface and for the first few static points this was spot on correct.

so I went for a walk around the block, and all my coords log within a 20sq/m area - any identical in reading.
despite the fact that I walked almost 1000 meters.

What do you guys this? is this a programming issue, a hardware problem, or an idiot (Me) problem?

I'm using python to do the logging and here;s the code:

Code: Select all

import threading
import time
import os
import serial
import pynmea2

def parseGPS(str):
    if str.find('GGA') > 0:
        msg = pynmea2.parse(str)
        print "Timestamp: %s -- Lat: %s %s -- Lon: %s %s -- Altitude: %s %s" % (msg.timestamp,msg.lat,msg.lat_dir,msg.lon,msg.lon_dir,msg.altitude,msg.altitude_units)
        print (msg.altitude_units)

serialPort = serial.Serial("/dev/ttyAMA0", 9600, timeout=0.5)

while True:
    str = serialPort.readline()
    parseGPS(str)

[topguy]

If you also print out the GGA string, and also the RMC strings, you could verify that the position data from GPS are changing.
Then you could conclude that the problem has to be somewhere else.

https://www.trimble.com/OEM_ReceiverHel ... s_GGA.html
https://www.trimble.com/OEM_ReceiverHel ... s_RMC.html

[Dranoweb]

I had arrived at a similar idea in the last hour or so, so yes will try that.
Unfortunately I will have to wait a few hours to test that as I can't leave my post, so to speak.

[Dranoweb]

Another consideration I just had - Epoch rollover.
a once every 20 years event with GPS modules where the 10 bit week number value resets.

That happened back in April. Perhaps I need to update the module's firmware.

though I I hear rumors that the ublox units are immune - but cannot confirm it

[bensimmo]

as above, check the stream from the gps is correct and changing (and not getting locked by something else)
What's you GPS setup, UART pin14/15 ?
How did you set it up?

anyway, just log directly from the /dev/serial0 or /dev/USB01 etc 'file'

after that, if the file seems correct, you can stream it into your program (see pynmea2) to checc it without moving anywhere.


as for epcoh, I don't think the Ublox6m are old enough for a fix to have not been in place from the start.
We'd have seen many more report of time problems. Is your time correct ? the coords shouldn't be wrong though.

[topguy]

Another consideration I just had - Epoch rollover.
a once every 20 years event with GPS modules where the 10 bit week number value resets.

Its unlikely, but you will see in the RMC message also I think that the date is wrong. Many GPS still work with outdated chipsets but give wrong date ( 1024 weeks old date )
Also you code doesnt really check if the GPS has a lock or not, it just print values reagrdless.

[Dranoweb]

My code checks to see if there is an elevation unit type "M" in the GGA string - this is not possible without a 3d fix.

Having just wandered around in the dark, I found that the date and time is shown correctly in the NMEA data over the serial.

So I think I can rule out epoch issues.

the fix does seem very intermittent - perhaps it is caching locations...

However having driven it several kilometers and given it time to settle and get a new fix, I'm skeptical.

I have used CGPS on screen as I walk around to determine if the coords are changing, and based on a short walk of about 100 meters, I can confirm that the coords do change and it determined that I walk around 12km/h which I know to be fairly normal.

I really wonder if there is some rounding going on.

[rpiMike]

Your code appears to be Python2, i would suggest moving to Python3. There may be bug fixes in newer code.

I was confused with your code because serial.readline returns bytes in Python3.

[Dranoweb]

I had intended it to be python3 - though I'm still somewhat new to the language.
and I have python3 installed - but the question is going to end up at "how does one do that exactly?"

the code is working quite smoothly for my purposes, and the string for "gps.write" was changed as the dd_to_sd function returned a float variable that I couldn't concatenate to another string without first converting it, so I kind of fell back to old methods there.
I'm also modifying examples to get to this point before further refinements.

Priority is getting a proof of concept happening first.

The main issue I have is that all coord values are recorded within a 20sqm area despite moving several kilometers and allowing it to regain a 3d fix for as much as half an hour.

and so the exercise here it to try to narrow the potential problem area, be it hardware, or software - or of course, human error

[rpiMike]

Your python2 code does work on my ublox C94-M8P connected via usb.

[Dranoweb]

I may have an idea that I had not spotted until laying this question out in the forum.


my final loop has nothing to slow it down. this could be overwriting coords before they are refreshed, effectively rounding or averaging the reading.

I'm going to reinstate the half second sleep and try with that - just that the weather has now changed and with the tiny antenna on the unit, I may have to wait till morning.

[Dranoweb]

This could potentially be a google earth import error - stay tuned while I adjust my pc with a log splitter....

I'll post back with my findings...

[B.Goode]

This could potentially be a google earth import error - stay tuned while I adjust my pc with a log splitter....

I'll post back with my findings...

First and only mention of Google Earth in this thread...


Perhaps if you give the volunteer helpers in these forums the whole picture you would be more likely to get a reply appropriate to your issue...

[Dranoweb]

It's writing to a CSV file.

As a checksum, I am importing said CSV file to google earth as a checksum.

This first mention of it is because I am only now using it to try to fault find - and was not doing so at the time of the original post.

I have added several of these points manually at this stage, and they match up with the truncated, imported coords, despite them only having 3 and 4 decimal places respectively.

[DougieLawson]

Why the duplicate thread? https://www.raspberrypi.org/forums/view ... 2&t=252139

[Dranoweb]

Setup:
Raspberry pi zero w
RTC attached by SPI
Geiger counter attached to GPIO via opto-isolator
Neo6M GPS attached by serial to GPIO with 2db gain passive antenna

Problem:
Waypoints are recorded as a "cluster" within a 20sq/m area around the first fix.
driving several kilometers away and waiting in clear open sky for 3d fix, still returns coords in this same area.

Solutions attempted:
Switching from NMEA coords to DMS and later to DD
using pynmea2 to build strings instead of directly parsing NMEA Data
using pynmea2 dd_to_SD function to get floating point DD
Replacing neo6m memory battery
slowing serial read loop (as its in a separate thread)
manually converting NMEA coords from GGA line in serial data (on paper)

outcome:
unchanged.

Why the duplicate thread - not a duplicate.
Different problem. that was working out how to concatenation a float to a string - which I resolved and am grateful for the assistance
I had originally thought this caused this issue.
Having successfully determined that the translation of the coords was correct - and that there are no errors in the math
I'm now working on the hardware aspect regarding this.

I felt that this was a distinctly different issue as it was likely hardware or interfacing related.

I am essentially out of ideas, short of waiting another month for yet another GPS unit, of which I now have 5.
I have spent 3 months on this project and my window of opportunity is running out, as I will no longer have access to the area I wish to survey.

I appreciate the help -

[Dranoweb]

OK so I'm close to a rant about various aspect of forums and post numbers, and experience levels etc - something you have all probably heard or seen before.

I'm going to call it quits for a day or two and come back to this when i have resolved some other issues.
I recently suffered an attack of MS that created yet another lesion on my brain.
as a result, typing is becoming extremely difficult - as are emotions.

None of which are going to help me to convey my gratitude or remain focused on the task at hand.

I'll speak with you all later about this.

[rpiMike]

What output do you get if you run the following?

Code: Select all

import serial
import pynmea2

serialPort = serial.Serial("/dev/ttyAMA0", 9600, timeout=1)

while True:
    str = serialPort.readline()
    if str.find('GGA') > 0:
        print str.strip()

[wolf.z]

On the risk of carrying coals to Newcastle: You are saying you checked with cgps, and everything was fine. Since cgps is a gpsd client, I suggest you are trying to read from a serial port, a running gpsd is using already. If this is the case, stop gpsd and retry. However, reading from gpsd rather than reading from ublox will give you all the advantages of tens of years of experience in handling GPS receievrs correctly.

[Dranoweb]

On the risk of carrying coals to Newcastle: You are saying you checked with cgps, and everything was fine. Since cgps is a gpsd client, I suggest you are trying to read from a serial port, a running gpsd is using already. If this is the case, stop gpsd and retry. However, reading from gpsd rather than reading from ublox will give you all the advantages of tens of years of experience in handling GPS receievrs correctly.

I have the facility to run cgps manually. once I do this, it prevents other applications from accessing the serial data.
as one would expect. at this point when I have confirmed that I have a fix (that takes about 2 hours) I restart the raspberry pi component of the device and switch back to using the script above.

After recording several points on paper as read by cgps - the same behavior is observed - leading me to believe it is a hardware or interface issue.

What output do you get if you run the following?

Code: Select all

import serial
import pynmea2

serialPort = serial.Serial("/dev/ttyAMA0", 9600, timeout=1)

while True:
str = serialPort.readline()
if str.find('GGA') > 0:
print str.strip()

I'll get back to you in 2 hours on that, as I need to do a cold start again.
the replacement battery for the neo6m module also exploded (just like the first)
before you ask - yes it's only getting 3.3v, this was checked thoroughly the first time.

I also have several additional modules, two of them will not achieve a fix under any circumstances, with any combination of antenna.

Overall I feel as though my gps module has issues.

I have received some communication as I woke this morning from an ex-coworker who pointed out that I have no time delay when buffering the serial data, and that this may be the source of my issue.

So I now have a slightly more clear head, have taken some medication, and have a number of solutions to test.
I will keep you posted.


FYI: The results of this device may be used as evidence to help a veteran get a DVA gold card, which will assist greatly as he has gone blind as a result of his service. - hence why this is both an urgent and emotional project.

[Dranoweb]

What output do you get if you run the following?

Code: Select all

import serial
import pynmea2

serialPort = serial.Serial("/dev/ttyAMA0", 9600, timeout=1)

while True:
    str = serialPort.readline()
    if str.find('GGA') > 0:
        print str.strip()

The code executes perfectly, and outputs the GGA string.
I have modified the last digit of the minutes for obvious reasons:

$GPGGA,022659.00,3905.70057,S,14804.27687,E,1,06,1.51,4.7,M,0.3,M,,*79
$GPGGA,022700.00,3905.70054,S,14804.27691,E,1,06,1.51,4.8,M,0.3,M,,*7F
$GPGGA,022701.00,3905.70062,S,14804.27706,E,1,06,1.51,4.9,M,0.3,M,,*75
$GPGGA,022702.00,3905.70059,S,14804.27711,E,1,06,1.51,4.9,M,0.3,M,,*78
$GPGGA,022703.00,3905.70062,S,14804.27724,E,1,06,1.51,5.1,M,0.3,M,,*7E
$GPGGA,022704.00,3905.70063,S,14804.27726,E,1,06,1.51,5.0,M,0.3,M,,*7B
$GPGGA,022705.00,3905.70068,S,14804.27742,E,1,06,1.51,5.1,M,0.3,M,,*72

[Dranoweb]

I just went for a walk after stepping through literally every element of the project bit by bit.

Here's what happened:

I altered the serial read loop to include a 1 second delay:

Code: Select all

while True:
    time.sleep(0.1)
    outputStr = serialPort.readline()
    parseGPS(outputStr)

I then went for a walk around the block (sort of)
And imported the resulting csv file into google earth pro.
I indicated that the counts per minute should be the waypoint name column.

Crucially I noticed that google earth has truncated my coords to 4 decimal places.
I now notice that I have a clearly formed track, in the shape of the path I followed, but scaled down something like 10times (visual estimate)

could there be some kind of rounding or scaling happening somewhere here?

In regards to using CGPS to do this - that would be fine, but I want to include other data than just the coords, and was unsuccessful with numerous attempts to acheive this with cgps or associated tools.

[Dranoweb]

On the risk of carrying coals to Newcastle: You are saying you checked with cgps, and everything was fine. Since cgps is a gpsd client, I suggest you are trying to read from a serial port, a running gpsd is using already. If this is the case, stop gpsd and retry. However, reading from gpsd rather than reading from ublox will give you all the advantages of tens of years of experience in handling GPS receievrs correctly.

I think you are on the money.

I reset everything and stopped all scripts on boot, then ran cgps and wrote down a few coords.
this time they are spot on.

I repeated this procedure 4 times to be sure, each time the results are repeatable.

It would seem my issue is with mynmea2
I assume some sort of math issue that's scaling down the seconds.

so I may manually parse the string, or get stuck into the cgps documentation and try to make it do the job.

[bensimmo]

Pynmea2 has been working fine for me for years, with ublox6 and ublox8

The 'shrink' problem is something else, I've seen it but can't remember what it was. It's something like using the wrong style of coordinates.

Google maps should be able to take the one supplied with no problem.

The batter just help make the first fix faster, don't worry about it.


gpsd can be killed with
sudo killall gpsd
Use gpsmon instead, it doesn't need gpsd.

You shouldn't need a sleep statement (your actually only sleeping 0.1seconds, unless that's a typo.
The serial read will limit the loop to at least once a second as that's the speed the data comes at.

I hear threading mentioned, I see no threading in the program.

Look at using https://www.gpsvisualizer.com
For plotting routes easier.

Have fun, but have a rest to.

Move to Python3 and get it working with that

You use Python3 at the command line by typing Python3 and not python

[bensimmo]

I just check and if you stay with Google earth, it can use Decimal Degrees.
So you only need to record
msg.latitude & msg.longitude

(Same as Google maps)
You don't need to split and combine.

https://support.google.com/earth/answer/148068?hl=en


EDIT simple test program for you (another one, this is python3)
EDIT2, added logging to a file for ease.

Code: Select all

#!/usr/bin/python3
import pynmea2  # NMEA GNSS splitter
import serial  # sudo pip3 install pyserial # GPIO/USB serial read
from csv import writer

GNSS_PORT = "/dev/ttyACM0"
#GNSS_PORT = "/dev/serial0"

serialPort = serial.Serial(GNSS_PORT, 9600, timeout=1.5)

with open('gpslog.csv', 'w', newline='') as f:
    data_writer = writer(f)
    data_writer.writerow([ 'latitude', 'longitude', 'altitude' ]) # data headers

    while True:
        strip = serialPort.readline()
        if strip.find(b'GGA') > 0:
            msg = pynmea2.parse(strip.decode('utf-8'))
            print( msg.latitude, msg.longitude, msg.altitude )
            data_writer.writerow([ msg.latitude, msg.longitude, msg.altitude ])