stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Tue Jan 10, 2012 9:32 pm

As part of my solar panel tracking project, and with advice and help from several posters here, I have set up a Debian 2.6.32 install on a VirtualBox drive on my Mac, with a view to getting a bit of a head start.

I think it was JamesH who suggested a Bluenext gps device, so I bought one from http://www.scan.co.uk for about £16.50. There are no Linux drivers supplied so it means using the Synaptic Package Manager to download the following:

1. gpsd, gpsd-clients

2. A simple mapping app eg TangoGPS

3. usb-modeswitch-data, usb-modeswitch (I'm not sure these are absolutely necessary, could try setting up without them first to see if they are really needed).

The install:1. Install: gpsd, gpsd-clients
2. Install a mapping program, e.g. TangoGPS
3. Reboot
3a Assuming you have installed Debian on a virtual drive such as VirtualBox, go to
the lower RH corner of the screen and click on the usb icon. This will bring up
a list of the usb devices connected, and you need to click the one which says
'Silicon Labs CP2104 usb to uart bridge controller' so it is ticked.
4. Open a Terminal Window, type su at the prompt and enter the root password.
5. cd /dev
6. ls - this will list all the folders in the /dev directory
7. Note down the folder of GPS device e.g. gps0
7. type dpkg-reconfigure gpsd
8. Optional: type gpspipe -r - wait a second and the screen should fill with the data strings being sent from the gps dongle.
9. Ctrl+C to Exit back to the prompt.
10. At the prompt, type lsmod
...and you should see a whole load of stuff appear, including the following
line near the bottom:
Bus 001 Device 004: ID 10c4:ea60 Cygnal Integrated
Products, Inc. CP210x Composite Device

If gpspipe -r and/or lsmod don't have any effect go back to step #6 and try a different device.

Typing xgps or cgps will start two small apps that will display the gps data, or open up yourmapping program and try it out.
Theres no need to run this setup again unless you change GPS device. I'm not sure how this process will compare with how it's done on a R Pi, but it looks as thoughwe'll find out soon. (Apologies to all the Linux experts here - I know the above is probably simple for you guys,but I've really struggled to get my head round Debian so far, and have repeatedly come up againstconfusing and badly written so-called 'tutorials' on the net.

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Tue Jan 10, 2012 9:44 pm

A few comments on this process:

- Does the gps0 device node actually appear automagically when you insert the GPS? This is probably somewhat 'lucky' in that the kernel can't tell the difference between a USB->serial converter in some other use and one inside a GPS without some list of device IDs. This might be inconsistent between modules (but if it works, it should continue to work). Perhaps look at the dmesg output to see which serial device is created when you plug in the GPS and use that instead of guessing based on the /dev filesystem.

- In step 10 you give the command 'lsmod' but show output for 'lsusb -v'.

Nothing should be different on the Pi, except that perhaps the default kernel will not include the necessary modules, but it sounds like it will.

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Tue Jan 10, 2012 10:40 pm

With my setup the device didn't appear to start with, and I tried many things (unsuccessfully) to try and view all the ports and see what was connected (and I still haven't a clue how to do that). Step 3a is the solution, and I think it just allows the Debian installation to see the Mac's usb port 'through' the VirtualBox if you see what I mean.

Probably in a normal Linux installation (ie not on a virtual drive) it would all happen automatically, as I think the usbserial module is loaded by the os when the device is plugged in. (I'm not certain that this is the case with Debian, which is why I installed usb-modeswitch-data and usb-modeswitch).

dmesg dishes out an awful lot of stuff to trawl through - I had a quick look through and found some references to usb-serial, but nothing that seemed to tell me it was gps0. But then again it is all largely meaningless to me!

Thanks for correcting me on step 10, yes it is the output of lsusb. I can't seem to find the edit button to correct my first post.

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Tue Jan 10, 2012 11:35 pm

When you attach the device most distros (including Debian) will load the necessary kernel modules, whether it's a virtual or real device doesn't really matter. What I was concerned about was that you are referencing a gps0 device node which probably won't appear for many USB GPSs on the market since they use the same USB->serial chips as are used for other things, so the OS can't assume they are a GPS unless it already knows about them.

However, I just took a look at the gpsd package and it includes some udev rules that assume certain USB->serial chips might be GPSs and create the gpsN links for you, so your way should work, at least on Debian and at least for common GPS modules. Using the generic way will be more reliable though.

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 11:02 am


Using the generic way will be more reliable though.


Unfortunately that option is not for me, as I have no idea how to decipher the output from dmesg (which I think is what you might be referring to).

Anyway, I"ve managed to write the first bit of Python to get some output from the gps (I can"t really claim any credit for this as all I"ve done is copy and paste from various sources until it sort of works):

import gps, os, time

session = gps.gps(host="localhost", port="2947")

session.poll()

session.stream()

while 1:

os.system("clear")

session.poll()

# a = altitude, d = date/time, m=mode,

# o=postion/fix, s=status, y=satellites

print

print "GPS reading"

print "————————————"

print "latitude " , session.fix.latitude

print "longitude " , session.fix.longitude

print "time utc " , session.utc, session.fix.time



print "Satellites (total of", len(session.satellites) , " in view)"

time.sleep(1)

for i in session.satellites:

print "t", i

time.sleep(1)

Executing that gives me:

GPS  Reading

latitude         51.nnnnnnnnn

longitude      -0.04nnn

time utc         1326279176.68

Satellites (total of 12 in view)



Can anyone tell me how to convert the time data to something understandable, and also how to break it down and extract, say, the date (ddmmyyyy) and the time (hhmm)? Words of one syllable or less please.

scubaguy
Posts: 11
Joined: Wed Jan 11, 2012 2:14 pm

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 2:31 pm

GPS time are the seconds since 00:00:00 on 1st January 1970 so the UTC of  1326279176.68 when you executed the query to your GPS unit works out as 10:52:57 on 11/01/2012.

I don't know how to work out the date exactly but there is probably a python script which can do it for you (I just used excel to decode the UTC - divide by 3600 to get hours, divide by 24 to get days and then add to 25569 [excel date code for 01/01/1970 00:00] and display as dd/mm/yyyy hh:mm:ss).  For info, 1st January 2012 UTC is 1325376000

User avatar
zag
Posts: 68
Joined: Tue Jan 03, 2012 12:10 am
Contact: Website

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 3:20 pm

stevep said:



...and have repeatedly come up against confusing and badly written so-called 'tutorials' on the net.
This is awesome work stevep. I've been collecting all the advice from this forum (and others) to write some guides on my forum. I hope you don't mind but I have transferred it all there so people can find it easily.
Unofficial Raspberry Pi Forums - www.raspberrypiforums.com

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 3:56 pm

@scubaguy – thanks. I've found out a fair bit of general info on how the time is handled, but haven't got to grips with the conversion within Python yet. Still looking…

@zag – that's far more than I deserve I think! I see you've corrected the lsmod vs lsusb error near the end. It's also worth noting another one in my Python listing above – if the penultimate line (print "t", i)* is indented the output will include a table of satellites with their prn numbers and position data (due to my ignorance about syntax in Python). I don't know whether you can edit your post to include the indents, but it might help folks like me who are just starting out. Cheers

*edit - just noticed this line needs changing in your second post as the backslash has gone awol!

User avatar
RaTTuS
Posts: 10539
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 4:04 pm

useful info for usb stuff :-

open a terminal and type

cat /var/log/syslog

then plug in your usb device and see what it says
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
walney
Posts: 233
Joined: Wed Nov 30, 2011 6:57 pm
Contact: Website

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 4:18 pm

...and that's how I was going to solve the lack of a realtime clock issue - just use the output from the gps and use it to set the system clock.

Which Bluenext did you use? The BN-903S?

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 5:30 pm

Yes, it's a BN903s.

As for cat /var/log/syslog giving me useful information, I'll take your word on that Rattus! I'm afraid all it does for me is produce a window full of stuff I don't understand.

mental2k
Posts: 21
Joined: Tue Nov 29, 2011 5:04 pm

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 5:46 pm

Maybe slightly off topic, but would using a serial GPS module be surer choice?  Its what I plan to do just to educate myself on the matter but still the UART communicaton should be standard right?

scubaguy
Posts: 11
Joined: Wed Jan 11, 2012 2:14 pm

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 5:51 pm

A quick google for "python gmt from utc" produces:

http://docs.python.org/library/time.html

and

http://docs.python.org/library.....etime.html

where you can read up how to convert easily in python.

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 7:11 pm

stevep said:


Unfortunately that option is not for me, as I have no idea how to decipher the output from dmesg (which I think is what you might be referring to).


I was referring to looking sysfs for USB serial ports and using the serial node directly rather than the system-guess-at gps0 link. I had written a longer post describing this that was lost due to server errors yesterday, so I don't blame you for being confused I don't have a USB->serial dongle with me to check for you, but IIRC the list of nodes is somewhere like /sys/class/usb-serial/devices ; browse around in /sys/class and you should find a list.


Can anyone tell me how to convert the time data to something understandable, and also how to break it down and extract, say, the date (ddmmyyyy) and the time (hhmm)? Words of one syllable or less please.


From my quick read of the gpsd documentation you should be getting an ISO format date out of it (ie. 2012-01-11T01:01:01Z or such), so I think you might be accessing the wrong member and this might be the system time, not the GPS time. The gpsd documentation isn't at all clear though so I'm not sure without poking at the actual data that comes out of a running GPS.

You can convert from POSIX timestamp format like you're getting with the datetime.fromtimestamp() and datetime.utcfromtimestamp() methods:


>>> from datetime import datetime
>>> d=datetime.utcfromtimestamp(float('1326279176.68'))
>>> d.year
2012
>>> d.hour
10


And so on. See http://docs.python.org/library.....etime.html

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 10:30 pm

Thanks for all the help and links from everyone – some of them I had already found this afternoon, so it is nice to see them confirmed as useful sources.

error404 said:


From my quick read of the gpsd documentation you should be getting an ISO format date out of it (ie. 2012-01-11T01:01:01Z or such), so I think you might be accessing the wrong member and this might be the system time, not the GPS time. The gpsd documentation isn"t at all clear though so I"m not sure without poking at the actual data that comes out of a running GPS.


This has been worrying me this afternoon. At the start of my script I did:

import gps, os, time

Later on comes:print "latitude " , session.fix.latitude
print "longitude " , session.fix.longitude
print "time utc " , session.utc, session.fix.timeTo my untutored eye that makes it look as though all the lat, long and time info comesfrom the gps. Certainly the lat and long must do. But because I start by loading the gpsand the time modules, does that mean that the time data is actually being pulled fromthe system rather than the gps? If so, how can I make the script read from the gps? Ihave tried:session = gps.gps(mode=gps.WATCH_NEWSTYLE)followed later by g = gps.gps(mode=gps.WATCH_NEWSTYLE)print "time utc " , g.utc," + ", g.fix.timebut am seriously beyond my limits here! (I'm even having problems formatting these forumposts, as you can see....)
 

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Wed Jan 11, 2012 10:38 pm

Not knowing gpsd's client API, and further the Python bindings for it, I can only tell you what I was able to figure out from the documentation (which is slim, at best).

You're definitely printing data coming from the gps module, Python is well segregated and modules won't conflict with each other in the way you suggest. What I was thinking is that gpsd (or the Python bindings) might *also* be timestamping each GPS report with the *system* time in addition to the *GPS* time and that you're reading the wrong one. Could easily be wrong, just something you should verify.

Should be easy enough to verify this, just make sure the system clock is set wrong

This also doesn't require much initialization, so you can run it interactively and dump all the members of the GPS report and examine them manually to decide what's interesting. Start 'python' on a command line and try this:


import gps, pprint

session = gps.gps(host="localhost", port="2947")

session.poll()

pprint.pprint(session)


stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Thu Jan 12, 2012 4:42 pm

Continuing to struggle with the time.

Quote - Should be easy enough to verify this, just make sure the system clock is set wrong

That is so simple I wish I had thought of it! So I did that, and wrote the following to check the output:

print "latitude " , session.fix.latitude

print "longitude " , session.fix.longitude

print "time utc " , session.utc, session.fix.time

print 'time ticks' , time.time()

time.sleep(1)

for i in session.satellites:

print "t", i

time.sleep(1)

My thinking is that 'time utc' will give the data from the gps, and 'time ticks' will return the system time.

The output shows:

latitude 51.1nnnnnn

longitutde -0.04nnn

time utc 1326382683 (=15:38, compared with 15:51 actual time)

time ticks 1326392882 (=18:28, compared with 18:27 Debian time)

Problems:

1. The Debian window initially shows the same time as the Mac on which it is running, but after a while begins to lag. At the moment after running for the last 7 hours it fallen behind by 17 minutes, compared with the Mac's system clock.

2. What should <print "time utc " , session.utc, session.fix.time> return? I would have thought this would return 2 values but only one appears when the script is run. I am still doubtful that this code actually reads the gps time correctly, and if it does I'm struggling to find a way to use the value it returns (due to my programming ignorance).

3. The value returned above is obviously different to the value returned by the <print 'time ticks' , time.time()> line, but I'd have thought that they would increment in step with each other. They don't, and I wonder if that is something to do with the time.sleep(1) lines being either wrong or in the wrong place.

 

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Fri Jan 13, 2012 12:24 am

stevep said:


1. The Debian window initially shows the same time as the Mac on which it is running, but after a while begins to lag. At the moment after running for the last 7 hours it fallen behind by 17 minutes, compared with the Mac"s system clock.


This is common (and almost unavoidable without an RTOS) in virtual machines. Either install the VM tools package in the Debian guest that will make some magic communication happen to fix this or don't worry about it. You won't have this problem on real hardware.


2. What should <print "time utc " , session.utc, session.fix.time> return? I would have thought this would return 2 values but only one appears when the script is run. I am still doubtful that this code actually reads the gps time correctly, and if it does I"m struggling to find a way to use the value it returns (due to my programming ignorance).


I really have no idea, since the documentation isn't clear. I would dump the members that exist and their values (the pprint example above) and guess based on their names and values to find out. Based on what the docs do say (that the Python result object will contain the same values as the JSON output of the gpsd server), and some example JSON output, it seems like the value you're looking for should be called 'time', so session.fix.time is probably the correct one to use. No idea what the 'utc' value is about, but the fact that it's associated with the 'session' and not the 'fix' would imply to me that it's not related to the GPS data itself.


3. The value returned above is obviously different to the value returned by the <print "time ticks" , time.time()> line, but I"d have thought that they would increment in step with each other. They don"t, and I wonder if that is something to do with the time.sleep(1) lines being either wrong or in the wrong place.


Do you mean that one or the other of the timestamps isn't changing at all, or that they don't match? They won't be in sync, if that's what you mean. I think the fact that you're starting 'stream' mode and then using 'poll' might be mucking around with things as well.

I really know absolutely nothing about gpsd and its API so I'm just guessing based on what little I can see from here. By coincidence I'm actually working on a heavily GPS-based project right now, but it's embedded so I can't easily hook a GPS up to my computer to experiment for you...but really what are you trying to accomplish? GPSD is already doing almost all of the work for you, and there's lots of existing software out there to use with it.

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Fri Jan 13, 2012 12:53 am

OK I didn't have anything better to do at the moment so I wired up gpsfake and tested it out for you. .utc always seems to be null on my machine, I don't know what it's for. gps.fix.time is the current GPS time.

One caveat I don't think you are dealing with is that even in 'stream' mode the client only seems to parse one update per poll() call, so unless you somehow synchronize your client with the GPS, you're going to fall behind. poll() blocks though, so you don't need to sleep. Try something like this:


import gps

from datetime import datetime

session = gps.gps()

session.stream()

while session.poll():

print datetime.utcfromtimestamp(float(session.fix.time))


Now you have a native Python data structure with the time. Not knowing what you're actually trying to accomplish that's about as much as I can help.

stevep
Posts: 71
Joined: Sat Dec 24, 2011 10:12 am

Re: Setting up a usb gps dongle

Fri Jan 13, 2012 8:25 pm

Ok I think I've reached a dead end until I get a bit of experience with Python, but here's an overview of what I want to achieve:

What I'm aiming to achieve is to extract the latitude, longitude and time data from the gps signal, maybe every 15 or 30 minutes.

I've managed the lat and long with moderate success, and I know the data must be coming from the gps as there is no other possible source.

I can get a value for the time (eg 1326362275.58, or secs from the epoch) but I am not convinced it is the gps time, or if it is, that my script is returning an accurate up to date value.

I cannot convert the 1326362275.58 to give me current date and time figures. All the Googling I've done seems to confuse me with functions that deal with system time. I have managed to write bits of script that will convert the system time, but I can't seem to write anything that extracts the gps time and then converts it.

When (or if) I manage to do this conversion I'll then use the data obtained to work out the altitude and azimuth of the sun. This data will then be used (with an interface board) to drive 2 stepper motors that will position a solar collector normal to the incoming solar radiation (using the maths from here)

As I mentioned in a previous post, I did this a long time ago using BASIC and a BBC Master (and it worked!).

Anyway, thanks for all the help so far - I've saved the thread for future reference. I'm now going to settle down with the Dummies Guide to Python to try and get up to speed!

error404
Posts: 351
Joined: Wed Dec 21, 2011 11:49 pm

Re: Setting up a usb gps dongle

Fri Jan 13, 2012 8:34 pm

stevep said:

I can get a value for the time (eg 1326362275.58, or secs from the epoch) but I am not convinced it is the gps time, or if it is, that my script is returning an accurate up to date value.
I cannot convert the 1326362275.58 to give me current date and time figures. All the Googling I've done seems to confuse me with functions that deal with system time. I have managed to write bits of script that will convert the system time, but I can't seem to write anything that extracts the gps time and then converts it.


I just confirmed this for you and gave you the code to convert it to a Python data structure.

Good luck with learning Python!

DanKegel
Posts: 8
Joined: Wed Feb 22, 2012 11:10 pm
Contact: Website

Re: Setting up a usb gps dongle

Sat Apr 21, 2012 5:33 am

I think the way to go is to let ntpd talk to gpsd, and ask the system for time.

Then to get position, run gpspipe -w and parse its output.

Here's the incredibly crappy way I did that in a mix of C and shell: FILE *fgps;
fgps = popen("gpspipe -w -n 5 | \
tr , '12' | \
awk '/lat/ {lat = $1} /lon/ {lon=$1} END { print lat, lon }'|\
sed 's/\"...\"://g'", "r");
if (!fgps) {
fprintf(stderr, "can't open gpspipe\n");
return 0;
}
if (2 != fscanf(fgps, "%s %s", lat, lon)) {
fprintf(stderr, "can't parse lat lon\n");
return 0;
}

DanKegel
Posts: 8
Joined: Wed Feb 22, 2012 11:10 pm
Contact: Website

Re: Setting up a usb gps dongle

Sat Apr 21, 2012 5:36 am

The gpsd that comes with the debian image, 2.95, is very old and doesn't support my dongle.  Ubuntu 12.04 comes with version 3.4 and works with my dongle.  I guess I'll try building that... or maybe I'll try the Fedora image, it may be more up to date.

DanKegel
Posts: 8
Joined: Wed Feb 22, 2012 11:10 pm
Contact: Website

Re: Setting up a usb gps dongle

Sat Apr 21, 2012 5:40 am

mental2k said:


Maybe slightly off topic, but would using a serial GPS module be surer choice?  Its what I plan to do just to educate myself on the matter but still the UART communicaton should be standard right?


USB works fine.  The only problem with it is the time is not very precise.  To get precise time, you need the PPS signal, which USB doesn't support at all; it is carried e.g. on one of the handshake lines of a real serial port.  It can also be listened for on a GPIO pin.   I haven't tried that yet, since I don't need really precise time yet.

rjcollingridge
Posts: 10
Joined: Wed May 23, 2012 8:15 am

Re: Setting up a usb gps dongle

Wed May 23, 2012 9:00 am

Is anyone else having issues getting the BlueNEXT BN-903S working? I've bought one and it works fine under Windows. Having tried it on my Mini-ITX running Debian and a Rpi running Debian and Arch Linux, in all cases the device appears OK but, I'm simply not getting any data out of it. GPDS doesn't see any data from it. I've checked permissions issues and I'm now stuck.

Return to “Other projects”