Delay to update clock at startup even with RTC

[sebnor31]

I am using a GPS module (Adafruit Ultimate GPS Hat) with built-in real time clock (RTC) connected to a RPi 3 (model B, running Raspbian Jessie).

A python program is automatically executed at each startup and requires the time (from the clock) to be accurate in order to perform properly.

The GPS module is properly connected, uses a backup battery to keep its fix and provides correct time.
However, it takes about 1 min after the RPi has initialized (i.e., desktop GUI is displayed) for the clock to update to current time.

The problem is my program has to start as soon as possible (when the boot up procedure is done).

Is there anyway to reduce the delay that it takes for the clock to update to current values (generated by the GPS on-board RTC) ?

[SonOfAMotherlessGoat]

Just a few questions:

Link to the GPS Hat you purchased? And a link to any instructions you followed to connect the GPS and the RTC.

How is the Python program invoked? Is it a systemD service or tacked on to the end of rc.local? Are you sure that the Pi is pulling the updated time from the onboard RTC or is it using NTP to update the time and the RTC isn't being used at all. I do know from having the non-hat version of the Ultimate GPS that the onboard battery does store the ephemeral data from the GPS constellation, but I haven't used the Hat with the onboard RTC.

If you can help us out with a little more detail and possibly some examples of the code being used we can help troubleshoot your problem. (Also, the AdaFruit forums are a great resource, have you checked out that area and look to see if there are others experiencing the same issues?)

[alphanumeric]

As far as I know, the Pi will not get the time from the GPS module unless you tell it too? I have this one, https://www.adafruit.com/products/746 My Pi will not get the time from it. The onboard RTC is only used by the GPS module itself. You have to tell the PI to read from it. Did you do that? Otherwise the PI will just sync the time from the Internet which has a short delay. What you likely need is one of these, https://www.adafruit.com/products/3013

[sebnor31]

Link to the GPS Hat you purchased? And a link to any instructions you followed to connect the GPS and the RTC.

The link to the GPS hat is : https://learn.adafruit.com/adafruit-ult ... i/overview
The instruction to connect the GPS as a RTC is: https://frillip.com/raspberry-pi-stratum-1-ntp-server/

Are you sure that the Pi is pulling the updated time from the onboard RTC or is it using NTP to update the time and the RTC isn't being used at all. I do know from having the non-hat version of the Ultimate GPS that the onboard battery does store the ephemeral data from the GPS constellation, but I haven't used the Hat with the onboard RTC.

I am sure that the time is pulled from the onboard RTC of the GPS module because I disabled any internet communication (including WiFi) and the time is updated properly after the mentioned delay. The HAT version should work in the same fashion than the non-hat as the GPS chip is the same.

How is the Python program invoked? Is it a systemD service or tacked on to the end of rc.local?

I am new to linux and raspberry pi world, so please accept my apologies if I cannot use the proper terms. My python program is invoked at boot up by modifying the crontab as follow:

Code: Select all

sudo crontab -e

And add the following line:

Code: Select all

@reboot sh /home/pi/..../data_capture_launcher.sh >/home/pi/logs/cronlog 2>&1

The data_capture_launcher.sh is an executable file:

Code: Select all

#!/bin/sh
# data_cature_launcher.sh

cd /
cd home/pi/Github/RPi_Camera_Reader
sudo python mainDataCapture.py
cd /

If you can help us out with a little more detail and possibly some examples of the code being used we can help troubleshoot your problem. (Also, the AdaFruit forums are a great resource, have you checked out that area and look to see if there are others experiencing the same issues?)

I checked the AdaFruit and your forum and found other posts about this clock update delay problem but, to the best of my knowledge, could not find any information that explains the reason why it is occurring and possibly how to reduce it.
My application targets autonomous driving where the RPi is powered-on when the car starts. Data (from camera, GPS, etc.) must be saved locally as soon as possible in files whose names include the time of creation, hence getting it from the OS clock by using the "datetime.now()" function of the python "datetime" module. Because the clock retains a previous value, I noticed that the last created data files are either overwritten or data are appended to them which is not the behavior expected as we want new files to be created with the current time stamp (when the car starts).
This would not be a problem if we can wait for the clock to update, but for our application, waiting about 1min 20 sec to start collected data is too long. That is why I would like to know if there is a way to reduce the time update the clock.
Let me know if you need more information or a better explanation, and some code.

[sebnor31]

As far as I know, the Pi will not get the time from the GPS module unless you tell it too? I have this one, https://www.adafruit.com/products/746 My Pi will not get the time from it. The onboard RTC is only used by the GPS module itself. You have to tell the PI to read from it. Did you do that? Otherwise the PI will just sync the time from the Internet which has a short delay. What you likely need is one of these, https://www.adafruit.com/products/3013

As I mentioned to my previous post, the Pi reads from the GPS module as all internet connections were disabled and the time was properly updated. I am using the Ultimate GPS Pi Hat which is based on the GPS chip in your first link https://www.adafruit.com/products/746 .
However, you are bringing an interesting suggestion with the product in https://www.adafruit.com/products/3013. Is using a dedicated RTC module would reduce the delay for Raspbian to update its clock or this has nothing to do with what type of RTC source is used but comes from the inner working of the OS?

[alphanumeric]

Thank you for posting that info. I am going to try this out for myself. I have been wanting to try it but kept putting it off. It might take me a while though. I don't have the GPS module setup at the moment.

[sebnor31]

Thank you for posting that info. I am going to try this out for myself. I have been wanting to try it but kept putting it off. It might take me a while though. I don't have the GPS module setup at the moment.

That would be wonderful. Thank you for your help. I think it is a very important feature to figure out for the people that want to use the Pi in applications where no internet connectivity is available and requires a real-time data capture as accurate time-stamping is of prime importance.

[alphanumeric]

I tried to do it in the past but there was no easy to follow guide, not for the RTC part anyway. Reading the GPS data I've done before.

[darkbibble]

I don't know if this is relevant, but when raspbian is booting there is a bit about the clock being setup in the wall of text flying past, wouldn't it be an idea to find this initial clock setup and change that??
as whats happening is the pi is setting up software clock during boot, then you are changing it to hardware clock once it finally finished booting raspbian

[bensimmo]

I have no idea of the board used in question, but I would check that it's not getting time from the GPS signals and so waiting until it has enough data/lock from that.

[alphanumeric]

The one I have will sync its time with the GPS satellite time. Once its done that though the onboard RTC will keep time with the power off as long as the battery is good. Just like your PC does. Once you power back up, it resyncs the time again. If the RTC is accurate you shouldn't be more than a couple of seconds out.

[alphanumeric]

cgps -s is timing out on me with no fix. The status LED on the GPS module says it has a fix though. I had this issue last time they updated Jessie. Looks like they broke it again. Adafruit really needs to get on top of updating their tutorials. In the past I could run sudo cat /dev/ttyAMA0 and get data, now that doesn't even work. It doesn't even open/launch. I guess I'll have to reimage my card and start over.

[alphanumeric]

I didn't forget about this. I started over from the link provided above. It will likely be tomorrow some time before I get it all done. My back is acting up so I have to lay low for tonight. I will post back with my results when done though.

[sebnor31]

I have no idea of the board used in question, but I would check that it's not getting time from the GPS signals and so waiting until it has enough data/lock from that.

The GPS has a lock at boot up and provides correct time at the proper sampling frequency (10 Hz).

[sebnor31]

cgps -s is timing out on me with no fix. The status LED on the GPS module says it has a fix though. I had this issue last time they updated Jessie. Looks like they broke it again. Adafruit really needs to get on top of updating their tutorials. In the past I could run sudo cat /dev/ttyAMA0 and get data, now that doesn't even work. It doesn't even open/launch. I guess I'll have to reimage my card and start over.

I am also having a similar issue with my backup Pi 3 but not my other board. cgps -s times out but I can read raw data with sudo cat /dev/serial0 though. This is another issue that may require its own post.

[alphanumeric]

Have a look at this thread, viewtopic.php?f=28&t=129826&p=899258&hilit=cgps+s
the DEVICES="/dev/ttyAMA0" edit fixed it for me, in the past. What you edit it too depends on whether your using USB or SPI. And the UART used has changed on the Pi 3. I've put that on the back burner for now. I'm going to try and have another go at the RTC aspect tonight. back pain permitting.

[alphanumeric]

Doesn't work at all for me? My time won't sync even with an internet connection.
picocom -b 9600 -f n /dev/ttyS0 didn't do anything when I ran it?
sudo ppstest /dev/pps0 gave the correct output as far as I can tell though.

For Pi2/3 he said use make -j5 so I did a

make -j5
sudo service ntp stop
sudo make -j5 install

is that correct?

sudo rm /var/lib/ntp/ntp.conf.dhcp gave a no such file or directory error message, but I think that might be normal?

ntpq -pn didn't get me the o127.127.20.0 .GPS.
mine had no o and said local or something instead of GPS?

[sebnor31]

Doesn't work at all for me? My time won't sync even with an internet connection.
picocom -b 9600 -f n /dev/ttyS0 didn't do anything when I ran it?
sudo ppstest /dev/pps0 gave the correct output as far as I can tell though.

For Pi2/3 he said use make -j5 so I did a

make -j5
sudo service ntp stop
sudo make -j5 install

is that correct?

sudo rm /var/lib/ntp/ntp.conf.dhcp gave a no such file or directory error message, but I think that might be normal?

ntpq -pn didn't get me the o127.127.20.0 .GPS.
mine had no o and said local or something instead of GPS?

Follow closely the tutorial at https://frillip.com/raspberry-pi-stratum-1-ntp-server/.
The RTC feature of the GPS is working properly.
However, my Pi 3 cannot communicate with the GPS module to get data. cgps -s times out.

[alphanumeric]

Doesn't work at all for me? My time won't sync even with an internet connection.
picocom -b 9600 -f n /dev/ttyS0 didn't do anything when I ran it?
sudo ppstest /dev/pps0 gave the correct output as far as I can tell though.

For Pi2/3 he said use make -j5 so I did a

make -j5
sudo service ntp stop
sudo make -j5 install

is that correct?

sudo rm /var/lib/ntp/ntp.conf.dhcp gave a no such file or directory error message, but I think that might be normal?

ntpq -pn didn't get me the o127.127.20.0 .GPS.
mine had no o and said local or something instead of GPS?

Follow closely the tutorial at https://frillip.com/raspberry-pi-stratum-1-ntp-server/.
The RTC feature of the GPS is working properly.
However, my Pi 3 cannot communicate with the GPS module to get data. cgps -s times out.

That's the tutorial I followed? The examples I posted above are from that tutorial.

[alphanumeric]

I've abandoned this for now. I just don't feel like going through that whole tutorial all over again.

Does the status LED on your GPS breakout indicate it has a fix when you boot up the Pi? Maybe it won't sync the time until it has a valid fix?

[bensimmo]

As a quick test to check it is not the new PI3's Bluetooth and the lesser able serial port UART being used messing it up.


Put a line in /boot/config.txt. With. dtoverlay=pi3-miniuart-bt
This gives the serial port a 'none messed up'* serial connection like the Pi2 and before.
Change ttyS0 back ttyAMA0 in everything, reboot and try again.


*Name I use as it messed everything up for me

[alphanumeric]

I've already moved on and reimaged my card with a stock Jessie image. I had done the UART fix mentioned at the start. Some of the tutorial worked and some stuff didn't. I may try it again at some point. Just not up to going through the whole thing all over again at the moment.