BoatRacer
Posts: 1
Joined: Sat Aug 11, 2018 2:58 am

Fancy GPS overlay on video

Sat Aug 11, 2018 3:17 am

I have read the few overlay threads, but I think my question has more to do with how to design them vs how to get data from the Pi and overlaying it on video. I realize this might not be the best section on the forum, hoping someone has experience here or can point me in the right direction.

So, off with the question and the background:

I race hydroplanes in APBA sanctioned racing events. These are small boats, about 10 feet long in the class I race in and we sit on our knees while racing. Of course, there are a number of GPS, video and telemetry based options available on the market made specifically for capturing data from the engine, GPS and video, etc. They are also expensive (usually $400+ for decent ones). In many cases, they aren't waterproof either, which is a problem.

I want one just to capture just video and GPS (I don't need sensor data like head temp, engine RPM, etc...at least not now). GoPros do it, but are also pricey, so I thought, hey, build one with my Pi, make it waterproof, etc. But it's important to get overlays for the video too. Many of the overlays you'll see look like speedometers, and also a small tracking overlay that shows the track of the boat as it goes around the course.

I guess my question is, does anyone know how to create those kinds of overlays and tie it in with the GPS data from the Pi?

RichInTheUSA
Posts: 29
Joined: Fri Nov 04, 2016 1:27 am

Re: Fancy GPS overlay on video

Sun Aug 12, 2018 5:26 pm

Hi, I don't want to discourage you from using a Pi to build this.... however, I have an Olympus TG-Tracker video camera that does what you are looking for and more. Specifically, it can overlay lat, long, temp, pressure, and other sensory data over the video (in several formats). Oh, and it's water proof to 30 meters. It's cheaper than a go -pro too. Hope this helps.

gordon77
Posts: 4973
Joined: Sun Aug 05, 2012 3:12 pm

Re: Fancy GPS overlay on video

Tue Aug 14, 2018 1:48 pm

To get you started...

This uses picamera to record video with GPS data on it.. GPS is on USB.

Code: Select all

#!/usr/bin/env python3
import picamera
import datetime as dt
import os
import RPi.GPIO as GPIO
import serial
import sys

#shutdown button gpio 
SD = 27 # pin 13 (Set SD = 21 for PiB ver1)
#software stop button gpio 
SS = 7  # pin 26
#running LED gpio 
LD = 25 # pin 22
#low supply volts gpio (don't change)
LV = 35

GPIO.setwarnings(False)
GPIO.setmode (GPIO.BCM)
# set input for shutdown button
GPIO.setup(SD,GPIO.IN,pull_up_down = GPIO.PUD_UP)
# set input for software stop button
GPIO.setup(SS,GPIO.IN,pull_up_down = GPIO.PUD_UP)
# running LED (flashes if receiving GPS)
GPIO.setup(LD,GPIO.OUT)
GPIO.output(LD,GPIO.HIGH)
#Low supply volts
GPIO.setup(LV,GPIO.IN,pull_up_down = GPIO.PUD_UP)

#set file record tme in seconds
record_time = 300

startfile = 100
maxfiles = 30
filestamp = startfile
savedir = "/home/pi/Videos/" 

#set settime = 1 to allow Pi time to be updated by gps
settime = 0
#set your time offset, UTC = 0, used by settime
offset = 0

# set add_gps_data = 1 to include 3D fix and number of satellites
add_gps_data = 1

#write filestamp file if first time
if os.path.exists('last.txt') == False:
    file = open("last.txt", "w")
    file.write(str(filestamp))
    file.close()
#read last filestamp file stored
file = open("last.txt", "r")
filestamp = int(file.read()) + 1
if filestamp > startfile + maxfiles:
    filestamp = startfile
file.close()

gps_con = 0

# check for gps on USB
if os.path.exists('/dev/ttyUSB0') == True:
   ser = serial.Serial('/dev/ttyUSB0',4800,timeout = 10)
   gps_con = 1

if os.path.exists('/dev/ttyUSB1') == True and gps_con == 0:
   ser = serial.Serial('/dev/ttyUSB1',4800,timeout = 10)
   gps_con = 1

with picamera.PiCamera() as camera:
    camera.resolution = (1280, 720)
    camera.framerate = 20
    camera.start_preview()
    #enable next 2 lines if picture inverted
    camera.vflip = True
    camera.hflip = True
    camera.annotate_background = picamera.Color('black')
    camera.annotate_text = ""
    text0 = ""
    text1 = ""
    text2 = ""
    text4 = ""

    fix = '0'
    LVS = 0
    
    while GPIO.input(SS) == 1:
        start = dt.datetime.now()
        if filestamp > startfile + maxfiles:
           filestamp = startfile
        camera.start_recording(savedir + str(filestamp) + '.h264')
        # write filestamp file
        file = open("last.txt", "w")
        file.write(str(filestamp))
        file.close()
        GPIO.output(LD,GPIO.HIGH)
        while (dt.datetime.now() - start).seconds < record_time and GPIO.input(SS) == 1 :
           # check for gps connected
            if os.path.exists('/dev/ttyUSB0') == True or os.path.exists('/dev/ttyUSB1') == True:
                if gps_con != 1 and os.path.exists('/dev/ttyUSB0') == True:
                    ser = serial.Serial('/dev/ttyUSB0',4800,timeout = 10)
                    gps_con = 1
                elif gps_con != 1 and os.path.exists('/dev/ttyUSB1') == True:
                    ser = serial.Serial('/dev/ttyUSB1',4800,timeout = 10)
                    gps_con = 1
                # get data from gps
                gps = ser.readline()
                #print (gps)
                if sys.version_info[0] == 3:
                   gps = gps.decode("utf-8","ignore")
                if gps[0 : 1] != "$":
                   ser.flushInput()
                
                if gps[1 : 6] == "GPGGA":
                    fix = '0'
                    gps1 = gps.split(',',12)
                    if len(gps) > 68 and (gps1[3] == "N" or gps1[3] == "S"):
                       fix = gps1[6]
                       text0 = " " + gps1[7] + " " + gps1[9] + "M"
                    
                if gps[1 : 6] == "GPGSA":
                    gps4 = gps.split(',',12)
                    if len(gps) > 50:
                       text4 = " " + gps4[2] + "D"
                       
                if gps[1 : 6] == "GPRMC" and fix != '0':
                    gps2 = gps.split(',',12)
                    if len(gps) > 60 and (gps2[4] == "N" or gps2[4] == "S"):
                        hour = int(gps2[1][0:2])
                        hour = hour + offset
                        if hour > 23:
                           hour = hour - 24
                        if hour < 0:
                           hour = hour + 24
                        minute = int(gps2[1][2:4])
                        second = int(gps2[1][4:6])
                        second = second + 1
                        if second == 60:
                           second = 0
                           minute = minute + 1
                           if minute == 60:
                              minute = 0
                              hour = hour + 1
                              if hour > 23:
                                 hour = 0
                        timex =""
                        if hour < 10:
                           timex = "0"
                        if second > 9 and minute < 10:
                           timex = timex + str(hour) + ":0" + str(minute)+ ":" + str(second)
                        elif second < 10 and minute > 9:
                           timex = timex + str(hour) + ":" + str(minute)+ ":0" + str(second)
                        elif second < 10 and minute <10:
                           timex = timex + str(hour) + ":0" + str(minute)+ ":0" + str(second)
                        else:
                           timex = timex + str(hour) + ":" + str(minute)+ ":" + str(second)
                        datex = "20" + gps2[9][4:6] + "-" + gps2[9][2:4] + "-" + gps2[9][0:2]
                        text1 = datex + " " + timex + " " + gps2[3] + " " + gps2[4] + " " + gps2[5] + " " + gps2[6]
                        if settime == 1:
                           path = "sudo date +%T -s '" + timex + "'"
                           os.system (path)
                           settime = 0
                        
                if gps[1 : 6] == "GPVTG" and fix != '0':
                    gps3 = gps.split(',',8)
                    if len(gps) > 26 and gps3[2] == "T":
                        text2 = " " + gps3[7] + " " + gps3[1]
                        
                if fix != '0':
                    GPIO.output(LD,GPIO.LOW)
                    if add_gps_data == 1 and LVS > 0:
                        camera.annotate_text = str(LVS) + " " +text1 + text4 + text0 + text2 #
                    if add_gps_data == 1 and LVS == 0:
                        camera.annotate_text = "  " +text1 + text4 + text0 + text2 #
                    else:
                        camera.annotate_text = text1 + " " + gps1[9] + "M" + text2
                        
                else:
                    camera.annotate_text = dt.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
            else:
                camera.annotate_text = dt.datetime.now().strftime('%Y-%m-%d %H:%M:%S')
  
            camera.wait_recording(0.04)
            GPIO.output(LD,GPIO.HIGH)
            
            # shut down if low volts (LV) or pin SD to gnd
            if GPIO.input(LV) == 0:
                LVS +=1

            else:
                LVS = 0
            if GPIO.input(SD) == 0 or LVS > 5:
                GPIO.output(LD,GPIO.LOW)
                path = 'sudo shutdown -h now '
                os.system (path)
        camera.stop_recording()
        GPIO.output(LD,GPIO.LOW)
        filestamp +=1
        # stop program if pin SS to gnd
        if GPIO.input(SS)== 0:
            GPIO.output(LD,GPIO.LOW)
            break
 
This uses OpenCV and the Pi Camera, and only works with Python (NOT Python3). GPS is on USB.
Install OpenCV with 'sudo apt-get install python-opencv', also run 'sudo modprobe bcm2835-v4l2' first.

Code: Select all

#!/usr/bin/env python
import cv2 
import math
import datetime
import serial
import os, sys
from decimal import *
getcontext().prec = 8

gps_con = 0

# check for gps on USB
if os.path.exists('/dev/ttyUSB0') == True:
   ser = serial.Serial('/dev/ttyUSB0',4800,timeout = 10)
   gps_con = 1

if os.path.exists('/dev/ttyUSB1') == True and gps_con == 0:
   ser = serial.Serial('/dev/ttyUSB1',4800,timeout = 10)
   gps_con = 1
   
# video input from webcam
frame_width = 640
frame_height = 480
video_capture = cv2.VideoCapture(0)
out = cv2.VideoWriter('outpy.avi',cv2.cv.CV_FOURCC('M','J','P','G'), 10, (frame_width,frame_height))

h_clk = 60
v_clk = 380
h_spd = 480
v_spd = 456
text1 =""
text2 = ""
fix = '0'
speed = 0
l_spd = 0

while True:
    # Capture video feed
    ret, frame = video_capture.read()
    gps = ser.readline()
    #print gps
    if gps[0 : 1] != "$":
       ser.flushInput()
                
    if gps[1 : 6] == "GPGGA":
       fix = '0'
       gps1 = gps.split(',',12)
       if len(gps) > 68 and (gps1[3] == "N" or gps1[3] == "S"):
          fix = gps1[6]
    elif gps[1 : 6] == "GPRMC" and fix != '0':
       gps2 = gps.split(',',12)
       if len(gps) > 60 and (gps2[4] == "N" or gps2[4] == "S"):
          timestamp = (gps2[1])[:2] + ":" + (gps2[1])[2:4] + ":" + (gps2[1])[4:6]
          
          lon = int(gps2[3][0:2]) + Decimal(int(gps2[3][2:4]))/Decimal(60) + Decimal(int(gps2[3][5:9]))/Decimal(360000)
          lat = int(gps2[5][0:3]) + Decimal(int(gps2[5][3:5]))/Decimal(60) + Decimal(int(gps2[5][6:10]))/Decimal(360000)
          text1 = timestamp + " " + str(lon) + " " + gps2[4] + " " + str(lat) + " " + gps2[6] 
    elif gps[1 : 6] == "GPVTG" and fix != '0':
       gps3 = gps.split(',',8)
       if len(gps) > 26 and gps3[2] == "T":
          text2 = "    " + gps3[7]
          speed = float((gps3[7]))
          l_spd = len(str(int(speed)))
          text2 = str(int(speed)) + " " + "   " + gps3[1]

    hrx = h_spd + int(49 * math.sin((speed/40)-2))
    hry = v_spd - int(49 * math.cos((speed/40)-2))
    cv2.line(frame,(h_spd,v_spd),(hrx,hry),(200,200,200),2,4)
    cv2.circle(frame,(h_spd,v_spd),52,(28,28,28),1,8)
    cv2.circle(frame,(h_spd,v_spd),32,(28,28,28),-1,4)
    
    if fix == '0':
       cv2.putText(frame,text1,(h_clk-47,v_clk+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,0,255),1)
    elif fix == '1':
       cv2.putText(frame,text1,(h_clk-47,v_clk+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,255),1)
       cv2.putText(frame,text2,(h_spd-(5 * (int(l_spd*1.4))),v_spd+10),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,255),1)
    elif fix == '2' or fix == '3':
       cv2.putText(frame,text,(h_clk-47,v_clk+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,0),1)
    
    # write ouput video file
    out.write(frame)
    # Display the resulting frame
    cv2.imshow('Video', frame)

    if cv2.waitKey(1) and 0xFF == ord('q'):
        break
 
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
Last edited by gordon77 on Tue Aug 14, 2018 3:40 pm, edited 1 time in total.

PiGraham
Posts: 3926
Joined: Fri Jun 07, 2013 12:37 pm
Location: Waterlooville

Re: Fancy GPS overlay on video

Tue Aug 14, 2018 2:07 pm

BoatRacer wrote:
Sat Aug 11, 2018 3:17 am
Many of the overlays you'll see look like speedometers, and also a small tracking overlay that shows the track of the boat as it goes around the course.
Can you draw an example graphic?
Is it like this example for Sony action cam?
https://www.youtube.com/watch?v=wbPyKHwFuTI

Looks like what you need to do is just log GPS data while recording video than match them up and overlay in post-processing.

gordon77
Posts: 4973
Joined: Sun Aug 05, 2012 3:12 pm

Re: Fancy GPS overlay on video

Thu Aug 16, 2018 1:31 pm

I've been experimenting with this and here's the latest using opencv.

It will display time, gps position, speed and direction and plot upto the last 100 gps positions, currently set to store every 10 seconds. Very beta !
The video maybe a bit jerky !

Code: Select all

#!/usr/bin/env python
import cv2 
import math
import datetime
import serial
import os, sys
import subprocess
import time
from decimal import *
getcontext().prec = 8

lonfiles = []
latfiles = []

frame_width  = 640
frame_height = 480
points = 100
graph_box = 100

if os.path.exists('/dev/video0') == False:
   rpistr = "sudo modprobe bcm2835-v4l2"
   p=subprocess.Popen(rpistr,shell=True, preexec_fn=os.setsid)
   time.sleep(5)
# check for gps on USB
gps_con = 0
if os.path.exists('/dev/ttyUSB0') == True:
   ser = serial.Serial('/dev/ttyUSB0',4800,timeout = 10)
   gps_con = 1

if os.path.exists('/dev/ttyUSB1') == True and gps_con == 0:
   ser = serial.Serial('/dev/ttyUSB1',4800,timeout = 10)
   gps_con = 1
   
# video input from webcam
video_capture = cv2.VideoCapture(0)
video_capture.set(3,frame_width)
video_capture.set(4,frame_height)
out = cv2.VideoWriter('output.mjpg',cv2.cv.CV_FOURCC('M','J','P','G'), 10, (frame_width,frame_height))
h_txt = 60
v_txt = frame_height - 100
h_spd = frame_width - 140
v_spd = frame_height - 24
text1 = ""
text2 = ""
fix   = '0'
speed = 0
l_spd = 0
gps_count = 0
gps = ""
sf = 50

while True:
    # Capture video feed
    ret, frame = video_capture.read()
    if gps_con == 1:
       gps = ser.readline()
       if gps[0 : 1] != "$":
          ser.flushInput()
                
    if gps[1 : 6] == "GPGGA":
       fix = '0'
       gps1 = gps.split(',',12)
       if len(gps) > 68 and (gps1[3] == "N" or gps1[3] == "S"):
          fix = gps1[6]
    elif gps[1 : 6] == "GPRMC" and fix != '0':
       gps2 = gps.split(',',12)
       if len(gps) > 60 and (gps2[4] == "N" or gps2[4] == "S"):
          timestamp = (gps2[1])[:2] + ":" + (gps2[1])[2:4] + ":" + (gps2[1])[4:6]
          lon = int(gps2[3][0:2]) + Decimal(int(gps2[3][2:4]))/Decimal(60) + Decimal(int(gps2[3][5:9]))/Decimal(360000)
          lat = int(gps2[5][0:3]) + Decimal(int(gps2[5][3:5]))/Decimal(60) + Decimal(int(gps2[5][6:10]))/Decimal(360000)
          if gps_count == 0 or ((lon != lonfiles[0] or lat != latfiles[0]) and timestamp[7:8] == "0"):
             lonfiles.insert(0,lon)
             latfiles.insert(0,lat)
             if gps_count >= points:
                del lonfiles[points] 
                del latfiles[points]
                gps_count = points
             else:
                gps_count +=1

          text1 = timestamp + " " + str(lon) + " " + gps2[4] + " " + str(lat) + " " + gps2[6] 
    elif gps[1 : 6] == "GPVTG" and fix != '0':
       gps3 = gps.split(',',8)
       if len(gps) > 26 and gps3[2] == "T":
          text2 = "    " + gps3[7]
          speed = float((gps3[7]))
          l_spd = len(str(int(speed)))
          text2 = str(int(speed)) + "     " + gps3[1]
    hrx = h_spd + int(49 * math.sin((speed/40)-2))
    hry = v_spd - int(49 * math.cos((speed/40)-2))

    if fix == '1':
       cv2.line(frame,(20,frame_height - 16),(frame_width - 20,frame_height -16),(20,20,20),25,4)
       cv2.putText(frame,text1,(h_txt-47,v_txt+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,255),1)
       cv2.circle(frame,(h_spd,v_spd),52,(58,58,58),-1,8)
       cv2.line(frame,(h_spd,v_spd),(hrx,hry),(200,200,200),2,4)
       cv2.circle(frame,(h_spd,v_spd),32,(28,28,28),-1,4)
       cv2.putText(frame,text2,(h_spd-(5 * (int(l_spd*1.4))),v_spd+10),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,255),1)
    elif fix == '2' or fix == '3':
       cv2.putText(frame,text,(h_txt-47,v_txt+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,0),1)
       cv2.line(frame,(h_spd,v_spd),(hrx,hry),(200,200,200),2,4)
       cv2.circle(frame,(h_spd,v_spd),52,(28,28,28),1,8)
       cv2.circle(frame,(h_spd,v_spd),32,(28,28,28),-1,4)
       cv2.putText(frame,text2,(h_spd-(5 * (int(l_spd*1.4))),v_spd+14),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,255,255),1)
    else:
       cv2.putText(frame,"Waiting for GPS data...",(h_txt-47,v_txt+90),cv2.FONT_HERSHEY_SIMPLEX,0.6,(0,0,255),1)
   
    
    if len(lonfiles) > 2:
       cv2.rectangle(frame,(frame_width - graph_box,0),(frame_width,graph_box),(0,0,0),-1)

       if timestamp[7:8] == "0":
          sf = 50
          lonmax = [x - lon for x in lonfiles]
          latmax = [x - lat for x in latfiles]
       
          lonsf = latsf = lonsn = latsn = 10000000
          if max(lonmax) != 0:
             lonsf = abs(Decimal((graph_box/2)-10)/(Decimal(max(lonmax))))
          if max(latmax) != 0:
             latsf = abs(Decimal((graph_box/2)-10)/(Decimal(max(latmax))))
          if min(lonmax) != 0:
             lonsn = abs(Decimal((graph_box/2)-10)/(Decimal(min(lonmax))))
          if min(latmax) != 0:
             latsn = abs(Decimal((graph_box/2)-10)/(Decimal(min(latmax))))
          sf = min(lonsf,latsf,latsn,lonsn)
       for mcounter in range (0,gps_count - 2,1):
          if mcounter == 0:
             cv2.line(frame,(((lonmax[mcounter])*sf) + frame_width - (graph_box/2),((latmax[mcounter])*sf) + (graph_box/2)),(((lonmax[mcounter+1])*sf)+ frame_width - (graph_box/2),((latmax[mcounter+1])*sf)+ (graph_box/2)),(0,0,255),3,4)
          else:
             cv2.line(frame,(((lonmax[mcounter])*sf) + frame_width - (graph_box/2),((latmax[mcounter])*sf) + (graph_box/2)),(((lonmax[mcounter+1])*sf)+ frame_width - (graph_box/2),((latmax[mcounter+1])*sf)+ (graph_box/2)),(200,200,0),1,4)
             
 
    # write ouput video file
    out.write(frame)
    # Display the resulting frame
    cv2.imshow('Video', frame)

    if cv2.waitKey(1) and 0xFF == ord('q'):
        break
 
# When everything is done, release the capture
video_capture.release()
cv2.destroyAllWindows()
Attachments
screen.jpg
simulation
screen.jpg (65.45 KiB) Viewed 1524 times

Return to “Graphics, sound and multimedia”