How to add text-overlay with raspivid?

[JBAnovling]

Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441

[jamesh]

I'm going to look in to this when I have time. At the moment it would need to be added after the encoding. You cannot add it from the ARM because the bandwidth to transfer an uncompressed frame to the arm, add the caption, then send it back for encoding is too high.

I will need to add a SW stage to the ISP pipeline that adds user supplied text to the uncompressed image - that would need a firmware change on the GPU.

It's not going to be soon though. I'm having to do actual work at the moment.

[fbutler]

It's not going to be soon though. I'm having to do actual work at the moment.

JBAnovling,
In the meantime you may be able to do something with GStreamer pipelines and Overlays to do this. It's quite simple to do static text overlays using GStreamer, but dynamic ones look a bit more involved. Do a Google on the terms: 'Gstreamer dynamic overlays' for more details on approaches to doing it.

[JBAnovling]

I'm going to look in to this when I have time.
....
It's not going to be soon though. I'm having to do actual work at the moment.

Thanks for the reply!
I'm going to ask for more spare time for you in my evening prayer Thanks for the effort so far!
A text-overlay will be a very nice feature to have with Raspberry Pi Camera I can think of many other camera projects which may use a feature like this.
Should I put it in as a suggestion somewhere? or did you make a todo-note about it?

[JBAnovling]

Do a Google on the terms: 'Gstreamer dynamic overlays' for more details on approaches to doing it.

Thanks for the reply!
I will google a bit on Gstreamer dynamic overlays Maybe this is a way to go....

[TESKAn]

There is a demo code that plays a video file on a OGL ES teapot. Currently it works like: File -> OMX decode -> texture, and PiCam is CamModule -> Encoder -> File. If it is possibile to skip "File" stage or even "Encoder" and "Decoder" stages, then you have a texture that displays live video from camera module. And once you have this, you are limited only by imagination (and time you have to spend learning OGL ) on what you can do - like a 3D HUD that displays info on screen in 3D .

[LetHopeItsSnowing]

Can I add myself to the list of people who is really interested in this functionality.

[FFAMax]

I am also interested.
So plus one vote.

[Tauri]

I just received may Raspberry Pi camera module and would like to overlay a timestamp on the video. Is there an easy way to do this or just by studying gstreamer?

[sviborg]

Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441

Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.

[FFAMax]

Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441

Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.

Hi!

As i undestand You do it in CPU, not in GPU, right?
So we waiting effective solution.
Very interesting to set text in some register and pass this work to GPU.
I think it better instead processing video in CPU.

[TESKAn]

Text can be generated and displayed with open GL, so GPU is doing all the work.

[FFAMax]

Text can be generated and displayed with open GL, so GPU is doing all the work.

1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way

[LetHopeItsSnowing]

Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video:
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441

Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.

Do you have any code / an example you could share?

[TESKAn]

Text can be generated and displayed with open GL, so GPU is doing all the work.

1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way

The way I am doing it now the overlay is only on the preview, so if I save video to SD card, I dont have any overlay on it, but all the work of displaying text on screen (together with any 3D textured objects I might want) is done by GPU. Also for live video I use code from raspivid, so all that work is done by the GPU as well. To record video with overlay, you would have to either get live video to texture, use that texture to display video on openGL scene and then store that scene to video, or you have to somehow record display buffer to file.

[FFAMax]

Text can be generated and displayed with open GL, so GPU is doing all the work.

1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way

The way I am doing it now the overlay is only on the preview, so if I save video to SD card, I dont have any overlay on it, but all the work of displaying text on screen (together with any 3D textured objects I might want) is done by GPU. Also for live video I use code from raspivid, so all that work is done by the GPU as well. To record video with overlay, you would have to either get live video to texture, use that texture to display video on openGL scene and then store that scene to video, or you have to somehow record display buffer to file.

Thank you for detail description!

If You meet method with recording display buffer to file, remember about this topic.

Also, if You can share code of Your current method, we will be grateful.

Thanks!

[TESKAn]

You asked for it . You can check it here: https://github.com/TESKAn/hello_camera_teapot
The code is hacked together, terribly (or not at all) commented, but should work if you compile and run it. Or just run the binary .
Currently program shows 3D navball and couple of info panels, in my case the data is supplied by my not - yet - finished autopilot.

[sviborg]

My project sources: https://bitbucket.org/baga/copter

Raspberry specific code located at GLContext.cpp and used for dispmax and EGL initialization. Other files are something like GLES2 sprite rendering engine and should work anywhere.

As for embedding overlay into video file. Maybe it's not really important to do this stuff in real-time. For ex. in my case with quad-rotor clean HD video can be recorded onboard and video with overlay text/graphics will be available at ground station via HDMI/Composite port connected to wireless AV transmitter. Also some telemetry can be saved with timestamps onboard and later processed/embedded into final video file.

[TESKAn]

I have similar thoughts - save flight data and overlay it later if required. After all, we have 16 GB SD card that shouldn't have a problem storing all the flight information .

[LetHopeItsSnowing]

I have similar thoughts - save flight data and overlay it later if required. After all, we have 16 GB SD card that shouldn't have a problem storing all the flight information .

One of the challenges I had overlaying data onto the video was synchronisation http://www.stuffaboutcode.com/2013/07/r ... h-obd.html.

I found that the longer the video the more slip you got, mainly due to things like dropped frames and slightly lower frame rates than expected. By dropping the video quality and frame rate I got better sync but still not great.

Anyway Im in the process of changing raspivid to allow you to do an inter process communication to pull out the current frame it is writing, this will hopefully allow me to log the frame with the data and make post processing data sync a LOT easier.

Ill let you know how I get on.

[LetHopeItsSnowing]

Hi,

I have created a custom version of raspivid which allows you to pull out the current frame while its running. This makes syncing data much easier as you can log the frame number with the data and use that to get an accurate time of when the data needs to appear in the video.

http://www.stuffaboutcode.com/2013/09/r ... pivid.html

Let me know how you get on / if its useful.

Mart

[FFAMax]

Hi!

Thanks!
Also thanks for detailed how-to

Hi,

I have created a custom version of raspivid which allows you to pull out the current frame while its running. This makes syncing data much easier as you can log the frame number with the data and use that to get an accurate time of when the data needs to appear in the video.

http://www.stuffaboutcode.com/2013/09/r ... pivid.html

Let me know how you get on / if its useful.

Mart

[mik79]

I would be interested too!

I had nearly the same idea as JBAnovling.

A Raspberry with camera and gps modul would much cooler than a gopro and telemetry

Michael

[Arp]

Me too!

Although I would use this as dashcam.

My current dashcam (webcam+usb-gps-dongle+laptop+ffmpeg) also had this sync problem. I solved it by having a script writing all the gps data into a line in a text file, once every second. The capturing from the webcam was done in ffmpeg, and this has an option wich alows to read in a line from a text file for every frame that is saved. So even if the framerate fluctuates, the synchronisation between video and gps data would be always precise within 1 second.

Here I would to this for GPS + OBD data, in an optimal case directly on the captured video from the camera. Might need a little overclocking of the raspberry... dont know

[ChiYuan]

It's been awhile since this subject posted. I wonder if anyone know add text overlay with raspivid is been implemented successfully somewhere. Thanks!