User avatar
JBAnovling
Posts: 14
Joined: Sun May 19, 2013 6:22 am
Location: Denmark
Contact: Website

How to add text-overlay with raspivid?

Tue May 21, 2013 8:48 am

Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video: :)
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441
More of my videos on http://www.youtube.com/user/JBAnovling

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25417
Joined: Sat Jul 30, 2011 7:41 pm

Re: How to add text-overlay with raspivid?

Tue May 21, 2013 9:29 am

I'm going to look in to this when I have time. At the moment it would need to be added after the encoding. You cannot add it from the ARM because the bandwidth to transfer an uncompressed frame to the arm, add the caption, then send it back for encoding is too high.

I will need to add a SW stage to the ISP pipeline that adds user supplied text to the uncompressed image - that would need a firmware change on the GPU.

It's not going to be soon though. I'm having to do actual work at the moment.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

User avatar
fbutler
Posts: 302
Joined: Thu Mar 15, 2012 4:09 pm
Location: Surrey, England

Re: How to add text-overlay with raspivid?

Tue May 21, 2013 10:10 am

jamesh wrote:It's not going to be soon though. I'm having to do actual work at the moment.
JBAnovling,
In the meantime you may be able to do something with GStreamer pipelines and Overlays to do this. It's quite simple to do static text overlays using GStreamer, but dynamic ones look a bit more involved. Do a Google on the terms: 'Gstreamer dynamic overlays' for more details on approaches to doing it.

User avatar
JBAnovling
Posts: 14
Joined: Sun May 19, 2013 6:22 am
Location: Denmark
Contact: Website

Re: How to add text-overlay with raspivid?

Tue May 21, 2013 6:51 pm

jamesh wrote:I'm going to look in to this when I have time.
....
It's not going to be soon though. I'm having to do actual work at the moment.
Thanks for the reply!
I'm going to ask for more spare time for you in my evening prayer ;) Thanks for the effort so far! :)
A text-overlay will be a very nice feature to have with Raspberry Pi Camera :) I can think of many other camera projects which may use a feature like this.
Should I put it in as a suggestion somewhere? or did you make a todo-note about it? ;)
More of my videos on http://www.youtube.com/user/JBAnovling

User avatar
JBAnovling
Posts: 14
Joined: Sun May 19, 2013 6:22 am
Location: Denmark
Contact: Website

Re: How to add text-overlay with raspivid?

Tue May 21, 2013 6:55 pm

fbutler wrote:Do a Google on the terms: 'Gstreamer dynamic overlays' for more details on approaches to doing it.
Thanks for the reply!
I will google a bit on Gstreamer dynamic overlays :) Maybe this is a way to go....
More of my videos on http://www.youtube.com/user/JBAnovling

TESKAn
Posts: 11
Joined: Tue May 21, 2013 8:09 pm

Re: How to add text-overlay with raspivid?

Tue May 21, 2013 8:19 pm

There is a demo code that plays a video file on a OGL ES teapot. Currently it works like: File -> OMX decode -> texture, and PiCam is CamModule -> Encoder -> File. If it is possibile to skip "File" stage or even "Encoder" and "Decoder" stages, then you have a texture that displays live video from camera module. And once you have this, you are limited only by imagination (and time you have to spend learning OGL :) ) on what you can do - like a 3D HUD that displays info on screen in 3D 8-) .

User avatar
LetHopeItsSnowing
Posts: 357
Joined: Sat May 26, 2012 6:40 am
Location: UK
Contact: Website

Re: How to add text-overlay with raspivid?

Tue Jul 30, 2013 7:45 pm

Can I add myself to the list of people who is really interested in this functionality.
"am I getting slower, or is stuff more complicated; either way I now have to write it down - stuffaboutcode.com"

FFAMax
Posts: 30
Joined: Fri Nov 09, 2012 10:30 am
Location: Moscow / GMT+4

Re: How to add text-overlay with raspivid?

Wed Jul 31, 2013 1:06 am

I am also interested.
So plus one vote.

Tauri
Posts: 3
Joined: Mon Aug 13, 2012 3:08 pm

Re: How to add text-overlay with raspivid?

Mon Aug 12, 2013 6:17 pm

I just received may Raspberry Pi camera module and would like to overlay a timestamp on the video. Is there an easy way to do this or just by studying gstreamer?

sviborg
Posts: 8
Joined: Thu Jan 31, 2013 10:56 am

Re: How to add text-overlay with raspivid?

Mon Aug 12, 2013 9:40 pm

JBAnovling wrote:Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video: :)
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441
Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.
Attachments
_overlay.jpg
_overlay.jpg (63.83 KiB) Viewed 26314 times

FFAMax
Posts: 30
Joined: Fri Nov 09, 2012 10:30 am
Location: Moscow / GMT+4

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 2:54 am

sviborg wrote:
JBAnovling wrote:Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video: :)
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441
Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.
Hi!

As i undestand You do it in CPU, not in GPU, right?
So we waiting effective solution.
Very interesting to set text in some register and pass this work to GPU.
I think it better instead processing video in CPU.

TESKAn
Posts: 11
Joined: Tue May 21, 2013 8:09 pm

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 12:41 pm

Text can be generated and displayed with open GL, so GPU is doing all the work.

FFAMax
Posts: 30
Joined: Fri Nov 09, 2012 10:30 am
Location: Moscow / GMT+4

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 1:16 pm

TESKAn wrote:Text can be generated and displayed with open GL, so GPU is doing all the work.
1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way

User avatar
LetHopeItsSnowing
Posts: 357
Joined: Sat May 26, 2012 6:40 am
Location: UK
Contact: Website

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 1:30 pm

sviborg wrote:
JBAnovling wrote:Is it possible to add a text-overlay directly to the video captured with raspivid ?

I would like to add some GPS-info and some other live data from my Quadcopter (using the UART to get the info from my quad).

It would be very nice to have some live data in my Raspberry Pi Quadcopter video: :)
http://www.raspberrypi.org/phpBB3/viewt ... 43&t=44441
Hi there! I'm also working on similar project (http://realimba.16mb.com/index.php?page=multirotor).
It's possible to add overlay via OpenGL rendering with alpha to top level surface.

The surface level can be selected when setting up EGL context with vc_dispmanx_element_add level parameter. I'm using 99999 value. Than just glClearColor(0, 0, 0, 0) and draw stuff.
Do you have any code / an example you could share?
"am I getting slower, or is stuff more complicated; either way I now have to write it down - stuffaboutcode.com"

TESKAn
Posts: 11
Joined: Tue May 21, 2013 8:09 pm

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 2:01 pm

FFAMax wrote:
TESKAn wrote:Text can be generated and displayed with open GL, so GPU is doing all the work.
1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way
The way I am doing it now the overlay is only on the preview, so if I save video to SD card, I dont have any overlay on it, but all the work of displaying text on screen (together with any 3D textured objects I might want) is done by GPU. Also for live video I use code from raspivid, so all that work is done by the GPU as well. To record video with overlay, you would have to either get live video to texture, use that texture to display video on openGL scene and then store that scene to video, or you have to somehow record display buffer to file.

FFAMax
Posts: 30
Joined: Fri Nov 09, 2012 10:30 am
Location: Moscow / GMT+4

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 2:15 pm

TESKAn wrote:
FFAMax wrote:
TESKAn wrote:Text can be generated and displayed with open GL, so GPU is doing all the work.
1. Camera -> GPU (set up overlay) -> SD
2. Camera -> GPU -> SD -> any soft + open GL -> GPU (set up text-overlay) -> SD

I am talking about optimization by first way
The way I am doing it now the overlay is only on the preview, so if I save video to SD card, I dont have any overlay on it, but all the work of displaying text on screen (together with any 3D textured objects I might want) is done by GPU. Also for live video I use code from raspivid, so all that work is done by the GPU as well. To record video with overlay, you would have to either get live video to texture, use that texture to display video on openGL scene and then store that scene to video, or you have to somehow record display buffer to file.
Thank you for detail description!

If You meet method with recording display buffer to file, remember about this topic.

Also, if You can share code of Your current method, we will be grateful.

Thanks!

TESKAn
Posts: 11
Joined: Tue May 21, 2013 8:09 pm

Re: How to add text-overlay with raspivid?

Tue Aug 13, 2013 4:01 pm

You asked for it ;) . You can check it here: https://github.com/TESKAn/hello_camera_teapot
The code is hacked together, terribly (or not at all) commented, but should work if you compile and run it. Or just run the binary ;) .
Currently program shows 3D navball and couple of info panels, in my case the data is supplied by my not - yet - finished autopilot.

sviborg
Posts: 8
Joined: Thu Jan 31, 2013 10:56 am

Re: How to add text-overlay with raspivid?

Wed Aug 14, 2013 9:29 pm

My project sources: https://bitbucket.org/baga/copter

Raspberry specific code located at GLContext.cpp and used for dispmax and EGL initialization. Other files are something like GLES2 sprite rendering engine and should work anywhere.

As for embedding overlay into video file. Maybe it's not really important to do this stuff in real-time. For ex. in my case with quad-rotor clean HD video can be recorded onboard and video with overlay text/graphics will be available at ground station via HDMI/Composite port connected to wireless AV transmitter. Also some telemetry can be saved with timestamps onboard and later processed/embedded into final video file.

TESKAn
Posts: 11
Joined: Tue May 21, 2013 8:09 pm

Re: How to add text-overlay with raspivid?

Thu Aug 15, 2013 8:13 am

I have similar thoughts - save flight data and overlay it later if required. After all, we have 16 GB SD card that shouldn't have a problem storing all the flight information :) .

User avatar
LetHopeItsSnowing
Posts: 357
Joined: Sat May 26, 2012 6:40 am
Location: UK
Contact: Website

Re: How to add text-overlay with raspivid?

Thu Aug 22, 2013 12:40 pm

TESKAn wrote:I have similar thoughts - save flight data and overlay it later if required. After all, we have 16 GB SD card that shouldn't have a problem storing all the flight information :) .
One of the challenges I had overlaying data onto the video was synchronisation http://www.stuffaboutcode.com/2013/07/r ... h-obd.html.

I found that the longer the video the more slip you got, mainly due to things like dropped frames and slightly lower frame rates than expected. By dropping the video quality and frame rate I got better sync but still not great.

Anyway Im in the process of changing raspivid to allow you to do an inter process communication to pull out the current frame it is writing, this will hopefully allow me to log the frame with the data and make post processing data sync a LOT easier.

Ill let you know how I get on.
"am I getting slower, or is stuff more complicated; either way I now have to write it down - stuffaboutcode.com"

User avatar
LetHopeItsSnowing
Posts: 357
Joined: Sat May 26, 2012 6:40 am
Location: UK
Contact: Website

Re: How to add text-overlay with raspivid?

Wed Sep 04, 2013 6:07 am

Hi,

I have created a custom version of raspivid which allows you to pull out the current frame while its running. This makes syncing data much easier as you can log the frame number with the data and use that to get an accurate time of when the data needs to appear in the video.

http://www.stuffaboutcode.com/2013/09/r ... pivid.html

Let me know how you get on / if its useful.

Mart
"am I getting slower, or is stuff more complicated; either way I now have to write it down - stuffaboutcode.com"

FFAMax
Posts: 30
Joined: Fri Nov 09, 2012 10:30 am
Location: Moscow / GMT+4

Re: How to add text-overlay with raspivid?

Wed Sep 04, 2013 6:10 am

Hi!

Thanks!
Also thanks for detailed how-to :)
LetHopeItsSnowing wrote:Hi,

I have created a custom version of raspivid which allows you to pull out the current frame while its running. This makes syncing data much easier as you can log the frame number with the data and use that to get an accurate time of when the data needs to appear in the video.

http://www.stuffaboutcode.com/2013/09/r ... pivid.html

Let me know how you get on / if its useful.

Mart

mik79
Posts: 4
Joined: Sun Mar 17, 2013 6:00 pm
Location: NRW Germany

Re: How to add text-overlay with raspivid?

Fri Sep 06, 2013 10:50 am

I would be interested too!

I had nearly the same idea as JBAnovling.

A Raspberry with camera and gps modul would much cooler than a gopro and telemetry ;)

Michael

Arp
Posts: 26
Joined: Wed Sep 11, 2013 1:26 pm
Location: NRW, Germany

Re: How to add text-overlay with raspivid?

Thu Sep 12, 2013 1:05 pm

Me too!

Although I would use this as dashcam.

My current dashcam (webcam+usb-gps-dongle+laptop+ffmpeg) also had this sync problem. I solved it by having a script writing all the gps data into a line in a text file, once every second. The capturing from the webcam was done in ffmpeg, and this has an option wich alows to read in a line from a text file for every frame that is saved. So even if the framerate fluctuates, the synchronisation between video and gps data would be always precise within 1 second.

Here I would to this for GPS + OBD data, in an optimal case directly on the captured video from the camera. Might need a little overclocking of the raspberry... dont know :D

ChiYuan
Posts: 2
Joined: Mon Feb 10, 2014 6:25 pm

Re: How to add text-overlay with raspivid?

Wed Feb 12, 2014 12:46 am

It's been awhile since this subject posted. I wonder if anyone know add text overlay with raspivid is been implemented successfully somewhere. Thanks!

Return to “Camera board”