Aydan
Posts: 579
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Using framebuffer as overlay for the camera preview

Fri Jun 22, 2018 11:06 am

Hello Everybody,

is there any way to put the camera preview layer behind the framebuffer and make the framebuffer partially transparent (e.g. by color keying)?

What I want to do is to show the camera preview and have UI components (Buttons/Labels) ontop of it.
I do not want or need to have those UI elements show up in camera captures.

Regards
Aydan

User avatar
topguy
Posts: 4540
Joined: Tue Oct 09, 2012 11:46 am
Location: Trondheim, Norway

Re: Using framebuffer as overlay for the camera preview

Fri Jun 22, 2018 1:32 pm

In which (programming) language do you intend to implement this in ?

Python picamera module has some functionality.
https://picamera.readthedocs.io/en/rele ... he-preview

There have been several threads on the subject before with valuable information, but not all of them are easy to find.
viewtopic.php?p=702598

The important part is to know that the GPU in the Pi support different "layers" and you can get software that uses the dispmanx API to show on different layers. ( ref: viewtopic.php?p=94738 )

Aydan
Posts: 579
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: Using framebuffer as overlay for the camera preview

Wed Jul 04, 2018 11:21 am

The goal would be to use picamera in combination with a canvas widget inside a tkinter application to show the camera preview.
I can kind of do this right now, with the drawback that the preview port will always be ontop of the framebuffer and I'd have to make it transparent or disable it completely when the app showing the preview loses focus, so other apps can be used.

The ideal situation would be to e.g. set a specific background color for the canvas widget, and the camera previev would only be visible where the framebuffer has this specific color.
If I remember correctly this is what DirectVideo on Windows does when hardware video decoding is used, because I remember wondering why I could see video fragments through the text of filenames in the file explorer, when the explorer window was partially obscuring a video playback.

I know that picamera supports overlays, but they have some limits e.g. having to be the same size as the preview port, and they don't easily lend themselves for displaying UI elements on them.

Regards
Aydan

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5203
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Using framebuffer as overlay for the camera preview

Wed Jul 04, 2018 12:34 pm

Aydan wrote:
Wed Jul 04, 2018 11:21 am
The ideal situation would be to e.g. set a specific background color for the canvas widget, and the camera previev would only be visible where the framebuffer has this specific color.
If I remember correctly this is what DirectVideo on Windows does when hardware video decoding is used, because I remember wondering why I could see video fragments through the text of filenames in the file explorer, when the explorer window was partially obscuring a video playback.
So your main issue is adding alpha somewhere to the frame buffer, not really the preview window at all.
The default frame buffer is an XRGB8888 layer, but there is a mailbox call that allows you to set it alpha observed. https://github.com/raspberrypi/linux/bl ... are.h#L126
The other issue may be that the frame buffer is at layer -127, which is the lowest layer it can be. There appears to be a mailbox calls defined for setting the layer

Code: Select all

      GET_LAYER                  = 0x0004000c,
      TST_LAYER                  = 0x0004400c,
      SET_LAYER                  = 0x0004800c,
that may allow you to amend the layer as required.
Aydan wrote:I know that picamera supports overlays, but they have some limits e.g. having to be the same size as the preview port, and they don't easily lend themselves for displaying UI elements on them.
I'm not sure where you get the size limitation from. AFAIK they are totally independent of the camera itself, and are just extra instances of the video_render component that you can throw an arbitrary bitmap at. https://picamera.readthedocs.io/en/late ... he-preview
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5203
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Using framebuffer as overlay for the camera preview

Wed Jul 04, 2018 12:36 pm

And chroma keying (using a specific colour to dictate alpha) is often very inefficient as a way to determine that alpha. Always go for a dedicated alpha channel if you can (in this case I think you're forced to).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5203
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Using framebuffer as overlay for the camera preview

Wed Jul 04, 2018 12:45 pm

To adjust the layer to being -5:

Code: Select all

sudo /opt/vc/bin/vcmailbox 0x0004800c 4 4 -5
Actually it's a config.txt setting to enable alpha, from which you can then set it again via the mailbox.

Code: Select all

framebuffer_ignore_alpha=0
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Aydan
Posts: 579
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, near Lake Constance

Re: Using framebuffer as overlay for the camera preview

Wed Jul 04, 2018 1:30 pm

@6by9, thanks for the tips.
Now the problem will be to convince X11 to write the alpha value of an application widget to the framebuffer, and not showing me the desktop/app behind the window. :lol:
This would not be a problem with colour keying.
I may just have to run my ap in direct framebuffer mode without X :(

Regards
Aydan

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 11 guests