EGL - How to use fb1?


25 posts
by yo1dog » Tue Oct 22, 2013 11:24 pm
I posted this question on StackOverflow but with no responses I think this may be a better place to ask.

I am trying to modify the hello_triangle example (/opt/vc/src/hello_pi/hello_triangle) on my Raspberry Pi to output to fb1 instead of fb0.

I have searched everything I can think of with no answers (it could be I don't know what to search for). It seems like the line I need to modify is this line:
Code: Select all
state->display = eglGetDisplay(EGL_DEFAULT_DISPLAY);


The docs state the parameter is of type NativeDisplayType and "Specifies the display to connect to. EGL_DEFAULT_DISPLAY indicates the default display." However, it does not give any information on how to list/find available displays.

EGL_DEFAULT_DISPLAY is defined as (NativeDisplayType)0 so I guessed that 0 referred to fb0, but using (NativeDisplayType)1 for fb1 caused eglGetDisplay to return EGL_NO_DISPLAY.

The original example outputs to fb0 as it should, no problems there.
I also know my display is working correctly as I can use fbi to display images on fb1 just fine.
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by texy » Wed Oct 23, 2013 6:39 am
Hi,
not exaclty cutting-edge for your particular needs, but there is a utility that copies fb0 to fb1 as a background task - search the forum for fbcp

Hope this helps,
Texy
"2.8inch TFT LCD + Touch screen" add-on boards for sale here :
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=93&t=65566
50p goes to the Foundation ;-)
Forum Moderator
Forum Moderator
Posts: 2355
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by yo1dog » Wed Oct 23, 2013 3:40 pm
I suspect a couple problems:

1. The displays attached to each framebuffer have different resolutions and color bit-depths. Although, I bet I could manually set/override this.
2. How fast does this update? Will it cause lag or tearing? I guess I should just try it but I am at work currently.

Thanks for this solution. I will try it out later. I would still like to figure out my original question as this seems a bit hacky.
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by Paeryn » Wed Oct 23, 2013 10:44 pm
The opengl output isn't drawn into the linux framebuffers, it's drawn into an entirely separate layer which is composited by the Pi's internal display manager together with any other layers (of which fb0 is one such layer) to generate the image which is shown.

On the RPi there is only one display. Taken from the source code for eglGetDisplay() for the RPi :-
Code: Select all
   Implementation notes:

   We only support one display. This is assumed to have a native display_id
   of 0 (==EGL_DEFAULT_DISPLAY) and an EGLDisplay id of 1
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Thu Oct 24, 2013 4:21 am
Hmm... So you are saying it is impossible for egl to use fb1. I wonder why they would implement that restriction. I won't pretend to remotely understand what goes on at that level.

If that is the case. Is there anyway to make the default video output (on-board HDMI/Composite) fb1 and my display fb0 (the default display) so it can be used by egl and openGL ES?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by Paeryn » Thu Oct 24, 2013 6:21 pm
yo1dog wrote:Hmm... So you are saying it is impossible for egl to use fb1. I wonder why they would implement that restriction. I won't pretend to remotely understand what goes on at that level.

Yes, EGL on the RPi doesn't use the linux framebuffers at all, not fb1 nor fb0. The graphics hardware works at a higher level (dispmanx), the display is made up of multiple layers, when linux starts it creates one layer to cover the full screen for fb0. When you create a window with EGL you also create a new layer and tell the hardware to show it in front of the others.

As far as I understand, the display you see is generated on the fly by the compositing hardware, so if you have a fullscreen fb0 with an EGL window in the middle showing your OpenGL / OpenVG / OpenMAX then there isn't a single bitmap image containing what you see.

If that is the case. Is there anyway to make the default video output (on-board HDMI/Composite) fb1 and my display fb0 (the default display) so it can be used by egl and openGL ES?

If you want to show your OpenGL window on a secondary display (reading your other post http://www.raspberrypi.org/phpBB3/viewtopic.php?f=29&t=59081#p442721 you have a USB display - I'm assuming you want it showing on there) then you could do it by having the EGL layer behind the framebuffer layer (ideally you'd want to create the layer but not have it actively visible but I'm not sure how you'd go about that, other than to have it hidden behind all other layers). Then when you've finished your OpenGL drawing make a copy of the window with glReadPixels() and write the data out to fb1.
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Fri Oct 25, 2013 2:11 am
Any idea how I write to fb1?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by AndyD » Fri Oct 25, 2013 3:00 am
yo1dog wrote:Any idea how I write to fb1?

There is an extensive blog written by a (former?) member of this forum called Raspberry Compote that explains writing to the frame-buffer on the Raspberry Pi. The source code that copies from the Raspberry Pi compositor to fb1 may also be of interest. It might be worth experimenting I have a feeling that using glReadPixels() may be slower than using vc_dispmanx_snapshot() (as is used in rpi-fbcp). I have no evidence to support that, but I vaguely remember someone in another thread complaining the glReadPixels() was slow.
Posts: 954
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
by yo1dog » Fri Oct 25, 2013 4:25 am
Going off that source I was able to add the following to the hello_triangle example:
Code: Select all
//...

static int screen_width    = 480;
static int screen_height   = 272;
static int screen_bitdepth = 2;
static int screen_datasize;

static DISPMANX_DISPLAY_HANDLE_T display;
static DISPMANX_RESOURCE_HANDLE_T screen_resource;
static int fbfd = 0;
static char *fbp = 0;

static VC_RECT_T rect;

//...

static void init_ogl(CUBE_STATE_T *state)
{
   screen_datasize = screen_width * screen_height * screen_bitdepth;

   display = vc_dispmanx_display_open(0);
   // do check and exit if failed

   fbfd = open("/dev/fb1", O_RDWR);
   // do check and exit if failed

   uint32_t image_prt;
   screen_resource = vc_dispmanx_resource_create(VC_IMAGE_RGB565, screen_width, screen_height, &image_prt);
   // do check and exit if failed

   fbp = (char*)mmap(0, screen_datasize, PROT_READ | PROT_WRITE, MAP_SHARED, fbfd, 0);
   // do check and exit if failed

   vc_dispmanx_rect_set(&rect, 0, 0, screen_width, screen_height);

   printf("Everything worked\n");

//...

   while (!terminate)
   {
      update_model(state);
      redraw_scene(state);

      printf("start\n");

      int ret = vc_dispmanx_snapshot(display, screen_resource, 0);
      printf("ret1: %i\n", ret);

printf("middle\n");

      ret = vc_dispmanx_resource_read_data(screen_resource, &rect, fbp, screen_width * screen_bitdepth);
      printf("ret2: %i\n", ret);

      printf("end\n");

//...

static void exit_func(void)
{

//...

   munmap(fbp, screen_datasize);
   close(fbfd);
   vc_dispmanx_resource_delete(screen_resource);
   vc_dispmanx_display_close(display);

//...

But the following prints out:

Everything worked
start
ret1: 0
middle
ret2: -1
end
start

And then it hangs forever. So on the first loop I see the cube on fb0 but no change to fb1 and vc_dispmanx_resource_read_data returns -1. On the second loop it hangs on vc_dispmanx_snapshot.

I noticed that when trying to use glReadPixels it also hangs forever.

EDIT: fbcp does not work and also seems to hang.
EDIT2: I put the same print statements around the vc_dispmanx_resource_read_data and vc_dispmanx_snapshot in fbcp source and it prints out exactly the same as my modified hello_triangle.

Any ideas?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by Paeryn » Fri Oct 25, 2013 2:54 pm
yo1dog wrote:Everything worked
start
ret1: 0
middle
ret2: -1
end
start

And then it hangs forever. So on the first loop I see the cube on fb0 but no change to fb1 and vc_dispmanx_resource_read_data returns -1. On the second loop it hangs on vc_dispmanx_snapshot.

I noticed that when trying to use glReadPixels it also hangs forever.

EDIT: fbcp does not work and also seems to hang.
EDIT2: I put the same print statements around the vc_dispmanx_resource_read_data and vc_dispmanx_snapshot in fbcp source and it prints out exactly the same as my modified hello_triangle.

Any ideas?

It looks like vc_dispmanx_resource_read_data() fails when trying to write to the mmap'ed memory. If you malloc a temporary buffer to read the resource into and the memcpy that to the mmap'ed memory then the program runs fine. Whether it actually copies the display I can't say (my RPi isn't connected to a display at the moment, and I don't have a usb display), I ran it copying into fb0.
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Fri Oct 25, 2013 3:17 pm
I think I tried that with the same results. I can confirm on Sunday.

I assumed the read is failing because it returns -1, but why is it hanging on vc_dispmanx_snapshot?

fbcp seems to work for everyone else, why would it not work on my setup?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by Paeryn » Sat Oct 26, 2013 3:27 am
I would guess something along the lines of, the read request locking the resource that holds the screenshot and fails to free it when the read fails. The next screenshot request could then be sat waiting for the resource to be unlocked.

I tried the fbcp program (again changing it to use to fb0), it locks up exactly the same for me.

I've just tried the modified version of your program (reading to malloc'ed memory and then memcpy'ing it to fb0) and it definitely copies the image from when the opengl window was open onto fb0.

The only problem with using vc_dispmanx_snapshot is that your usb display fb1 will only be able to show exactly what is on the main display that the RPi is outputting via HDMI / Composite.
I got the glReadPixels version working without you seeing the actual opengl window on screen, but only in 32bit colour. I couldn't get gles1 to return anything other than a 32bit RGBA bitmap even when explicitly asking for a 16bit 565 window with eglSaneChooseConfigBRCM.
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Tue Oct 29, 2013 4:49 pm
I was able to read the snapshot into memory and then memcpy it to fb1. That worked and I can now see a scaled version of fb0 on fb1.

Problem: I don't want to scale, I just want to crop 480x272 pixels (resolution of my LCD) of fb0 and display it in fb1.

One solution would be to force fb0 to have the same resolution (480x272) so that no cropping or scaling would be needed and a direct copy could be used. I tried plugging in an HDMI monitor and in /boot/config.txt setting hdmi_mode=3 (480p) and hdmi_force_hotplug=1 (just in case) but the resolution was still higher than 480x272. Can anyone think of a way to force fb0 to be the resolution I need?

Another solution would be to use glReadPixels and read the pixel data I need into fb1. Problem is that the RPI implementation of openGL ES only supports RGB888 but my LCD is 16bit so I need RGB565. Converting the formats manually (using the CPU) is too slow and drops the framerate significantly (maybe the algorithm I am using is inefficient?).

My current solution is to make the snapshot the same size as fb0 and read it into memory. When memcpy-ing into fb1 I only copy the the top-left 480x272 pixels using the following code:

Code: Select all
//...
// get the resolution of fb0
int fb0_screen_width;
int fb0_screen_height;
graphics_get_display_size(0, &fb0_screen_width, &fb0_screen_height);

// set the resolution of fb1
int fb1_screen_width = 480;
int fb1_screen_height = 272;
int fb1_screen_bitdepth = 2; // 16bit RGB565

// allocate some memory for reading the snapshot into as fb0_pointer
unsigned short * fb0_pointer = malloc(fb0_screen_width * fb0_screen_height * fb1_screen_bitdepth); // note we use fb1_screen_bitdepth becuase the resource we define below uses 16bit RGB565 as the pixel format, same as fb1.

// open fb1 and map it to memory as fb1_pointer
int fb1fd = open("/dev/fb1", O_RDWR)
unsigned short * fb1_pointer = mmap(0, fb1_screen_width * fb1_screen_height * fb1_screen_bitdepth, PROT_READ | PROT_WRITE, MAP_SHARED, fb1fd, 0);

// create the rectangle the snapshot should be scaled into (I think?). Use the original size (size of fb0) so nothing actually gets scaled.
VC_RECT_T rect;
vc_dispmanx_rect_set(&rect, 0, 0, fb0_screen_width, fb0_screen_height);

// create the dispmanx resource and use 16bit RGB565 as the format. Not sure what this does or how the width and height are used, or what the image_prt is used for?
uint32_t image_prt;
screen_resource = vc_dispmanx_resource_create(VC_IMAGE_RGB565, fb0_screen_width, fb0_screen_height, &image_prt);

//...

// take a snapshot and read the pixel data into fb0_pointer.
vc_dispmanx_snapshot(display, screen_resource, 0);
vc_dispmanx_resource_read_data(screen_resource, &rect, fb0_pointer, fb1_screen_width * fb1_screen_bitdepth); // not sure what this last parameter does but if you make it less than this it cuts off the bottom on fb1

// copy only the top left pixels to fb1
int y;
for (y = 0; y < fb1_screen_height; y++)
   memcpy(
      fb1_pointer + y * fb1_screen_width, // beginning of the row for fb1
      fb0_pointer + y * fb0_screen_width, // beginning of the row for fb0
      fb1_screen_width * fb1_screen_bitdepth); // one row of pixel data for fb1


As you can see from the comments above, I don't understand everything that is going on here and I bet there is a better way of doing this. The snapshot and reading above drops the framerate a bit. Is there a faster way to do this? Is there really not a way to snapshot just a portion of fb0?

How do programs like fbi and xserver use only fb1 without having to render to fb0?

Is it possible to use the GPU to convert the pixel format so that I can use glReadPixels and don't have to use EGL to render to fb0 and copy to fb1?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by karlkiste » Tue Oct 29, 2013 7:24 pm
yo1dog wrote:How do programs like fbi and xserver use only fb1 without having to render to fb0?


They just render to fb1. I'm displaying pictures like this:

gm convert picture.jpg -scale ${xres}x${yres}! -depth 8 RGB:/dev/fb1

There's no trick to it.
To just see anything, you can just

cat /dev/urandom >/dev/fb1

You're doing something completely different. It's an EGL buffer you're rendering to, not a framebuffer. If it was the framebuffer, you could just read /dev/fb0 and copy it to /dev/fb1. Using graphicsmagick, you could also scale or crop. To just crop, "dd" could even be sufficient. Or "cut". But it is not fb0 you're rendering to. That's what makes your task a bit more complicated.
Posts: 149
Joined: Tue Jan 22, 2013 8:50 am
Location: berlin, germany
by yo1dog » Tue Oct 29, 2013 9:59 pm
Can I use openGL ES to render without using EGL? Will it still take advantage of the GPU? Is there a way to use the GPU to convert the rendered image from openGL ES to RGB 565 or a more efficient way to do it with the CPU that won't kill my framerate?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by karlkiste » Wed Oct 30, 2013 4:57 am
yo1dog wrote:Can I use openGL ES to render without using EGL? Will it still take advantage of the GPU? Is there a way to use the GPU to convert the rendered image from openGL ES to RGB 565 or a more efficient way to do it with the CPU that won't kill my framerate?


I'm not very much into this subject, I only understand the overall principle. I don't know the difference between OpenGL ES and EGL, I only think of those as "GPU features". Those features usually go directly to the screen output, without touching the CPU. The framebuffer is a memory area which belongs to the CPU, is read by the GPU, mixed with the "GPU features" and then sent to the display. This mixing is done by dedicated hardware, and goes in parallel to all other function. No time is needed for that.

You're copying data from the "GPU features" back to the CPU. This is something unusual, and it takes time. Therefore, I'm not sure if it is really improving speed to use the "GPU features". You could instead create the graphics directly on the CPU and you would not have to read the result back that way. Your Screen has quite a low resolution, so it's probably possible to use only software rendering.
Posts: 149
Joined: Tue Jan 22, 2013 8:50 am
Location: berlin, germany
by Paeryn » Wed Oct 30, 2013 5:21 am
yo1dog wrote:One solution would be to force fb0 to have the same resolution (480x272) so that no cropping or scaling would be needed and a direct copy could be used. I tried plugging in an HDMI monitor and in /boot/config.txt setting hdmi_mode=3 (480p) and hdmi_force_hotplug=1 (just in case) but the resolution was still higher than 480x272. Can anyone think of a way to force fb0 to be the resolution I need?

I doubt you can set the display that small, I think the smallest you can go is 640x480.

yo1dog wrote:Another solution would be to use glReadPixels and read the pixel data I need into fb1. Problem is that the RPI implementation of openGL ES only supports RGB888 but my LCD is 16bit so I need RGB565. Converting the formats manually (using the CPU) is too slow and drops the framerate significantly (maybe the algorithm I am using is inefficient?).

The RPi does support 565, if you ask for it you will see the colour banding. To get it you have to pick the right config. The first config returned by eglChooseConfig() when asking for a 565 will NOT be 565 but 888 as the spec says that the configs are sorted with colour sizes that biggest first. Broadcom have an extra function eglSaneChooseConfigBRCM() which will return the list sorted with minimum first.

For some reason though glReadPixels only supports GL_RGBA,GL_UNSIGNED_BYTE. It should also support a second format returned from glGet() GL_IMPLEMENTATION_COLOR_READ_FORMAT_OES,GL_IMPLEMENTATION_COLOR_COLOR_TYPE_OES but glGet hardcodes these to return GL_RGBA and GL_UNSIGNED_BYTE rather than letting them have values corresponding to the colour buffer format.

Some notes on your code that you gave,
Code: Select all
// create the dispmanx resource and use 16bit RGB565 as the format. Not sure what this does or how the width and height are used, or what the image_prt is used for?
uint32_t image_prt;
screen_resource = vc_dispmanx_resource_create(VC_IMAGE_RGB565, fb0_screen_width, fb0_screen_height, &image_prt);

It creates an image resource in the gpu's memory to hold the snapshot with a resolution of fb0_screen_width x fb0_screen_height. The image_prt just needs to be a pointer, it's meant to hold a pointer to the bitmap but as that memory isn't available to the ARM it will always be a null pointer.
Code: Select all
vc_dispmanx_resource_read_data(screen_resource, &rect, fb0_pointer, fb1_screen_width * fb1_screen_bitdepth); // not sure what this last parameter does but if you make it less than this it cuts off the bottom on fb1

The last parameter is the stride in bytes between successive lines of the destination bitmap. It's useful if the memory you are copying to is of a bitmap that is wider than the source bitmap (or requires extra padding at the end of each line).
Code: Select all
// copy only the top left pixels to fb1
int y;
for (y = 0; y < fb1_screen_height; y++)
   memcpy(
      fb1_pointer + y * fb1_screen_width, // beginning of the row for fb1
      fb0_pointer + y * fb0_screen_width, // beginning of the row for fb0
      fb1_screen_width * fb1_screen_bitdepth); // one row of pixel data for fb1

The memcpy parameters are wrong, it should be
Code: Select all
   memcpy(
      fb1_pointer + y * fb1_screen_width * fb1_screen_bitdepth, // beginning of the row for fb1
      fb0_pointer + y * fb0_screen_width * fb1_screen_bitdepth, // beginning of the row for fb0
      fb1_screen_width * fb1_screen_bitdepth); // one row of pixel data for fb1

And, just nitpicking, but your variable name fb1_screen_bitdepth isn't descriptive of the value it holds as the bitdepth is 16, fb1_screen_bytes_per_pixel would be more accurate.

yo1dog wrote:As you can see from the comments above, I don't understand everything that is going on here and I bet there is a better way of doing this. The snapshot and reading above drops the framerate a bit. Is there a faster way to do this? Is there really not a way to snapshot just a portion of fb0?

I've tried various ways of getting the egl resource but to no avail. There is an egl function that should copy the colour buffer to a native pixmap - eglCopyBuffers(), but looking at the egl source code native pixmaps aren't supported, the code sets a null pointer for the bitmap data.
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Wed Oct 30, 2013 6:19 am
Thanks for all the great info and your time.

Paeryn wrote:The RPi does support 565, if you ask for it you will see the colour banding. To get it you have to pick the right config. The first config returned by eglChooseConfig() when asking for a 565 will NOT be 565 but 888 as the spec says that the configs are sorted with colour sizes that biggest first. Broadcom have an extra function eglSaneChooseConfigBRCM() which will return the list sorted with minimum first.

For some reason though glReadPixels only supports GL_RGBA,GL_UNSIGNED_BYTE. It should also support a second format returned from glGet() GL_IMPLEMENTATION_COLOR_READ_FORMAT_OES,GL_IMPLEMENTATION_COLOR_COLOR_TYPE_OES but glGet hardcodes these to return GL_RGBA and GL_UNSIGNED_BYTE rather than letting them have values corresponding to the colour buffer format.

So, the pixels rendered by openGL can be 565 format but when we read the pixel data it always converts it to 888? That seems silly. :P

Paeryn wrote:The memcpy parameters are wrong, it should be
Code: Select all
   memcpy(
      fb1_pointer + y * fb1_screen_width * fb1_screen_bitdepth, // beginning of the row for fb1
      fb0_pointer + y * fb0_screen_width * fb1_screen_bitdepth, // beginning of the row for fb0
      fb1_screen_width * fb1_screen_bitdepth); // one row of pixel data for fb1


Actually, because fb0/fb1_pointer's is of type short, it is not necessary to multiply by fb1_screen_bitdepth (2) because c knows the type is short (which is 2 bytes in size). When you add 1 to a pointer it advances the size of the type (sizeof(short) = 2) in bytes.

Paeryn wrote:And, just nitpicking, but your variable name fb1_screen_bitdepth isn't descriptive of the value it holds as the bitdepth is 16, fb1_screen_bytes_per_pixel would be more accurate.

I appreciate the correction. :)

Paeryn wrote:I've tried various ways of getting the egl resource but to no avail. There is an egl function that should copy the colour buffer to a native pixmap - eglCopyBuffers(), but looking at the egl source code native pixmaps aren't supported, the code sets a null pointer for the bitmap data.


So back to square one, eh?
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by karlkiste » Wed Oct 30, 2013 6:49 am
yo1dog wrote:[...] but the resolution was still higher than 480x272. Can anyone think of a way to force fb0 to be the resolution I need?


Just stumbled over it: Of course you can set that resolution:

in config.txt

disable_overscan=1
framebuffer_width=480
framebuffer_height=272

And, very important, do NOT plug in an HDMI monitor! The monitor will tell via EDID that it can't display such a low resolution, and the pi will set a resolution supported by the monitor. If no HDMI is connected, the output will go to CVBS, and the resolution can be set to whatever you want.
Posts: 149
Joined: Tue Jan 22, 2013 8:50 am
Location: berlin, germany
by yo1dog » Wed Oct 30, 2013 3:33 pm
Thanks! I'll try this tonight.
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by Paeryn » Wed Oct 30, 2013 5:19 pm
yo1dog wrote:Actually, because fb0/fb1_pointer's is of type short, it is not necessary to multiply by fb1_screen_bitdepth (2) because c knows the type is short (which is 2 bytes in size). When you add 1 to a pointer it advances the size of the type (sizeof(short) = 2) in bytes.

Oops, my mistake, the code I wrote I had the pointers as char*, didn't realise you'd declared them as short*. Sorry :oops:
She who travels light - forgot something.
User avatar
Posts: 99
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England
by yo1dog » Fri May 30, 2014 4:32 pm
Sorry to dig up an old post, but I figured I should post my solution.

Get a file descriptor to fb1.
Code: Select all
fbfd = open("/dev/fb1", O_RDWR);


I then map some memory to that file descriptor.
Code: Select all
fbp = mmap(0, screen_datasize, PROT_READ | PROT_WRITE, MAP_SHARED, fbfd, 0);


Finally I take a snapshot, and copy the section of the pixel data I need into to the mapped memory
Code: Select all
 
vc_dispmanx_snapshot(display, screen_resource, 0);
vc_dispmanx_resource_read_data(screen_resource, &rect, data, screen_width * screen_bitdepth);

int y;
for (y = 0; y < screen_height; y++)
    memcpy(
        fbp + y * screen_width,
        data + y * orig_screen_width,
        screen_width * screen_bitdepth);


This still drops the frame-rate by 10-20 fps and under my targeted 60 fps, unfortunately. Anyone see any improvements that can be made?

Here is my full source. It helps if you diff the code with the original to see the added and changed lines. Make sure to ignore whitespace changes.
Code: Select all
/*
Copyright (c) 2012, Broadcom Europe Ltd
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
    * Redistributions of source code must retain the above copyright
      notice, this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.
    * Neither the name of the copyright holder nor the
      names of its contributors may be used to endorse or promote products
      derived from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/

// A rotating cube rendered with OpenGL|ES. Three images used as textures on the cube faces.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <assert.h>
#include <unistd.h>

#include "bcm_host.h"

#include "GLES/gl.h"
#include "EGL/egl.h"
#include "EGL/eglext.h"

#include <sys/mman.h>
#include <fcntl.h>

#include "cube_texture_and_coords.h"

#define PATH "./"

#define IMAGE_SIZE 128

#ifndef M_PI
   #define M_PI 3.141592654
#endif

typedef struct
{
   uint32_t screen_width;
   uint32_t screen_height;
// OpenGL|ES objects
   EGLDisplay display;
   EGLSurface surface;
   EGLContext context;
   GLuint tex[6];
// model rotation vector and direction
   GLfloat rot_angle_x_inc;
   GLfloat rot_angle_y_inc;
   GLfloat rot_angle_z_inc;
// current model rotation angles
   GLfloat rot_angle_x;
   GLfloat rot_angle_y;
   GLfloat rot_angle_z;
// current distance from camera
   GLfloat distance;
   GLfloat distance_inc;
// pointers to texture buffers
   char *tex_buf1;
   char *tex_buf2;
   char *tex_buf3;
} CUBE_STATE_T;

static void init_ogl(CUBE_STATE_T *state);
static void init_model_proj(CUBE_STATE_T *state);
static void reset_model(CUBE_STATE_T *state);
static GLfloat inc_and_wrap_angle(GLfloat angle, GLfloat angle_inc);
static GLfloat inc_and_clip_distance(GLfloat distance, GLfloat distance_inc);
static void redraw_scene(CUBE_STATE_T *state);
static void update_model(CUBE_STATE_T *state);
static void init_textures(CUBE_STATE_T *state);
static void load_tex_images(CUBE_STATE_T *state);
static void exit_func(void);
static volatile int terminate;
static CUBE_STATE_T _state, *state=&_state;

static int screen_width      = 480;
static int screen_height   = 272;
static int screen_bitdepth = 2;
static int screen_datasize;

static DISPMANX_DISPLAY_HANDLE_T display;
static DISPMANX_RESOURCE_HANDLE_T screen_resource;
static int fbfd = 0;
static unsigned short *fbp = 0;
static unsigned short *data = 0;
static int orig_screen_width = 0;
static int orig_screen_height = 0;

static VC_RECT_T rect;

/***********************************************************
 * Name: init_ogl
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description: Sets the display, OpenGL|ES context and screen stuff
 *
 * Returns: void
 *
 ***********************************************************/
static void init_ogl(CUBE_STATE_T *state)
{
   screen_datasize = screen_width * screen_height * screen_bitdepth;

   display = vc_dispmanx_display_open(0);
   if (!display)
   {
      printf("Error opening display!\n");
      exit(1);
   }

   fbfd = open("/dev/fb1", O_RDWR);
   if (!fbfd)
   {
      printf("Error opening file!\n");
      exit(1);
   }

   fbp = mmap(0, screen_datasize, PROT_READ | PROT_WRITE, MAP_SHARED, fbfd, 0);
   if ((int)fbp == -1)
   {
      printf("Error mapping!\n");
      exit(1);
   }

   int32_t success = 0;
   EGLBoolean result;
   EGLint num_config;

   static EGL_DISPMANX_WINDOW_T nativewindow;

   DISPMANX_ELEMENT_HANDLE_T dispman_element;
   DISPMANX_DISPLAY_HANDLE_T dispman_display;
   DISPMANX_UPDATE_HANDLE_T dispman_update;
   VC_RECT_T dst_rect;
   VC_RECT_T src_rect;

   static const EGLint attribute_list[] =
   {
      EGL_RED_SIZE, 8,
      EGL_GREEN_SIZE, 8,
      EGL_BLUE_SIZE, 8,
      EGL_ALPHA_SIZE, 8,
      EGL_SURFACE_TYPE, EGL_WINDOW_BIT,
      EGL_NONE
   };

   EGLConfig config;

   // get an EGL display connection
   state->display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
   assert(state->display!=EGL_NO_DISPLAY);

   // initialize the EGL display connection
   result = eglInitialize(state->display, NULL, NULL);
   assert(EGL_FALSE != result);

   // get an appropriate EGL frame buffer configuration
   result = eglChooseConfig(state->display, attribute_list, &config, 1, &num_config);
   assert(EGL_FALSE != result);

   // create an EGL rendering context
   state->context = eglCreateContext(state->display, config, EGL_NO_CONTEXT, NULL);
   assert(state->context!=EGL_NO_CONTEXT);

   // create an EGL window surface
   success = graphics_get_display_size(0 /* LCD */, &state->screen_width, &state->screen_height);
   assert( success >= 0 );

   vc_dispmanx_rect_set(&rect, 0, 0, state->screen_width, state->screen_height);
   data = malloc(state->screen_width * state->screen_height * screen_bitdepth);
   
   uint32_t image_prt;
   screen_resource = vc_dispmanx_resource_create(VC_IMAGE_RGB565, state->screen_width, state->screen_height, &image_prt);
   if (!screen_resource)
   {
      printf("Error getting resource!\n");
      exit(1);
   }
   
   orig_screen_width = state->screen_width;
   orig_screen_height = state->screen_height;
   
   state->screen_width = 480;
   state->screen_height = 272;

   dst_rect.x = 0;
   dst_rect.y = 0;
   dst_rect.width = state->screen_width;
   dst_rect.height = state->screen_height;

   src_rect.x = 0;
   src_rect.y = 0;
   src_rect.width = state->screen_width << 16;
   src_rect.height = state->screen_height << 16;

   dispman_display = display;//vc_dispmanx_display_open( 0 /* LCD */);
   dispman_update = vc_dispmanx_update_start( 0 );

   dispman_element = vc_dispmanx_element_add ( dispman_update, dispman_display,
      0/*layer*/, &dst_rect, 0/*src*/,
      &src_rect, DISPMANX_PROTECTION_NONE, 0 /*alpha*/, 0/*clamp*/, 0/*transform*/);

   nativewindow.element = dispman_element;
   nativewindow.width = state->screen_width;
   nativewindow.height = state->screen_height;
   vc_dispmanx_update_submit_sync( dispman_update );

   state->surface = eglCreateWindowSurface( state->display, config, &nativewindow, NULL );
   assert(state->surface != EGL_NO_SURFACE);

   // connect the context to the surface
   result = eglMakeCurrent(state->display, state->surface, state->surface, state->context);
   assert(EGL_FALSE != result);

   // Set background color and clear buffers
   glClearColor(0.15f, 0.25f, 0.35f, 1.0f);

   // Enable back face culling.
   glEnable(GL_CULL_FACE);

   glMatrixMode(GL_MODELVIEW);
}

/***********************************************************
 * Name: init_model_proj
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description: Sets the OpenGL|ES model to default values
 *
 * Returns: void
 *
 ***********************************************************/
static void init_model_proj(CUBE_STATE_T *state)
{
   float nearp = 1.0f;
   float farp = 500.0f;
   float hht;
   float hwd;

   glHint( GL_PERSPECTIVE_CORRECTION_HINT, GL_NICEST );

   glViewport(0, 0, (GLsizei)state->screen_width, (GLsizei)state->screen_height);

   glMatrixMode(GL_PROJECTION);
   glLoadIdentity();

   hht = nearp * (float)tan(45.0 / 2.0 / 180.0 * M_PI);
   hwd = hht * (float)state->screen_width / (float)state->screen_height;

   glFrustumf(-hwd, hwd, -hht, hht, nearp, farp);

   glEnableClientState( GL_VERTEX_ARRAY );
   glVertexPointer( 3, GL_BYTE, 0, quadx );

   reset_model(state);
}

/***********************************************************
 * Name: reset_model
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description: Resets the Model projection and rotation direction
 *
 * Returns: void
 *
 ***********************************************************/
static void reset_model(CUBE_STATE_T *state)
{
   // reset model position
   glMatrixMode(GL_MODELVIEW);
   glLoadIdentity();
   glTranslatef(0.f, 0.f, -50.f);

   // reset model rotation
   state->rot_angle_x = 45.f; state->rot_angle_y = 30.f; state->rot_angle_z = 0.f;
   state->rot_angle_x_inc = 0.5f; state->rot_angle_y_inc = 0.5f; state->rot_angle_z_inc = 0.f;
   state->distance = 40.f;
}

/***********************************************************
 * Name: update_model
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description: Updates model projection to current position/rotation
 *
 * Returns: void
 *
 ***********************************************************/
static void update_model(CUBE_STATE_T *state)
{
   // update position
   state->rot_angle_x = inc_and_wrap_angle(state->rot_angle_x, state->rot_angle_x_inc);
   state->rot_angle_y = inc_and_wrap_angle(state->rot_angle_y, state->rot_angle_y_inc);
   state->rot_angle_z = inc_and_wrap_angle(state->rot_angle_z, state->rot_angle_z_inc);
   state->distance    = inc_and_clip_distance(state->distance, state->distance_inc);

   glLoadIdentity();
   // move camera back to see the cube
   glTranslatef(0.f, 0.f, -state->distance);

   // Rotate model to new position
   glRotatef(state->rot_angle_x, 1.f, 0.f, 0.f);
   glRotatef(state->rot_angle_y, 0.f, 1.f, 0.f);
   glRotatef(state->rot_angle_z, 0.f, 0.f, 1.f);
}

/***********************************************************
 * Name: inc_and_wrap_angle
 *
 * Arguments:
 *       GLfloat angle     current angle
 *       GLfloat angle_inc angle increment
 *
 * Description:   Increments or decrements angle by angle_inc degrees
 *                Wraps to 0 at 360 deg.
 *
 * Returns: new value of angle
 *
 ***********************************************************/
static GLfloat inc_and_wrap_angle(GLfloat angle, GLfloat angle_inc)
{
   angle += angle_inc;

   if (angle >= 360.0)
      angle -= 360.f;
   else if (angle <=0)
      angle += 360.f;

   return angle;
}

/***********************************************************
 * Name: inc_and_clip_distance
 *
 * Arguments:
 *       GLfloat distance     current distance
 *       GLfloat distance_inc distance increment
 *
 * Description:   Increments or decrements distance by distance_inc units
 *                Clips to range
 *
 * Returns: new value of angle
 *
 ***********************************************************/
static GLfloat inc_and_clip_distance(GLfloat distance, GLfloat distance_inc)
{
   distance += distance_inc;

   if (distance >= 120.0f)
      distance = 120.f;
   else if (distance <= 40.0f)
      distance = 40.0f;

   return distance;
}

/***********************************************************
 * Name: redraw_scene
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description:   Draws the model and calls eglSwapBuffers
 *                to render to screen
 *
 * Returns: void
 *
 ***********************************************************/
static void redraw_scene(CUBE_STATE_T *state)
{
   // Start with a clear screen
   glClear( GL_COLOR_BUFFER_BIT );

   // Draw first (front) face:
   // Bind texture surface to current vertices
   glBindTexture(GL_TEXTURE_2D, state->tex[0]);

   // Need to rotate textures - do this by rotating each cube face
   glRotatef(270.f, 0.f, 0.f, 1.f ); // front face normal along z axis

   // draw first 4 vertices
   glDrawArrays( GL_TRIANGLE_STRIP, 0, 4);

   // same pattern for other 5 faces - rotation chosen to make image orientation 'nice'
   glBindTexture(GL_TEXTURE_2D, state->tex[1]);
   glRotatef(90.f, 0.f, 0.f, 1.f ); // back face normal along z axis
   glDrawArrays( GL_TRIANGLE_STRIP, 4, 4);

   glBindTexture(GL_TEXTURE_2D, state->tex[2]);
   glRotatef(90.f, 1.f, 0.f, 0.f ); // left face normal along x axis
   glDrawArrays( GL_TRIANGLE_STRIP, 8, 4);

   glBindTexture(GL_TEXTURE_2D, state->tex[3]);
   glRotatef(90.f, 1.f, 0.f, 0.f ); // right face normal along x axis
   glDrawArrays( GL_TRIANGLE_STRIP, 12, 4);

   glBindTexture(GL_TEXTURE_2D, state->tex[4]);
   glRotatef(270.f, 0.f, 1.f, 0.f ); // top face normal along y axis
   glDrawArrays( GL_TRIANGLE_STRIP, 16, 4);

   glBindTexture(GL_TEXTURE_2D, state->tex[5]);
   glRotatef(90.f, 0.f, 1.f, 0.f ); // bottom face normal along y axis
   glDrawArrays( GL_TRIANGLE_STRIP, 20, 4);

   eglSwapBuffers(state->display, state->surface);
}

/***********************************************************
 * Name: init_textures
 *
 * Arguments:
 *       CUBE_STATE_T *state - holds OGLES model info
 *
 * Description:   Initialise OGL|ES texture surfaces to use image
 *                buffers
 *
 * Returns: void
 *
 ***********************************************************/
static void init_textures(CUBE_STATE_T *state)
{
   // load three texture buffers but use them on six OGL|ES texture surfaces
   load_tex_images(state);
   glGenTextures(6, &state->tex[0]);

   // setup first texture
   glBindTexture(GL_TEXTURE_2D, state->tex[0]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf1);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   // setup second texture - reuse first image
   glBindTexture(GL_TEXTURE_2D, state->tex[1]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf1);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   // third texture
   glBindTexture(GL_TEXTURE_2D, state->tex[2]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf2);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   // fourth texture  - reuse second image
   glBindTexture(GL_TEXTURE_2D, state->tex[3]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf2);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   //fifth texture
   glBindTexture(GL_TEXTURE_2D, state->tex[4]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf3);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   // sixth texture  - reuse third image
   glBindTexture(GL_TEXTURE_2D, state->tex[5]);
   glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, IMAGE_SIZE, IMAGE_SIZE, 0,
                GL_RGB, GL_UNSIGNED_BYTE, state->tex_buf3);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, (GLfloat)GL_NEAREST);
   glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, (GLfloat)GL_NEAREST);

   // setup overall texture environment
   glTexCoordPointer(2, GL_FLOAT, 0, texCoords);
   glEnableClientState(GL_TEXTURE_COORD_ARRAY);

   glEnable(GL_TEXTURE_2D);
}

/***********************************************************
 * Name: load_tex_images
 *
 * Arguments:
 *       void
 *
 * Description: Loads three raw images to use as textures on faces
 *
 * Returns: void
 *
 ***********************************************************/
static void load_tex_images(CUBE_STATE_T *state)
{
   FILE *tex_file1 = NULL, *tex_file2=NULL, *tex_file3 = NULL;
   int bytes_read, image_sz = IMAGE_SIZE*IMAGE_SIZE*3;

   state->tex_buf1 = malloc(image_sz);
   state->tex_buf2 = malloc(image_sz);
   state->tex_buf3 = malloc(image_sz);

   tex_file1 = fopen(PATH "Lucca_128_128.raw", "rb");
   if (tex_file1 && state->tex_buf1)
   {
      bytes_read=fread(state->tex_buf1, 1, image_sz, tex_file1);
      assert(bytes_read == image_sz);  // some problem with file?
      fclose(tex_file1);
   }

   tex_file2 = fopen(PATH "Djenne_128_128.raw", "rb");
   if (tex_file2 && state->tex_buf2)
   {
      bytes_read=fread(state->tex_buf2, 1, image_sz, tex_file2);
      assert(bytes_read == image_sz);  // some problem with file?
      fclose(tex_file2);
   }

   tex_file3 = fopen(PATH "Gaudi_128_128.raw", "rb");
   if (tex_file3 && state->tex_buf3)
   {
      bytes_read=fread(state->tex_buf3, 1, image_sz, tex_file3);
      assert(bytes_read == image_sz);  // some problem with file?
      fclose(tex_file3);
   }
}

//------------------------------------------------------------------------------

static void exit_func(void)
// Function to be passed to atexit().
{
   // clear screen
   glClear( GL_COLOR_BUFFER_BIT );
   eglSwapBuffers(state->display, state->surface);

   // Release OpenGL resources
   eglMakeCurrent( state->display, EGL_NO_SURFACE, EGL_NO_SURFACE, EGL_NO_CONTEXT );
   eglDestroySurface( state->display, state->surface );
   eglDestroyContext( state->display, state->context );
   eglTerminate( state->display );

   // release texture buffers
   free(state->tex_buf1);
   free(state->tex_buf2);
   free(state->tex_buf3);

   free(data);
   munmap(fbp, screen_datasize);
   close(fbfd);
   vc_dispmanx_resource_delete(screen_resource);
   vc_dispmanx_display_close(display);

   printf("\ncube closed\n");
} // exit_func()

//==============================================================================

int main ()
{
   bcm_host_init();

   // Clear application state
   memset( state, 0, sizeof( *state ) );

   // Start OGLES
   init_ogl(state);

   // Setup the model world
   init_model_proj(state);

   // initialise the OGLES texture(s)
   init_textures(state);

   time_t start = time(NULL);
   int numFrames = 0;
   while (!terminate)
   {
      update_model(state);
      redraw_scene(state);

      vc_dispmanx_snapshot(display, screen_resource, 0);
      vc_dispmanx_resource_read_data(screen_resource, &rect, data, screen_width * screen_bitdepth);

      int y;
      for (y = 0; y < screen_height; y++)
         memcpy(
            fbp + y * screen_width,
            data + y * orig_screen_width,
            screen_width * screen_bitdepth);
            
      numFrames ++;
      
      if (numFrames == 1000)
       break;
   }
   
   int elapsed = (int)(time(NULL) - start);
   printf("%i\n", elapsed);
   printf("%i\n", numFrames);
   printf("%i\n", numFrames / elapsed);
   
   exit_func();
   return 0;
}
Last edited by yo1dog on Sun Jun 01, 2014 2:23 am, edited 2 times in total.
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by AndyD » Fri May 30, 2014 11:41 pm
yo1dog wrote:Sorry to dig up an old post, but I figured I should post my solution...

As mentioned by texy above, I believe a number of people are using rpi-fbcp which uses the same approach.
Posts: 954
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia
by yo1dog » Sun Jun 01, 2014 2:22 am
I haven't worked on it in a while, but I tried that approach. I don't remember why I decided to go a different direction. Maybe there was a delay or the framerate was too low. Maybe I just wanted a solution that didn't require an external process.

Either way, I hope this helps.

Also, it helps if you diff the above code with the original to see the added and changed lines. Make sure to ignore whitespace changes.
Posts: 13
Joined: Tue Oct 22, 2013 11:18 pm
by AndyD » Sun Jun 01, 2014 3:50 am
Yes, sorry I should have thought about that a bit more. Your approach is different in that you have a single application creating the output and sending it to /dev/fb1. That makes a lot of sense!
Posts: 954
Joined: Sat Jan 21, 2012 8:13 am
Location: Melbourne, Australia