Getting Irrlicht engine to run on GL ES 2.

[saldavonschwartz]

I apologize if this is not the right section in the forums to post this question. If so, feel free to move it to the right section.
I am posting in this section because I believe the root of my issue might be in EGL and X11 in the Pi, which I am not familiar with (I am familiar with GL programming on Windows, OS X and iOS though).

I am new to the Pi and also new to the Irrlicht engine (http://irrlicht.sourceforge.net). I've been trying to compile the engine to use it with GL ES 2 on a Pi 3 running Raspbian 8, unsuccessfully. In case it matters, I am trying to do this because my ultimate goal is to run Minetest on the Pi, which is built on top of Irrlicht. And I want to run it on GL ES 2 because the alternative, the experimental desktop GL 2.1 drivers for Pi, seem to perform worse than using GL ES 2.

What I tried so far:

1. Cloned the source from here https://github.com/zaki/irrlicht
2. Ran make on the ogl-es branch. Had to do the following changes for it to compile:
a- COGLESExtensionHandler.cpp and CEGLManager.h: change #include <GLES/egl.h> to #include <EGL/egl.h>
b- -I/opt/vc/include and -L/opt/vc/lib on the Makefile.

At this point I got the engine to compile into a static lib.
I then tried making and running example01 (loads a simple model: sydney.md2). By default the example renders with the software renderer (E_DRIVER_TYPE::EDT_SOFTWARE).
This worked: it created a new window and rendered the 3d model.

I then tried recompiling and running with E_DRIVER_TYPE::EDT_OGLES2. Does not work. Fails in CEGLManager.cpp:

Code: Select all

#if defined(_IRR_COMPILE_WITH_WINDOWS_DEVICE_)
	EglWindow = (NativeWindowType)Data.OpenGLWin32.HWnd;
	Data.OpenGLWin32.HDc = GetDC((HWND)EglWindow);
	EglDisplay = eglGetDisplay((NativeDisplayType)Data.OpenGLWin32.HDc);
#elif defined(_IRR_COMPILE_WITH_X11_DEVICE_)
	EglWindow = (NativeWindowType)Data.OpenGLLinux.X11Window;
	EglDisplay = eglGetDisplay((NativeDisplayType)Data.OpenGLLinux.X11Display);
#elif defined(_IRR_COMPILE_WITH_ANDROID_DEVICE_)
	EglWindow =	(ANativeWindow*)Data.OGLESAndroid.Window;
	EglDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY);
#elif defined(_IRR_COMPILE_WITH_FB_DEVICE_)
	EglWindow = (NativeWindowType)Data.OpenGLFB.Window;
	EglDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY);
#endif

	// We must check if EGL display is valid.
	if (EglDisplay == EGL_NO_DISPLAY)
    {
		os::Printer::log("Could not get EGL display.");
		terminate();
        return false;
    }

It specifically executes the code in the X11 elif and does not get a display back from eglGetDisplay.
I've been searching for a while online but have not found information on this. So any help would be greatly appreciated.

For comparison, I started doing some general tests of my own (without Irrlicht) and I was able to get a GL context going with EGL but without a window (eglGetDisplay(EGL_DEFAULT_DISPLAY)), so maybe this means I can't have GL ES 2 in an X11 window on Pi? Like I said, I do not have experience with it.

[Paeryn]

The stock OpenGLES/Egl doesn't interract with X11 on the RPi, instead it uses dispman (the RPi's native display manager), the linux framebuffer which X11 uses is really just a dispman window too.

There are examples of how to use Egl and dispman to get your OpenGLES window on screen in /opt/vc/src/hello_pi/* but note that it is totally separate from X11, X11 has no concept of the dispman windows.

[saldavonschwartz]

The stock OpenGLES/Egl doesn't interract with X11 on the RPi, instead it uses dispman (the RPi's native display manager), the linux framebuffer which X11 uses is really just a dispman window too.

There are examples of how to use Egl and dispman to get your OpenGLES window on screen in /opt/vc/src/hello_pi/* but note that it is totally separate from X11, X11 has no concept of the dispman windows.

Perfect! Ok that is what I needed to know. If you don't mind I have a few follow up questions.

1. if I want gLes2 inside a window on pi I need to use dispman correct?
2. if I want gles2 fullscreen, should I still use dispman or should I go straight to the frame buffer? Most gles examples Iv'e seen in my short 16hrs with the pi seem to just get a default display from egl and write to the frame buffer without a window.
3. If I use the experimental desktop gl 2.1 drivers, do I still use dispman or x11 or straight to the frame buffer?
4. If you are familiar at all with Minecraft Pi edition, which approach do you think it's using? I see it runs inside a window.

[Paeryn]

Perfect! Ok that is what I needed to know. If you don't mind I have a few follow up questions.

1. if I want gLes2 inside a window on pi I need to use dispman correct?

No, the default drivers don't draw into what X11 knows as windows, it draws onto a separate element that gets drawn in front of the whole X11 screen. As far as dispman is concerned the entire X11 screen (and all of its windows) is just one big dispman window. Your Egl window is separate and will be on another layer somewhere in front.

2. if I want gles2 fullscreen, should I still use dispman or should I go straight to the frame buffer? Most gles examples Iv'e seen in my short 16hrs with the pi seem to just get a default display from egl and write to the frame buffer without a window.

The standard broadcom Egl doesn't draw directly to the framebuffer, you have to go through dispman to get a dispman element for the Egl surface.
You create a dispman element for the window and add it to the screen and pass that to Egl as the native window to get your Egl surface. You could skip adding the dispman element to the display and manually copy the Egl surface (e.g. via a glReadPixels) into an X11 window or to the framebuffer yourself after each frame but it will slow your program down.

3. If I use the experimental desktop gl 2.1 drivers, do I still use dispman or x11 or straight to the frame buffer?

The experimental drivers use X11 as far as I'm aware, I don't know if it supports writing directly to the framebuffer, I've never tried it.

4. If you are familiar at all with Minecraft Pi edition, which approach do you think it's using? I see it runs inside a window.

I've never used Minecraft but I think it does it by pretending to be in an X11 window (moving the dispman window around to match where the X11 window would draw it), try moving another window to partially cover the minecraft window, if the minecraft window still appears on top then that is how they do it.

[saldavonschwartz]

Thanks again for all the info.

So, I don't have access to the minecraft code but I have access to the quake3 code, which is another popular game that uses gles on the pi (gles1.1, fixed pipeline, but still). And I was hoping you could give me some insight into what it might be doing interns of initializing the context and surface.

From the source (es_glimp.c) Quake3 appears to use EGL and SDL. And I'm not sure how these are fitting together. I see that the code first initializes SDL and then queries SDL_VideoDriverName. This returns x11. But then the code goes on to do the usual EGL initialization, context creation, surface creation and whatnot.

Particularly, the sequence of events is:

1. init SDL.
2. EGL: getDisplay (EGL_DEFAULT_DISPLAY)
3. EGL: initialize
4. EGL: chooseConfig
5. EGL: createContext
6. EGL: createWindowSurface.

When it creates the surface, it does it like so:

Code: Select all

g_EGLWindowSurface = eglCreateWindowSurface(g_EGLDisplay, g_EGLConfig, hWnd, NULL);

but hWnd is 0.

So how does it all fix together here? Quake3 is full screen and you can't even tab to another application while it is running. So it doesn't seem to be doing what you suggested minecraft does. But it's still using SDL. Is it creating an x11 window just to have input? or is it not creating a window? What do you make of it?

From the source code in /opt/vc a dispman element is a valid parameter to pass to eglCreateWindowSurface (for the native_window param). But the quake3 codebase doesn't have any dispman code and you said gles cannot work with x11.

[Paeryn]

Thanks again for all the info.

So, I don't have access to the minecraft code but I have access to the quake3 code, which is another popular game that uses gles on the pi (gles1.1, fixed pipeline, but still). And I was hoping you could give me some insight into what it might be doing interns of initializing the context and surface.

From the source (es_glimp.c) Quake3 appears to use EGL and SDL. And I'm not sure how these are fitting together. I see that the code first initializes SDL and then queries SDL_VideoDriverName. This returns x11. But then the code goes on to do the usual EGL initialization, context creation, surface creation and whatnot.

Particularly, the sequence of events is:

1. init SDL.
2. EGL: getDisplay (EGL_DEFAULT_DISPLAY)
3. EGL: initialize
4. EGL: chooseConfig
5. EGL: createContext
6. EGL: createWindowSurface.

When it creates the surface, it does it like so:

Code: Select all

g_EGLWindowSurface = eglCreateWindowSurface(g_EGLDisplay, g_EGLConfig, hWnd, NULL);

but hWnd is 0.

So how does it all fix together here? Quake3 is full screen and you can't even tab to another application while it is running. So it doesn't seem to be doing what you suggested minecraft does. But it's still using SDL. Is it creating an x11 window just to have input? or is it not creating a window? What do you make of it?

From the source code in /opt/vc a dispman element is a valid parameter to pass to eglCreateWindowSurface (for the native_window param). But the quake3 codebase doesn't have any dispman code and you said gles cannot work with x11.

Ahh, looks like you can pass NULL as the nativewindow to eglCreateWindowSurface(), in that case it seems to be automatically creating a fullscreen dispman window for you (just tried it on the hello_triangle demo), I never realised it would do that since all the example code explicitly creates a dispman window even for fullscreen. Quake3 will be using SDL to create an X11 window to handle capturing input as you said.

[jdonald]

I have this working as a proof-of-concept.

It seems many past attempts went at this by attempting a workaround for each crash after linking libbrcmEGL.so. My approach was instead inspired by the mention of Quake 3 here. I took the code from /opt/vc/src/hello_pi/hello_triangle/hello_triangle.c and inserted into Irrlicht's CEGLManager. While the setup of EglWindow, EglDisplay, EglContext, EglConfig, and EglSurface is a bit scattered in the Irrlicht code, triangle.c has one function that takes care of all of that in one go.

Once this was hooked up, initially I kept getting a black screen even before the engine started. It turns out that linking with libGL.so alone causes conflicts that break even the sample applications, so had to disable all OpenGL linkage in Irrlicht and the target application.

You can grab my repo and build it like so:

Code: Select all

# assuming build-essential, clang++, libegl1-mesa-dev, libgles2-mesa-dev, and other requirements installed
git clone --recursive --depth 1 -b ogl-es  https://github.com/jdonald/irrlicht.git
CC=clang CXX=clang++ make -j8 -C irrlicht/source/Irrlicht/

Clang isn't mandatory, but avoiding gcc is one practical way to prevent running out of RAM on a Pi 3. I'm then able to build Minetest thanks to celinnia's instructions from the Minetest forums, which in turn came from some Freeminer gist:

Code: Select all

# assuming cmake, libx11-dev, libjpeg-dev, libpng-dev, zlibg1-dev, libbz2-dev, libsqlite3-dev, libleveldb-dev and other requirements installed
# disable vc4-kms-v3d in /boot/config.txt and reboot if necessary
git clone --depth 1 -b stable-0.4 https://github.com/minetest/minetest.git; cd minetest
# edit src/CMakeLists.txt, comment out the line: find_package(OpenGL REQUIRED)
CC=clang CXX=clang++ cmake -DCMAKE_PREFIX_PATH=/usr/lib/arm-linux-gnueabihf -DENABLE_GLES=1 \
  -DRUN_IN_PLACE=TRUE -DIRRLICHT_SOURCE_DIR=$HOME/irrlicht/source/Irrlicht/ \
  -DIRRLICHT_LIBRARY=$HOME/irrlicht/lib/Linux/libIrrlicht.a -DIRRLICHT_INCLUDE_DIR=$HOME/irrlicht/include \
  -DBUILD_SERVER=NO -DCMAKE_CXX_FLAGS="-Wno-c++11-extensions" \
  -DCMAKE_EXE_LINKER_FLAGS="/opt/vc/lib/libbrcmEGL.so /opt/vc/lib/libbcm_host.so" -DEGL_INCLUDE_DIR=/opt/vc/include \
  -DEGL_LIBRARY=/opt/vc/lib/libbrcmEGL.so -DOPENGLES2_INCLUDE_DIR=/opt/vc/include \
  -DOPENGLES2_LIBRARY=/opt/vc/lib/libbrcmGLESv2.so
make -j8
# edit minetest.conf to set video_driver = ogles1, enable_shaders = 0
bin/minetest
# manually drag window to align with overlay so that mouseclicks will work, then play

Again clang is optional, but makes it practical to build and rebuild on the Pi without having to wait an hour. A cross-compilation environment would be even better, but things look complicated enough above already! Using RUN_IN_PLACE is also up to user preference. I've found it's handy to have a minetest.conf and saved worlds that don't interfere with the system-installed Minetest.

Once running, it became apparent that what initially appeared to be random artifacts were actually mountains behind the visible geometry. This led to a fix for DEPTH_SIZE. Also note that Minecraft Pi edition has a mechanism to ensure that its SDL window aligns with the dispmanx overlay, but with Minetest lacking anything similar it's up to the user to drag and align the window. Furthermore it would be nice to resolve crash-on-exit, but at least this gives developers a starting point!

Rendering performance is markedly better than the Mesa GLES driver at the time of writing. When log/chat text is disabled (press F2) as well as the HUD (press F1), 40+ fps is the norm on a Pi 3.

[anholt]

Were you comparing to Mesa using irrlicht's GLES2 support, or a normal build? The minetest rendering I get on debian is awful -- legacy GL calls that are swamping the CPU with work, which aren't available in GLES2. Minetest on debian is also using GLX instead of EGL, so you get a bunch of bandwidth overhead due to regrettable choices in the GLX spec which EGL fixed. If you're not testing EGL GLES2 vs EGL GLES2, it's really not a useful comparison.

[jdonald]

That's good to know about the issues with GLX.

The answer to your question is that yes this is an apples-to-apples comparison of Mesa EGL+GLES vs Broadcom proprietary driver EGL+GLES. One caveat: despite the title of this thread I'm testing video::EDT_OGLES1 (video_driver = ogles1 in minetest.conf), so this is actually GLES1, not GLES2. I went that route because it worked for celinnia on the Minetest forums. I expect there's more work to get video::EDT_OGLES2 working properly in Irrlicht.

On Pi Stack Exchange I posted a very rough Minetest three-way comparison of Mesa GLX+GL, Mesa EGL+GLES1, Broadcom driver EGL+GLES1. On a Pi 3 (or 2) with Raspbian Stretch (or Jessie) there doesn't seem to be such a noticeable slowdown with GLX.