GUI application (no console/window manager)

[pozz]

I'm new to RaspberryPI, maybe my questions will be dummy for you.

I need to develop a simple GUI application on RaspberryPI connected to a TFT display with touchpanel.

My concerns is how to avoid seeing console messages (booting, login, ...) and/or window manager, but only my GUI application.

In other words, the user will not have the possibility (through the display interface) to close my application, start another application, minimze it and so on.

Is it possible?

[topguy]

Yes...

It is not trivial to hide absolutely everything from the linux startup but blocking access to the console is pretty easy once your application starts.
And the script that starts your application should contain a loop so that it just restarts if it exits or crashes.

Come to think of it... depending on how you connect your TFT display and how the drivers work you might not see anything on that display to start with, everything is default shown on HDMI output and only your application will access the TFT display.

[pozz]

It is not trivial to hide absolutely everything from the linux startup but blocking access to the console is pretty easy once your application starts.
And the script that starts your application should contain a loop so that it just restarts if it exits or crashes.

I remember there is a file used during Linux initialization (inittab or something similar) that runs applications and automatically respawn them if they will crash.
Is inittab used in RPI Linux distribution? Can it be used to launch my GUI application at startup?

Come to think of it... depending on how you connect your TFT display and how the drivers work you might not see anything on that display to start with, everything is default shown on HDMI output and only your application will access the TFT display.

I'm going to use PiTFT Plus 2.8" or 3.5" boards. They are connected to the MPU through SPI bus.
Is it possible to redirect the console to HDMI port only? Is it possible to create a GUI application that draws to the display on the SPI bus, even if the main display output is on HDMI?

If yes, how to those things? Thank you.

[ghans]

Since Jessie inittab isn't in used by Raspbian anymore , it is all handled by systemd. You
should read up on it , it controls the startup sequence, software watchdog and more.

Actually it can be quite difficult to get a GPIO LCD display reconized as second screen. It is not plug-and-play.
So you are in full control of what is displayed on it as long the display handling is contained in your
application code , which is the default. Both screens will be independent , and it will be more difficult
to control (configure) the output on the HDMI screen because much more parties are using it (firmware/Linux kernel/systemd/getty).

ghans

[topguy]

Using systemd to start you application is the cleanest way of doing it, like ghans suggested.

From the guides at Adafruit regarding use/configuration of PiTFT screens it is clear that you can choose if you want console and desktop on the TFT screen or not. It also seems like you will get a new framebuffer device ( fb1 ) in addition to the standard (fb0) which is on the HDMI.

The next difficult choice you have is to choose a programming/GUI framework, that can use the new linux framebuffer as output, as the basis for your application. Which language(s) do you prefer ?

[pozz]

Since Jessie inittab isn't in used by Raspbian anymore , it is all handled by systemd. You
should read up on it , it controls the startup sequence, software watchdog and more.

As usual, is not simple to understand in some minutes. However at the moment that my request is possible with systemd. I'll study later on this.

[pozz]

From the guides at Adafruit regarding use/configuration of PiTFT screens it is clear that you can choose if you want console and desktop on the TFT screen or not. It also seems like you will get a new framebuffer device ( fb1 ) in addition to the standard (fb0) which is on the HDMI.

I didn't notice the new framebuffer device /dev/fb1.

[quote"topguy"]The next difficult choice you have is to choose a programming/GUI framework, that can use the new linux framebuffer as output, as the basis for your application. Which language(s) do you prefer ?[/quote]
Actually I don't have a preferred language for this. I will choose the best for my application and hw available. What do you suggest?

I read Adafruit suggests using pygame.Is pygame usueful only for games? My GUI application isn't a game. It will have only some texts, images/icons, buttons, and so on. No videos (maybe very simple animations).

[topguy]

Is pygame usueful only for games?

No, it can be used for whatever you want.

It probably has no inbuilt concept of what a "button" is, so you may have to draw it yourself and also check if that touch actually hits that button or not. ref: https://pythonprogramming.net/pygame-bu ... rectangle/

But general manipulation of text, images, icons and simple animations that is what PyGame is made for.

[pozz]

Is pygame usueful only for games?

No, it can be used for whatever you want.

It probably has no inbuilt concept of what a "button" is, so you may have to draw it yourself and also check if that touch actually hits that button or not. ref: https://pythonprogramming.net/pygame-bu ... rectangle/

But general manipulation of text, images, icons and simple animations that is what PyGame is made for.

Ok, but is there a better alternative? What do you suggest? Could python/tkinter be ok as pygame?

[topguy]

"better" depends a lot of what else this program is going to do other than showing a few buttons.
Also it depends on your expertise in different programming languages. Are we only discussing Python alternatives ?

tkinter needs X-windows, so then you would need to start a X-server on the framebuffer. And that works as shown by Adafruit. And you can skip the "desktop" and just start *your* application in fullscreen if you want.

[raysan5]

Not sure if it could work for you but if you want to code graphics in C without X-Windows, you can use raylib (http://www.raylib.com).

Source code: https://github.com/raysan5/raylib
Raspberry Pi usage instructions: https://github.com/raysan5/raylib/wiki/ ... spberry-Pi

Additionally, for simple tools, I also created raygui (https://github.com/raysan5/raygui), a single-file header-only IMGUI small library.

Here it is a sample of what you can do with raygui + raylib:

[pozz]

"better" depends a lot of what else this program is going to do other than showing a few buttons.

Nothing more. It should communicate with a slave device connected to one USB port (through /dev/ttyACM device file) and, depending on some data, show something on the display.
Maybe a web server will run on RPi, so the display will be use to configure Ethernet port (IP address, netmask, ...)

Also it depends on your expertise in different programming languages. Are we only discussing Python alternatives ?

I usually write firmware in C for small MCU and I have some previous knowledge of Visual Basic. I have also a little knowledge of non-GUI Python.
I would avoid C++ and Java, at least if there isn't a good reason to use those programming languages.
If python is ok, I would use it.

tkinter needs X-windows, so then you would need to start a X-server on the framebuffer. And that works as shown by Adafruit. And you can skip the "desktop" and just start *your* application in fullscreen if you want.

If it isn't strictly necessary, I think I can live well without X-windows. I think it means more memory size, more computational requirements (more complex to start my application in full-screen mode?)

The only pro of X-windows that I can see is it should give "more standard" tools, because it usually runs on desktop computers.

[topguy]

If raysan can confirm that raylib can work on a linux framebuffer and not only OpenGLES then I would maybe also suggest RayLib since its programmed in C.

There are others like tekUI and PicoGUI that seem to support framebuffer and are C oriented, but I have no experience with either.

For the least amount of code needed maybe a GUI framework made on top of PyGame is interesting: http://ocemp.sourceforge.net/guiwidget.html

Many possible solutions, you just have to decode which camels to swallow and start somewhere.

[Heater]

The is always the wonderful Qt GUI Toolkit.

With Qt5 you can render your GUI into the frame buffer. No X Windows required. You can uses accelerated OpenGL. You can develop your GUI on a speed PC with Linux or Windows. or on a Mac. Then move the source to the Pi and build it there.

[topguy]

You can uses accelerated OpenGL.

Not on a SPI connected small TFT screen me thinks.

[scott ellis]

You can using raspi2fb or something like it to send /dev/fb0 to /dev/fb1.

https://github.com/AndrewFromMelbourne/raspi2fb

You might think it can't be too efficient, but the GPU can do the scaling for you before copying to the TFT. So minimal ARM usage.

If you scale down the resolution of /dev/fb0 to only what you need for the TFT (config.txt), it's even easier. That also helps with calibrating the touchscreen coordinates and syncing with your GUI code.

Of course, the GUI still thinks it's running on /dev/fb0, and you can plug-in an HDMI display and see things on both screens, but then it gets pretty confusing dealing with touch events.

I am working on a project now using an Adafruit pitft35 display using Qt5.7 and QML using the eglfs platform plugin and it's working pretty nicely. The Adafruit pitft28 also works.

You don't need to use QML, Qt widget apps are fine.

I am using a custom Yocto built image not Raspbian, but the core idea transfers.

[Morphology]

If raysan can confirm that raylib can work on a linux framebuffer and not only OpenGLES then I would maybe also suggest RayLib since its programmed in C

I have just been through this exercise with Ray, and since Raylib relies on OpenGL ES and that doesn't support writing direct to the framebuffer (or if it does, it's far from straightforward), Raylib cannot be used to write to a TFT monitor that uses, say, FB1.

However, Ray's library does include a number of functions that can write text / images to a memory buffer, which could then be copied to fb1 (or the routines modified to copy to fb1).

I was hoping to be able to use hardware graphics acceleration to write to fb1 but, that doesn't appear to be possible (unless anybody knows differently??). So the best methods I have found so far (apart from re-directing X to use /dev/fb1) is -rst-'s excellent Raspberry Compote blog, which explores various methods for writing low level graphics direct to the framebuffer: http://raspberrycompote.blogspot.co.uk/

Morph

[pozz]

Thank you all for so many suggestions. It seems there are only a few valid solutions for making a small GUI application targeting a small TFT connected to the SPI without launching a full X-Window server.

Nobody suggest Gtk, even if it seems it is able to run on a simple framebuffer (even on /dev/fb1?). One question about Gtk: is there Gtk+3 for RPi, or only Gtk+ 2?

Just to summarize, please correct if I misunderstood. Notice I am filtering C/Python solutions that are able to draw directly on the framebuffer (no X-Windows).

• The simplest solution is Gtk on framebuffer /dev/fb1. I hope it is possible to redirect Gtk application framebuffer to /dev/fb1. Do you confirm? I'm only sorry if I have to learn an old GUI library (Gtk+2), because the latest library (Gtk+3) is not available for RPi.
  The second solution is pygame. Again, is it possible to redirect pygame to /dev/fb1 without X-Windows? I think it is possible (this is the suggestion from Adafruit for their PiTFT displays). The problem here is pygame is more oriented to games and not GUI, so you need to develop custom widgets and monitor events. Or use a GUI toolkit on top of pygame (Ocemp).
• Everything runs on /dev/fb0 (with or without hw acceleration). There are many possibilities depending on the language. The most important:
  C language: Gtk, raylib
  Python: pyGtk/pyGObject, pyQt5, pygame
  The real problem here is /dev/fb0 is the HDMI video port, but the target display for my GUI is the TFT on the SPI that is /dev/fb1.
  The solution is raspi2fb. The result shouldn't be slow.

[AndyD]

...I hope it is possible to redirect Gtk application framebuffer to /dev/fb1. Do you confirm?...

Have a look here Using GTK+ on the Framebuffer

You can set the environment variable GDK_DISPLAY to specify a different framebuffer device.

[pozz]

Have a look here Using GTK+ on the Framebuffer

You can set the environment variable GDK_DISPLAY to specify a different framebuffer device.

Interesting, thank you.

And what about Gtk+3 on framebuffer (GtkFB)? Is it avaiable for RPi?

[Morphology]

I haven't tried Gtk so I cannot answer your question, though this article seems to imply you can install Gtk+3:

http://www.raspberry-projects.com/pi/pr ... alling-gtk

The main problem currently with anything that writes directly to /dev/fb1 on the RPi is that you cannot benefit from hardware graphics acceleration, whereas using something that writes to /dev/fb0 you can. So using AndyD's raspi2fb to mirror /dev/fb0 to /dev/fb1 may be the way to go for me, personally.

What I had found with writing to /dev/fb1 using the sort of techniques outlined in -rst-'s low level framebuffer blog: http://raspberrycompote.blogspot.com is that updating a pixel via a pointer mapped to /dev/fb1 can seemingly be paused by some hardware-related process.

My app was attempting to update the screen in real time during callback functions, but I found it was sometimes failing to update the screen before the next time the callback function was called. The problem turned out to be in the following function:

Code: Select all

void put_pixel_RGB565(char *fbp, int x, int y, unsigned short c)
{
    unsigned int pix_offset = x * 2 + y * finfo.line_length;
    *((unsigned short*)(fbp + pix_offset)) = c;
}

All it is doing is calculating an offset into the framebuffer and updating that memory location with a 2-byte value.

If I commented out the second line *((unsigned short*)... etc the code ran absolutely fine, so the only conclusion was that something in the way that the Pi handles the /dev/fb1 memory was preventing this write happening in a timely manner, which wasn't something I was expecting.

I haven't tried it yet, but I am hoping that writing to /dev/fb0 using something that can make use of hardware graphics acceleration (raylib, for example) and then using raspi2fb to mirror /dev/fb0 to /dev/fb1 may overcome the issue as it would decouple the updating of the /dev/fb1 from my real-time process.

Morph

[pozz]

The main problem currently with anything that writes directly to /dev/fb1 on the RPi is that you cannot benefit from hardware graphics acceleration, whereas using something that writes to /dev/fb0 you can. So using AndyD's raspi2fb to mirror /dev/fb0 to /dev/fb1 may be the way to go for me, personally.

It seems to me very strange that the hardware acceleration on /dev/fb0 and raspi2fb benefit /dev/fb1 too. I can't understand why.
raspi2fb literally copies screenshots of main RPi display to framebuffer /dev/fb1, by using usual mmap() technique.

This is the code used in raspi2fb:

Code: Select all

        uint16_t *fbp = mmap(0,
                         finfo.smem_len,
                         PROT_READ | PROT_WRITE,
                         MAP_SHARED,
                         fbfd,
                         0);
        ....
        uint16_t *fbIter = fbp;
        uint16_t *frontCopyIter = frontCopyP;
        uint16_t *backCopyIter = backCopyP;

        uint32_t pixel;
        for (pixel = 0 ; pixel < pixels ; pixel++)
        {
            if (*frontCopyIter != *backCopyIter)
            {
                *fbIter = *frontCopyIter;
            }

            ++frontCopyIter;
            ++backCopyIter;
            ++fbIter;
        }

        uint16_t *tmp = backCopyP;
        backCopyP = frontCopyP;
        frontCopyP = tmp;

What I had found with writing to /dev/fb1 using the sort of techniques outlined in -rst-'s low level framebuffer blog: http://raspberrycompote.blogspot.com is that updating a pixel via a pointer mapped to /dev/fb1 can seemingly be paused by some hardware-related process.

My app was attempting to update the screen in real time during callback functions, but I found it was sometimes failing to update the screen before the next time the callback function was called.

The problem turned out to be in the following function:

Code: Select all

void put_pixel_RGB565(char *fbp, int x, int y, unsigned short c)
{
    unsigned int pix_offset = x * 2 + y * finfo.line_length;
    *((unsigned short*)(fbp + pix_offset)) = c;
}

All it is doing is calculating an offset into the framebuffer and updating that memory location with a 2-byte value.

If I commented out the second line *((unsigned short*)... etc the code ran absolutely fine, so the only conclusion was that something in the way that the Pi handles the /dev/fb1 memory was preventing this write happening in a timely manner, which wasn't something I was expecting.

I can't see any difference between this approach and raspi2fb. IMHO they are identical, with the overhead to update main display too. What is the frequency of your callback function that update the screen? Do you update all the pixels inside your callback functions? Maybe it is better to update only pixels that are actually changed.

I haven't tried it yet, but I am hoping that writing to /dev/fb0 using something that can make use of hardware graphics acceleration (raylib, for example) and then using raspi2fb to mirror /dev/fb0 to /dev/fb1 may overcome the issue as it would decouple the updating of the /dev/fb1 from my real-time process.

I don't know. As soon as I have my RPi and TFT, I will make some tests.

[Morphology]

yes, I would expect raspi2fb to behave exactly the same as my put_pixel() function.

The important differences for my app of using this 2-Phase approach would be 1) that my app can update the image in /dev/fb0 taking advantage of, say, OpenGL ES and so will benefit from hardware graphics acceleration and 2) if raspi2fb hiccups whilst copying fb0 to fb1 it can do so in its own thread, and won't interrupt my program whilst it does so.

My app was only re-drawing very small parts of the screen (say, 10 x 20 pixel blocks at a a time), but it was just trying to do so 50-100 times per second (ie faster than the screen refresh rate).

Morph

[AndyD]

...
I can't see any difference between this approach and raspi2fb. IMHO they are identical, with the overhead to update main display too. What is the frequency of your callback function that update the screen? Do you update all the pixels inside your callback functions? Maybe it is better to update only pixels that are actually changed.
...

I wrote raspi2fb as an experiment into improving the performance of the original fbcp. In my opinion, It was only partially successful. As you say @poz it simply copies the pixels to the framebuffer, with the the optimization that it only copies pixels that have changed.

I have written a few other programs for use with the SPI TFT displays on the Raspberry Pi, and I write directly to /dev/fb1. I am not sure why @Morphology is experiencing pauses writting to /dev/fb1, but I can't say I have seen the same issue.

[pozz]

...I hope it is possible to redirect Gtk application framebuffer to /dev/fb1. Do you confirm?...

Have a look here Using GTK+ on the Framebuffer

You can set the environment variable GDK_DISPLAY to specify a different framebuffer device.

I finally received my Raspberry Computer Module and PiTFT 2.8. pygame works well following tutorial on Adafruit.

I tested pyGobject settings GDK_DISPLAY to /dev/fb1, but I have the following error:

Code: Select all

Unable to init server: Could not connect: Connection refused

I think Gtk/Gdk libraries installed with "sudo apt-get install libgtk-3-dev" and "sudo apt-get install python3-gi" can't be used with a framebuffer.