Official Touch Screen and Xorg

[rotwang]

I want both the touch screen AND the hdmi output active simultaneously. This is so that I can run an Impress presentation with the HDMI connected to a projector, and the Impress control console running on the touchscreen.

I understand this will involve creating a new framebuffer device. Unfortunately pressure of work commitments means I will have very limited time to work on this. Does anyone want to pick up the ball and run with this one. I will have time to run tests if that helps.
So Wishlist is :-
Xorg can use either or both displays.
Should be able to make use of GPU acceleration (when that arrives)


And by the way, if you have an Adafruit-style laser cut acrylic case, then leaving out the bottom plate will allow you to assemble it around a raspberry pi mounted to the screen.

[klricks]

How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/

[rotwang]

How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/

Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.


{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)

[alexeames]

{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)

Your wish has been granted. Clive created one this morning and we are scooping up topics and moving them as we find them

[fruitoftheloom]

How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/

Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.


{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)

Dual display usage

It is possible to use both display outputs at the same time, but it does require software to choose the right display. Omxplayer is one application that has been modified to enable secondary display output.

[rotwang]

How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/

Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.

>>

>>Dual display usage

>>It is possible to use both display outputs at the same time, but it does require software to choose the right display. Omxplayer >>is one application that has been modified to enable secondary display output.

Once again, somebody isn't reading the question -- to make it absolutely clear, I need to acheive the following setup
Running under X, a dual-head display with DSI and HDMI active simultaneously and with independent content.
HDMI output will be going to a projector, while the touchscreen will be running the presenter console.

[gsh]

Currently you can't do that because it requires a second /dev/fb implementation for the touchscreen and one doesn't yet exist.

If you can understand how the fbdev currently works and can easily provide a second one then it would be possible with a small amount of input from Dom and I to pass a second framebuffer across from the GPU

Gordon

[rotwang]

Currently you can't do that because it requires a second /dev/fb implementation for the touchscreen and one doesn't yet exist.

If you can understand how the fbdev currently works and can easily provide a second one then it would be possible with a small amount of input from Dom and I to pass a second framebuffer across from the GPU

Gordon

A sensible reply at last!
I assume this will involve compiling kernel modules, which is tedious given the lack of a decent cross-compiler setup, but I'm up for it. How should we organise this for least wasted effort in case anybody else has the same requirement?
Roger

[rotwang]

Bump

[ChT]

I have a similar requirement as the thread opener:
I need a touchscreen running a browser for user input, and a big screen running a browser without user input as a large display. The touchscreen is used to score a game (count points), the big screen to display the score to the public. I managed to do this with the touchscreen from Mimo: the Displaylink driver adds another /dev/fb device which I can use for Xorg.

Now with the official touchscreen I have a nearly perfect setup but I can run Xorg on either touchscreen or HDMI only.
I had a look at fbtft, which is used for other, smaller, touchscreens, but it looks it doesn't support the official touchscreen yet.
Is it a restriction in the hardware that we can have Xorg on either DSI or HDMI only?

I can run omxplayer in parallel to Xorg, so basically it would be possible to display something on the HDMI in parallel to have Xorg on the touchscreen. Does anyone know a browser which runs on top of dispmanx (my understanding is that's the way omxplayer works)? Are there plans that epiphany may support this in the future?

Christoph

[rotwang]

Bump

Progress so far :-
Setup buildroot on desktop m/c -- this appears to be the easiest way to get a cross-compiler setup that works. I can now build a kernel in under 10mins, possibly even faster once I reconfigure to remove unwanted drivers. I get kernels that will boot at least.

Next phase -- cursory inspection shows that startup.elf is doing a hell of a lot more than just loading the kernel and starting it running, so the next question is what is doing the deciding as to which display to use, HDMI or DSI. This seems to be happening before any of my code gets a chance to run. This becomes even more important when considering the compute module where the choices are much wider. And of course there's the gpu-accelerated X driver heaving into view.

[gsh]

I believe all you need to do first is to create a second fbdev driver. This can then be used for the second X windows display (I'm guessing a lot here).

The firmware sets up the HDMI and LCD displays and the primary display gets a frame buffer set up. The details of this buffer is read from the GPU through the mailbox interface when the fbdev is being initialised, check out the device-tree contents to see which drivers access the firmware driver. So what we really need is a second fbdev to load up the second frame buffer from the firmware...

This will require that we add the second buffer information to the firmware, but that sounds quite easy in comparison to everything else!

Once a second fbdev has been created I _think_ X windows will use it automatically (or it may require extra setup)

Gordon

[tvjon]

So presumably, as the main difference is that the LCD uses a DSI, whereas Gert's VGA666 is a DPI, your proposed technique would work with that too?

[rotwang]

I thought this might be a can of worms, instead of which it's turning into a cauldron full of serpents. I did some more investigating (most of which was wasted effort due to a dodgy HDMI lead) but further tests this morning confirm that the display which is going to be the console is chosen by either the ROM bootloader in the broadcom chip or bootcode.bin whichever generates the big rainbow square. This happens long before the kernel gets control. If I have both the touch screen and an HDMI display connected at poweron then the big rainbow will only appear on the touchscreen which will become the console.
The HDMI output is being initialised (because the monitor shows a black screen, rather than a "no signal" message).
The upshot of all this is that when the kernel gets control, it can only see one video device, which the GPU has chosen already.
To conform to the general arrangement, the GPU needs to advise the kernel of how many video devices it has dicovered (this also has to work with the compute module, which may have two DSI displays plus HDMI). The kernel then needs to iterate through this list, creating frame buffers and corresponding /dev/fb<n> devices and establishing the link between framebuffer memory ares and the GPU.
Well, thats the thought for today, anything to do with firmware I can obviously do nothing about, but I can make time for testing things if/when anyone has anything to test.
<postscript>
Xorg should take care of itself, given an xorg.conf which tells it which framebuffer devices to use.

[rotwang]

I thought this might be a can of worms, instead of which it's turning into a cauldron full of serpents. I did some more investigating (most of which was wasted effort due to a dodgy HDMI lead) but further tests this morning confirm that the display which is going to be the console is chosen by either the ROM bootloader in the broadcom chip or bootcode.bin whichever generates the big rainbow square. This happens long before the kernel gets control. If I have both the touch screen and an HDMI display connected at poweron then the big rainbow will only appear on the touchscreen which will become the console.
The HDMI output is being initialised (because the monitor shows a black screen, rather than a "no signal" message).
The upshot of all this is that when the kernel gets control, it can only see one video device, which the GPU has chosen already.
To conform to the general arrangement, the GPU needs to advise the kernel of how many video devices it has dicovered (this also has to work with the compute module, which may have two DSI displays plus HDMI). The kernel then needs to iterate through this list, creating frame buffers and corresponding /dev/fb<n> devices and establishing the link between framebuffer memory ares and the GPU.
Well, thats the thought for today, anything to do with firmware I can obviously do nothing about, but I can make time for testing things if/when anyone has anything to test.
<postscript>
Xorg should take care of itself, given an xorg.conf which tells it which framebuffer devices to use.

What's the next step up from a cauldron of serpents, an ocean basin full of kraaken?
It certainly looks to be the case that the firmware sets up a SINGLE video device, DSI takes priority over HDMI and if all else fails composite. This is all done well before the kernel is started, all that the kernel appears to do is to take the one device it has been given, it appears to have no knowledge of any other.
To confuse me even further, invoking "sudo Xorg -configure" from the CLI which SHOULD write out an xorg.conf file based on the hardware it is running on, proceeds to load fbdev and fbturbo, then exits saying "no devices to configure". In order to even begin to produce a framebuffer device for the touchscreen, you need to know it's there, which the current firmware seems to determined not to admit to.
Still, onwards and upwards, I now know an increasing list of things it isn't.

[gsh]

There is no point in continuing this until someone has written the beginnings of the changes for the new fbdev which supports both displays. I was thinking that progress could be made in actually rewriting the fbdev driver to support more than one display even if there is only one (this is something that will have to be done separately to the modification of the firmware to respond to requests about the second display)

I would suggest awaiting further changes

Gordon

[rotwang]

There is no point in continuing this until someone has written the beginnings of the changes for the new fbdev which supports both displays. I was thinking that progress could be made in actually rewriting the fbdev driver to support more than one display even if there is only one (this is something that will have to be done separately to the modification of the firmware to respond to requests about the second display)

I would suggest awaiting further changes

Gordon

Thanks for taking the time to at least think about this, I'll just have to put that project back on the shelf for the moment, Do bear in mind that the compute module has TWO dsi connections, so at some point some awkward bugger like me is going to want to use both of them. In the mean time , if any one has a kernel+firmware that they want testing, do feel free to PM me.
Roger

[ssvb]

I believe all you need to do first is to create a second fbdev driver. This can then be used for the second X windows display (I'm guessing a lot here).

The firmware sets up the HDMI and LCD displays and the primary display gets a frame buffer set up. The details of this buffer is read from the GPU through the mailbox interface when the fbdev is being initialised, check out the device-tree contents to see which drivers access the firmware driver. So what we really need is a second fbdev to load up the second frame buffer from the firmware...

This will require that we add the second buffer information to the firmware, but that sounds quite easy in comparison to everything else!

Once a second fbdev has been created I _think_ X windows will use it automatically (or it may require extra setup)

The X server will not use the second /dev/fbX device automatically, but it will work with a simple xorg.conf tweak. Something like this: https://linux-sunxi.org/Dual_Monitor_Su ... figuration

[ChT]

I just want to move this topic up to the front again.
It would be nice to have X on both, touchscreen as well as HDMI.
As I understand, omxplayer is using dispmanx to display contents on the screen not used by X. So would it be possible to write an fbdev on top of dispmanx?


Christoph

[tbaumgartner]

Was there any progress on using DSI and HDMI output at the same time?

I would like to use two separate X servers running on the rpi. One X server presents a menu system on the DSI screen while the other X server shows the application started by the menu system on the HDMI output.

[gtrawoger]

I am also interested in this as well.

I hit the 'no frame buffer wall' for the second display. I was encouraged when I got omxplayer working on the second display (controlled with the touchscreen', output on HDMI). And I wanted to also use fbi to display images, but realized quickly that that was not working. After googling for hours I realized that this is where we are at: no frame buffer, no proper second display.

I am a noob at this (I know basic Linux admin, am a PHP developer) so I don't think I can develop too much. But I am highly motivated to work at this. I need to get this to work, since I already invested many hours and a bit of money to have this work.

Does anyone have any pointers to where to go next from here? Omxplayer has it figured out. How can we use what it uses?

[gtrawoger]

Ok. I have found some more information. The way I understand it, there is no way currently to get a second framebuffer until the firmware gets updated.

So, how could one use the second display (either HDMI or touchscreen) to display something (GUI or picture)? This thread seems to have found two options: viewtopic.php?f=108&t=129211&p=880255

One option is to use QT5 with EGLFS plugin compiled: viewtopic.php?p=863624#p863624
The other option is to use Kivy to create the GUI on the non-framebuffer device: viewtopic.php?p=883986#p883986

I will see if these options will work for my needs. Hopefully it might work for some of you as well.

[nachopro]

Hi Sorry for bump this thread but I want to know if exist a news about this theme.

Thanks!

[John Westlake]

Has there been any progress on implementing the second /dev/fb in firmware yet?

With the CM3 being pushed towards more professional applications - having a second Frame buffer would allow independent images on the HDMI (external) and DPI (internal) displays.

We would like to use two displays, one presents a menu system on (in our case) the internal DPI screen while the other display shows the application started by the menu system on the HDMI (external) output.

We would be happy to pay for such software development (for public release) - if it was a reasonable amount...

[ghans]

I think this feature will never be added to the old driver. All hopes rest on the work of Eric Anholt and his friends upstream .... So the global rollout of their efforts will come with the release of Raspbian 10 or 11 ?

In the meantime , one could switch to Arch or Gentoo and help Mr. Anholt with testing - DSI support has been added to the new driver some time ago

ghabs