Official Touch Screen and Xorg

The official Raspbery Pi touch display

22 posts
by rotwang » Sun Sep 13, 2015 1:23 pm
I want both the touch screen AND the hdmi output active simultaneously. This is so that I can run an Impress presentation with the HDMI connected to a projector, and the Impress control console running on the touchscreen.

I understand this will involve creating a new framebuffer device. Unfortunately pressure of work commitments means I will have very limited time to work on this. Does anyone want to pick up the ball and run with this one. I will have time to run tests if that helps.
So Wishlist is :-
Xorg can use either or both displays.
Should be able to make use of GPU acceleration (when that arrives)


And by the way, if you have an Adafruit-style laser cut acrylic case, then leaving out the bottom plate will allow you to assemble it around a raspberry pi mounted to the screen.
Last edited by rotwang on Tue Sep 22, 2015 11:14 am, edited 1 time in total.
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by klricks » Sun Sep 13, 2015 1:49 pm
How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/
Go here for my RPi writeup. Basic config, Serial Port add-on etc:
http://blackeagle12.net/Comp/RPi/Rpi.html Click web icon on right side --->
Posts: 4262
Joined: Sat Jan 12, 2013 3:01 am
Location: Grants Pass, OR, USA
by rotwang » Sun Sep 13, 2015 2:43 pm
klricks wrote:How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/


Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.


{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by alexeames » Mon Sep 14, 2015 5:00 pm
rotwang wrote:{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)


Your wish has been granted. Clive created one this morning and we are scooping up topics and moving them as we find them :)
Alex Eames RasPi.TV, RasP.iO
User avatar
Forum Moderator
Forum Moderator
Posts: 2806
Joined: Sat Mar 03, 2012 11:57 am
Location: UK
by fruitoftheloom » Mon Sep 14, 2015 5:16 pm
rotwang wrote:
klricks wrote:How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/


Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.


{Note to moderators - PLEASE can we have a new topic under peripherals specifically for the touch screen, at the moment any posts about the touchscreen are shot-gunned all over the place)

Dual display usage

It is possible to use both display outputs at the same time, but it does require software to choose the right display. Omxplayer is one application that has been modified to enable secondary display output.
.
Ex Computer Repair & Service Technician.
RPi 3B, HP Envy 4500 Wireless Printer, Google Chromecast, Android Smart Phone, HD 1080p TV and 3/4G Mobile Internet make ideal companions.
Posts: 13310
Joined: Tue Mar 25, 2014 12:40 pm
Location: Bognor Regis UK
by rotwang » Tue Sep 15, 2015 2:11 pm
fruitoftheloom wrote:
rotwang wrote:
klricks wrote:How to enable dual display output is explained here: https://www.raspberrypi.org/blog/the-ea ... i-display/


Actually it very much doesn't explain, at least as far as X11 is concerned, If it did I wouldn't have asked the question.

Read my original post, I need to run Libre-office Impress with the HDMI output AND the touchscreen both active simultaneously, HDMI output goes to projector, Presenter console runs on touchscreen.

>>
>>Dual display usage

>>It is possible to use both display outputs at the same time, but it does require software to choose the right display. Omxplayer >>is one application that has been modified to enable secondary display output.


Once again, somebody isn't reading the question -- to make it absolutely clear, I need to acheive the following setup
Running under X, a dual-head display with DSI and HDMI active simultaneously and with independent content.
HDMI output will be going to a projector, while the touchscreen will be running the presenter console.
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by gsh » Tue Sep 15, 2015 5:17 pm
Currently you can't do that because it requires a second /dev/fb implementation for the touchscreen and one doesn't yet exist.

If you can understand how the fbdev currently works and can easily provide a second one then it would be possible with a small amount of input from Dom and I to pass a second framebuffer across from the GPU

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1143
Joined: Sat Sep 10, 2011 11:43 am
by rotwang » Tue Sep 15, 2015 7:30 pm
gsh wrote:Currently you can't do that because it requires a second /dev/fb implementation for the touchscreen and one doesn't yet exist.

If you can understand how the fbdev currently works and can easily provide a second one then it would be possible with a small amount of input from Dom and I to pass a second framebuffer across from the GPU

Gordon


A sensible reply at last!
I assume this will involve compiling kernel modules, which is tedious given the lack of a decent cross-compiler setup, but I'm up for it. How should we organise this for least wasted effort in case anybody else has the same requirement?
Roger
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by rotwang » Tue Sep 22, 2015 11:15 am
Bump
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by ChT » Sat Sep 26, 2015 8:28 am
I have a similar requirement as the thread opener:
I need a touchscreen running a browser for user input, and a big screen running a browser without user input as a large display. The touchscreen is used to score a game (count points), the big screen to display the score to the public. I managed to do this with the touchscreen from Mimo: the Displaylink driver adds another /dev/fb device which I can use for Xorg.

Now with the official touchscreen I have a nearly perfect setup but I can run Xorg on either touchscreen or HDMI only.
I had a look at fbtft, which is used for other, smaller, touchscreens, but it looks it doesn't support the official touchscreen yet.
Is it a restriction in the hardware that we can have Xorg on either DSI or HDMI only?

I can run omxplayer in parallel to Xorg, so basically it would be possible to display something on the HDMI in parallel to have Xorg on the touchscreen. Does anyone know a browser which runs on top of dispmanx (my understanding is that's the way omxplayer works)? Are there plans that epiphany may support this in the future?

Christoph
Posts: 2
Joined: Sat Sep 26, 2015 8:10 am
by rotwang » Sat Sep 26, 2015 12:45 pm
rotwang wrote:Bump


Progress so far :-
Setup buildroot on desktop m/c -- this appears to be the easiest way to get a cross-compiler setup that works. I can now build a kernel in under 10mins, possibly even faster once I reconfigure to remove unwanted drivers. I get kernels that will boot at least.

Next phase -- cursory inspection shows that startup.elf is doing a hell of a lot more than just loading the kernel and starting it running, so the next question is what is doing the deciding as to which display to use, HDMI or DSI. This seems to be happening before any of my code gets a chance to run. This becomes even more important when considering the compute module where the choices are much wider. And of course there's the gpu-accelerated X driver heaving into view.
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by gsh » Sun Sep 27, 2015 1:22 pm
I believe all you need to do first is to create a second fbdev driver. This can then be used for the second X windows display (I'm guessing a lot here).

The firmware sets up the HDMI and LCD displays and the primary display gets a frame buffer set up. The details of this buffer is read from the GPU through the mailbox interface when the fbdev is being initialised, check out the device-tree contents to see which drivers access the firmware driver. So what we really need is a second fbdev to load up the second frame buffer from the firmware...

This will require that we add the second buffer information to the firmware, but that sounds quite easy in comparison to everything else!

Once a second fbdev has been created I _think_ X windows will use it automatically (or it may require extra setup)

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1143
Joined: Sat Sep 10, 2011 11:43 am
by tvjon » Sun Sep 27, 2015 2:19 pm
So presumably, as the main difference is that the LCD uses a DSI, whereas Gert's VGA666 is a DPI, your proposed technique would work with that too?
Posts: 508
Joined: Mon Jan 07, 2013 9:11 am
by rotwang » Wed Sep 30, 2015 11:43 am
I thought this might be a can of worms, instead of which it's turning into a cauldron full of serpents. I did some more investigating (most of which was wasted effort due to a dodgy HDMI lead) but further tests this morning confirm that the display which is going to be the console is chosen by either the ROM bootloader in the broadcom chip or bootcode.bin whichever generates the big rainbow square. This happens long before the kernel gets control. If I have both the touch screen and an HDMI display connected at poweron then the big rainbow will only appear on the touchscreen which will become the console.
The HDMI output is being initialised (because the monitor shows a black screen, rather than a "no signal" message).
The upshot of all this is that when the kernel gets control, it can only see one video device, which the GPU has chosen already.
To conform to the general arrangement, the GPU needs to advise the kernel of how many video devices it has dicovered (this also has to work with the compute module, which may have two DSI displays plus HDMI). The kernel then needs to iterate through this list, creating frame buffers and corresponding /dev/fb<n> devices and establishing the link between framebuffer memory ares and the GPU.
Well, thats the thought for today, anything to do with firmware I can obviously do nothing about, but I can make time for testing things if/when anyone has anything to test.
<postscript>
Xorg should take care of itself, given an xorg.conf which tells it which framebuffer devices to use.
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by rotwang » Sat Oct 03, 2015 5:44 pm
rotwang wrote:I thought this might be a can of worms, instead of which it's turning into a cauldron full of serpents. I did some more investigating (most of which was wasted effort due to a dodgy HDMI lead) but further tests this morning confirm that the display which is going to be the console is chosen by either the ROM bootloader in the broadcom chip or bootcode.bin whichever generates the big rainbow square. This happens long before the kernel gets control. If I have both the touch screen and an HDMI display connected at poweron then the big rainbow will only appear on the touchscreen which will become the console.
The HDMI output is being initialised (because the monitor shows a black screen, rather than a "no signal" message).
The upshot of all this is that when the kernel gets control, it can only see one video device, which the GPU has chosen already.
To conform to the general arrangement, the GPU needs to advise the kernel of how many video devices it has dicovered (this also has to work with the compute module, which may have two DSI displays plus HDMI). The kernel then needs to iterate through this list, creating frame buffers and corresponding /dev/fb<n> devices and establishing the link between framebuffer memory ares and the GPU.
Well, thats the thought for today, anything to do with firmware I can obviously do nothing about, but I can make time for testing things if/when anyone has anything to test.
<postscript>
Xorg should take care of itself, given an xorg.conf which tells it which framebuffer devices to use.


What's the next step up from a cauldron of serpents, an ocean basin full of kraaken?
It certainly looks to be the case that the firmware sets up a SINGLE video device, DSI takes priority over HDMI and if all else fails composite. This is all done well before the kernel is started, all that the kernel appears to do is to take the one device it has been given, it appears to have no knowledge of any other.
To confuse me even further, invoking "sudo Xorg -configure" from the CLI which SHOULD write out an xorg.conf file based on the hardware it is running on, proceeds to load fbdev and fbturbo, then exits saying "no devices to configure". In order to even begin to produce a framebuffer device for the touchscreen, you need to know it's there, which the current firmware seems to determined not to admit to.
Still, onwards and upwards, I now know an increasing list of things it isn't.
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by gsh » Mon Oct 05, 2015 9:14 am
There is no point in continuing this until someone has written the beginnings of the changes for the new fbdev which supports both displays. I was thinking that progress could be made in actually rewriting the fbdev driver to support more than one display even if there is only one (this is something that will have to be done separately to the modification of the firmware to respond to requests about the second display)

I would suggest awaiting further changes

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1143
Joined: Sat Sep 10, 2011 11:43 am
by rotwang » Mon Oct 05, 2015 11:32 am
gsh wrote:There is no point in continuing this until someone has written the beginnings of the changes for the new fbdev which supports both displays. I was thinking that progress could be made in actually rewriting the fbdev driver to support more than one display even if there is only one (this is something that will have to be done separately to the modification of the firmware to respond to requests about the second display)

I would suggest awaiting further changes

Gordon


Thanks for taking the time to at least think about this, I'll just have to put that project back on the shelf for the moment, Do bear in mind that the compute module has TWO dsi connections, so at some point some awkward bugger like me is going to want to use both of them. In the mean time , if any one has a kernel+firmware that they want testing, do feel free to PM me.
Roger
Posts: 216
Joined: Sat Dec 07, 2013 1:12 pm
by ssvb » Tue Oct 06, 2015 10:15 pm
gsh wrote:I believe all you need to do first is to create a second fbdev driver. This can then be used for the second X windows display (I'm guessing a lot here).

The firmware sets up the HDMI and LCD displays and the primary display gets a frame buffer set up. The details of this buffer is read from the GPU through the mailbox interface when the fbdev is being initialised, check out the device-tree contents to see which drivers access the firmware driver. So what we really need is a second fbdev to load up the second frame buffer from the firmware...

This will require that we add the second buffer information to the firmware, but that sounds quite easy in comparison to everything else!

Once a second fbdev has been created I _think_ X windows will use it automatically (or it may require extra setup)

The X server will not use the second /dev/fbX device automatically, but it will work with a simple xorg.conf tweak. Something like this: https://linux-sunxi.org/Dual_Monitor_Su ... figuration
Posts: 112
Joined: Sat May 19, 2012 6:15 pm
by ChT » Sat Nov 14, 2015 8:18 pm
I just want to move this topic up to the front again.
It would be nice to have X on both, touchscreen as well as HDMI.
As I understand, omxplayer is using dispmanx to display contents on the screen not used by X. So would it be possible to write an fbdev on top of dispmanx?


Christoph
Posts: 2
Joined: Sat Sep 26, 2015 8:10 am
by tbaumgartner » Sat Feb 20, 2016 10:50 am
Was there any progress on using DSI and HDMI output at the same time?

I would like to use two separate X servers running on the rpi. One X server presents a menu system on the DSI screen while the other X server shows the application started by the menu system on the HDMI output.
Posts: 1
Joined: Sat Feb 20, 2016 10:44 am
by gtrawoger » Sat Feb 27, 2016 4:37 pm
I am also interested in this as well.

I hit the 'no frame buffer wall' for the second display. I was encouraged when I got omxplayer working on the second display (controlled with the touchscreen', output on HDMI). And I wanted to also use fbi to display images, but realized quickly that that was not working. After googling for hours I realized that this is where we are at: no frame buffer, no proper second display.

I am a noob at this (I know basic Linux admin, am a PHP developer) so I don't think I can develop too much. But I am highly motivated to work at this. I need to get this to work, since I already invested many hours and a bit of money to have this work.

Does anyone have any pointers to where to go next from here? Omxplayer has it figured out. How can we use what it uses?
Posts: 2
Joined: Sat Feb 27, 2016 4:22 pm
by gtrawoger » Mon Feb 29, 2016 6:04 pm
Ok. I have found some more information. The way I understand it, there is no way currently to get a second framebuffer until the firmware gets updated.

So, how could one use the second display (either HDMI or touchscreen) to display something (GUI or picture)? This thread seems to have found two options: https://www.raspberrypi.org/forums/view ... 1&p=880255

One option is to use QT5 with EGLFS plugin compiled: https://www.raspberrypi.org/forums/view ... 24#p863624
The other option is to use Kivy to create the GUI on the non-framebuffer device: https://www.raspberrypi.org/forums/view ... 86#p883986

I will see if these options will work for my needs. Hopefully it might work for some of you as well.
Posts: 2
Joined: Sat Feb 27, 2016 4:22 pm