User avatar
perfo
Posts: 81
Joined: Mon Jan 30, 2012 2:42 am

Read Pixel colour from screen.

Thu Mar 14, 2019 8:44 am

I've put this in the general programming as I'm open to suggestions whether in C or python. I would prefer python as I understand it a lot more but maybe able to work with C if it isn't too complex.
So As I stated I want to pick a pixel at (X,Y) being displayed on screen (HDMI output).
is the Frame buffer the way to go ? Does the frame buffer contain whatever is currently being shown on the screen ? I ask as I found a bit of code in C that lists pixel data from the frame buffer but it seems to give me the same results no matter if the screen has a white back ground or a black one so I'm thinking there may be more to it...

Would any one have suggestions on the simplest way to do this ?
It seem fairly common to do this to the camera input (capture scan) or a jpeg but I can't find much about doing it to the screen.

Thanks

Andyroo
Posts: 1556
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 8:53 am

If no one comes up with an answer you could always use scrot to capture the screen and then look at the resulting file :lol:
Need Pi spray - these things are breeding in my house...

User avatar
perfo
Posts: 81
Joined: Mon Jan 30, 2012 2:42 am

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 10:10 am

Thanks Andyroo.

The thing is I need to do this live so whatever method I use has to be able to process it at least as fast as the frame rate of the video (ie 60 FPS)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21510
Joined: Sat Jul 30, 2011 7:41 pm

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 11:19 am

perfo wrote:
Thu Mar 14, 2019 10:10 am
Thanks Andyroo.

The thing is I need to do this live so whatever method I use has to be able to process it at least as fast as the frame rate of the video (ie 60 FPS)
That sort of speed seems unlikely, depending on requrerements, but you can work it out. How many pixels do you want to read per frame?

The frame buffer approach seems appopriate. You can open the /dev/fb0 file, mmap to memory and read from that perhaps? https://gist.github.com/Darfk/5790622 seems to do this.

Note that this will not work with camera output or video display as they are not presented to the framebuffer. You might be able to use dispmanx_snapshot in those cases?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
perfo
Posts: 81
Joined: Mon Jan 30, 2012 2:42 am

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 12:48 pm

Hello Jamesh
I only want to read one pixel per frame no matter what the resolution is. I was hoping with only reading one it may enable it to be speedy enough,

I'm not interested in the camera for this application so that's not a problem but what do you mean by the video display is not presented to the frame buffer. It's the HDMI output I want to sniff pixels from is this the video display ? or do you mean the composite video output on the other port ?

I'm having a fiddle with the file you gave me a link to ...... I'm on 32bpp and it's set up for 8 but I'll play and see if I can figure anything out...

thanks...
Last edited by perfo on Thu Mar 14, 2019 1:07 pm, edited 1 time in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 21510
Joined: Sat Jul 30, 2011 7:41 pm

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 1:01 pm

perfo wrote:
Thu Mar 14, 2019 12:48 pm
Hello Jamesh
I only want to read one pixel per frame no matter what the resolution is. I was hoping with only reading one it may enable it to be speedy enough,

I'm not interested in the camera for this application so that's not a problem but what do you mean by the video display is not presented to the frame buffer. It's the HDMI output I want to sniff pixels from is this the video display ? or do you mean the composite video output on the other port ?
thanks...
If you running some sort of video output via the GPU (e.g. using the HW aceleration) then the framebuffer never sees that information - its is composited over the top of the frame buffer in HW straight to HDMI. So reading a pixel will not get the video but the underlying framebuffer.

So ifyou want to get a pxiel from video, you need to use dispmanx_snapshot (or whatever its called), which will grab the entire image, which you would then grab the pixel from. However, that will be a LOT slower.

If I think of another way of doing it will let you know!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
perfo
Posts: 81
Joined: Mon Jan 30, 2012 2:42 am

Re: Read Pixel colour from screen.

Thu Mar 14, 2019 4:56 pm

Thanks for the tips James...
I'm using omxplayer to play a video and trying to grab a pixel from that and so far I wasn't getting much sense. Maybe you've just explained why that is...

I have other posts on this forum asking questions as I've tried different ways to achieve my project. My project is very simple in principle. I want an output (gpio) pin to go high when the video shows a particular frame.

I've tried using the vsync from dispmanx but that doesn't appear to be in sync with the frames. So that's why I'm thinking if I can hard code a particular pixel in the video to be a particular colour for pin on and another for pin off then if I can grab that pixel in code and control the GPIO I will always be in p[perfect sync with the frame regardless of refresh rates or fps of video etc..

Return to “General programming discussion”