6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

ATTN: Developers using RGB888 from IL or MMAL Camera

Mon May 16, 2016 9:55 pm

Hi All.

This is a quick call for any application or library developers who are using MMAL_ENCODING_RGB24 / MMAL_ENCODING_BGR24, or OMX_COLOR_Format24bitRGB888 / OMX_COLOR_Format24bitBGR888 direct from the camera component under IL or MMAL. Could you respond below?
Please do not reply if you are using raspistillyuv, raspividyuv, V4L2, or PiCamera as those can all be fixed easily. It's the other random 3rd party apps that I'm trying to identify.

In testing some other features I've found that the camera component has swapped RGB for BGR in the enum handling. Ideally I'd like to correct it in the camera component, but that will require updates for anyone using those formats. There will be a mechanism to determine whether the camera is fixed or not, so a client can support old and new firmwares correctly.

Thanks.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Tue May 17, 2016 11:33 am

(Pinging my own thread whilst I ask for a sticky - this is a significant change that is going to happen, so need developers to be aware of it)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed May 18, 2016 10:21 am

So quiet. I guess this means no one uses RGB888 or BGR888 directly and I'll be in the clear as long as raspistillyuv, V4L2, and PiCamera are all updated.

Timescales for this being implemented is likely to be the start of next week.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed May 25, 2016 3:53 pm

Still all quiet, so I'm assuming that no one is using RGB/BGR888 directly.
That's good news if so, as I can avoid breakages. I'll look at sorting the relevant patches tonight, and pass them to Pi Towers for release.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hairu526
Posts: 18
Joined: Thu Mar 06, 2014 1:12 am

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Tue May 31, 2016 11:58 pm

We use MMAL_ENCODING_BGR24 in FreeLSS (https://github.com/hairu/freelss/) in order to get RGB. I would have to imagine that many other projects do also, and simply aren't seeing your post. Is there a way to detect the if the MMAL version has the fix so that we can code in backwards compatibility to the older firmware?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23365
Joined: Sat Jul 30, 2011 7:41 pm

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed Jun 01, 2016 6:44 am

hairu526 wrote:We use MMAL_ENCODING_BGR24 in FreeLSS (https://github.com/hairu/freelss/) in order to get RGB. I would have to imagine that many other projects do also, and simply aren't seeing your post. Is there a way to detect the if the MMAL version has the fix so that we can code in backwards compatibility to the older firmware?
OP states there will be a mechanism to detect the change.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed Jun 01, 2016 6:51 am

Hi hairu526, thanks for the response.
If lots of people had shouted then I was going to add a very simple parameter, but no one did until now so it's a little more involved. Googling hadn't produced any results, but I guess it may skip github code.

All mmal components support MMAL_PARAMETER_SUPPORTED_ENCODINGS on each port to retrieve the 4cc values for the encodings it supports (Missing out opaque). The order of RGB24 and BGR24 has been swapped in the new firmware revisions, and it actually works on the stills port (it failed before). I'll push up my raspistillyuv update to github and provide a link to it. Scan the list and find which occurs first to determine version.

If others shout saying they are using it then I may reevaluate and add something easier, or add a helper function to the mmal utils directory, but for now that is the route I'd gone. (Actually I may do the helper function anyway. Watch this space).
(edit to correct typos)
Last edited by 6by9 on Wed Jun 01, 2016 10:21 am, edited 1 time in total.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed Jun 01, 2016 10:20 am

I've reworked my patch to include a util function to perform the check, and then use it in raspistillyuv. It's pushed to https://github.com/6by9/userland if you want to have a look.
One big caveat - I haven't checked that I've got the two encodings the right way around in raspistillyuv for backwards compatibility yet. I will check before it's merged, but it's up to show the intent.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

hairu526
Posts: 18
Joined: Thu Mar 06, 2014 1:12 am

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Thu Jun 02, 2016 2:08 am

Thank you very much.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Thu Jun 02, 2016 7:33 am

Checked last night and, other than removing extra rubbish that had got left behind, it works as expected.
The patches should go to Dom today, although if/how we stagger release may warrant discussion - get the client software checking first, then change the firmware?

Thanks for your cooperation.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Thu Jun 02, 2016 7:01 pm

All commits to the framework stuff (V4L2, MMAL, Raspistillyuv) have been made to the rpi-update image and their respective source repos.
The last change to actually switch RGB to BGR and vice versa is being held until PiCamera is updated.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Mon Jun 20, 2016 2:54 pm

PiCamera has just been updated to support handling this correctly.
I'm making the request of Pi Towers to include the fix in the firmware now, so it will be bumped in the official Raspbian repo very soon.

SHOUT VERY SOON IF THIS IS GOING TO CAUSE YOU MAJOR GRIEF.

I'm trusting hairu526 has already taken appropriate steps to handle it in his app, and no one else has responded.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

EULERPI
Posts: 51
Joined: Sun May 15, 2016 2:44 pm

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Tue Jul 12, 2016 4:35 am

6by9 wrote:Hi All.

This is a quick call for any application or library developers who are using MMAL_ENCODING_RGB24 / MMAL_ENCODING_BGR24, or OMX_COLOR_Format24bitRGB888 / OMX_COLOR_Format24bitBGR888 direct from the camera component under IL or MMAL. Could you respond below?
Please do not reply if you are using raspistillyuv, raspividyuv, V4L2, or PiCamera as those can all be fixed easily. It's the other random 3rd party apps that I'm trying to identify.

In testing some other features I've found that the camera component has swapped RGB for BGR in the enum handling. Ideally I'd like to correct it in the camera component, but that will require updates for anyone using those formats. There will be a mechanism to determine whether the camera is fixed or not, so a client can support old and new firmwares correctly.

Thanks.
I'm developing an application using raspicam_cv 0.1.3 C++ API/G C++/OPENCV3.0/OPENMP/Pi 3B and since updating/upgrading Raspbian/firmware to the latest version on Sunday the Pi Camera V2 NOIR is sending RGB rather than BGR to OPENCV.

Also the field of view may have widened after the firmware upgrade which if so could cause me a major headache as I need to image process a shallow 3D scene at ~mm per pixel resolution from sufficient (fixed) distance to avoid big shadow and slant angle problems. I'm using 1024 x 640 image size at present.

Regards

Nick Murphy

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7140
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: ATTN: Developers using RGB888 from IL or MMAL Camera

Wed Jul 13, 2016 12:06 pm

EULERPI wrote:I'm developing an application using raspicam_cv 0.1.3 C++ API/G C++/OPENCV3.0/OPENMP/Pi 3B and since updating/upgrading Raspbian/firmware to the latest version on Sunday the Pi Camera V2 NOIR is sending RGB rather than BGR to OPENCV.
Answered on your other thread - viewtopic.php?f=43&t=154059
EULERPI wrote:Also the field of view may have widened after the firmware upgrade which if so could cause me a major headache as I need to image process a shallow 3D scene at ~mm per pixel resolution from sufficient (fixed) distance to avoid big shadow and slant angle problems. I'm using 1024 x 640 image size at present.
Which is not relevant here. If you want to discuss FOV, start a new thread.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Camera board”