Extracting colors

[rleonardi]

Dear all,

I am using Raspberry Pi 2 B, the Raspberry Camera Module and raspistill.

I would like to remove the red, the green and/or the blue from the preview and from the jpg pictures. For example, to watch a preview which only contains the information generated by the green and blue photodetectors of the CMOS.
For example, if I have a red object, a green object, a blue object and I remove the red information, I would like to see only the residual green and blue components of the red object both in preview and in the .jpg file, maybe just a dark blob if the object is only pure red.

I tried "-awb off -awbg <value, value>" and set low value for red and/or blue gains, but I think that red and blue objects where still visible, just changed to green.

Now I'm looking into the forums and into the soruce code of raspistill, MMAL, OpenMAX, etc, to see if there is a way to do this. For example how to say to the GPU to discard the red pixels while doing debayer, or to discard the red component when encoding the preview and the jpg file, etc.
I'll be also looking if there is a way to disable the red, the green and/or the blue when the preview and the pictures are shown on the monitor.

Could you possibly let me know if you have any suggestions?

Thank you in advance.
Regards.
Riccardo

[rleonardi]

Hi,
By doing further testing, I see that by specifying "-awb off" and removing the awbg (0 gains should be used), the preview is completely black but the jpg is correctly created and the picture is completely green. Why the preview is black?
Thanks again

[6by9]

There is no way to modify the demosaicing algorithm to drop certain colours. All the processing post demosaic is in the YUV or YCbCr domains, so there is no option there to munge the colours. If you ask for RGB, then it is converted back from YUV to RGB as a final step.

Why preview is black when you leave awbg at 0? I have no idea. Add -set to your raspistill command line and it'll print out all the gains as they change. Anything obvious there?
Possibly that something special cases 0 as not initialised (I vaguely recall implementing the manual AWB gains that way). Try a small value instead (internally values are likely to be 8p8 fixed point, so 0.004 (1/256) should get through.

[rleonardi]

... Testing ... By using "-awb off -awbg 2.0, 0.0", the preview is black and the captured jpg is green. By using "-awb off -awbg 2.0, 0.01", the preview is red and the captured jpg is red. Why just a little +0.01 to the gain changes so much the behaviour? ...
Thanks

[6by9]

Possibly that something special cases 0 as not initialised (I vaguely recall implementing the manual AWB gains that way)

[rleonardi]

Hi,

Please find the following results of my further testing according to your information. I think there is still a strange behaviour of the preview and of the jpg.

Thanks

------------------------------------

pi@raspberrypi ~/code/userland/build $ sudo ./bin/raspistill -f -awb off -ss 50000 -awbg 2.0,0.0 -md 2 -r -o ./bin/picture%04d.jpg -e jpg -q 100 -w 640 -h 480 -t 0 -k -set
...
mmal: Exposure now 49973, analog gain 352/256, digital gain 544/256
mmal: AWB R=0/256, B=0/256
...

Preview black, picture green

------------------------------------

pi@raspberrypi ~/code/userland/build $ sudo ./bin/raspistill -f -awb off -ss 50000 -awbg 2.0,0.004 -md 2 -r -o ./bin/picture%04d.jpg -e jpg -q 100 -w 640 -h 480 -t 0 -k -set
...
mmal: Exposure now 49973, analog gain 256/256, digital gain 565/256
mmal: AWB R=512/256, B=1/256
...

Preview black, picture almost black

------------------------------------

pi@raspberrypi ~/code/userland/build $ sudo ./bin/raspistill -f -awb off -ss 50000 -awbg 0.004,0.004 -md 2 -r -o ./bin/picture%04d.jpg -e jpg -q 100 -w 640 -h 480 -t 0 -k -set
...
mmal: Exposure now 49973, analog gain 352/256, digital gain 635/256
mmal: AWB R=1/256, B=1/256
...

Preview black, picture almost black

------------------------------------

pi@raspberrypi ~/code/userland/build $ sudo ./bin/raspistill -f -awb off -ss 50000 -awbg 0.0,0.0 -md 2 -r -o ./bin/picture%04d.jpg -e jpg -q 100 -w 640 -h 480 -t 0 -k -set
...
mmal: Exposure now 49973, analog gain 336/256, digital gain 647/256
mmal: AWB R=0/256, B=0/256
...

Preview black, picture green

------------------------------------

pi@raspberrypi ~/code/userland/build $ sudo ./bin/raspistill -f -awb off -ss 50000 -md 2 -r -o ./bin/picture%04d.jpg -e jpg -q 100 -w 640 -h 480 -t 0 -k -set
...
mmal: Exposure now 49973, analog gain 400/256, digital gain 546/256
mmal: AWB R=0/256, B=0/256
...

Preview black, picture green

[6by9]

I'm afraid on this one I'm going to have to say it will be too involved to investigate, and you're pushing the system far into realms that weren't intended.

Options I can see for you:
- can you take Raspistillyuv or raspividyuv in RGB mode and then you do the pixel manipulation in userspace?
- how about doing all the image processing yourself? viewtopic.php?f=43&t=109137 gives you details of how to get the raw Bayer image frames in from the sensor (including unpacking them into easy to handle raw16 instead of nasty raw10), and then you can do your own demosaic and everything. You've disabled almost all the automatic control algorithms anyway, so the GPU was only giving you pixel processing.

[rleonardi]

Hi all,

If a way does not exist to act on the debayer and encoding at GPU level, does a way exist to remove red, green and/or blue from the preview, for example acting on the way it is rendered on the screen.

Surfing into the forums, I can see posts about raspiraw, etc. As far as I understand, it is possible to access the CMOS, set its register, handle directly the raw data. So, are there solutions, different from raspistill, to do what I'm looking for?

Thanks again

[6by9]

If a way does not exist to act on the debayer and encoding at GPU level, does a way exist to remove red, green and/or blue from the preview, for example acting on the way it is rendered on the screen.

Did you read what I wrote in the previous message? There's nothing stopping you modifying the data buffers between camera and preview from your application.

Surfing into the forums, I can see posts about raspiraw, etc. As far as I understand, it is possible to access the CMOS, set its register, handle directly the raw data. So, are there solutions, different from raspistill, to do what I'm looking for?

Er, raspiraw would be exactly the link I provided in the last message.
I'd be very surprised if anyone has done exactly what you want as it seems a very bizarre request. You are very likely going to have to code up the image processing steps yourself. OpenCV may give you one route into it, or writing weird shaders in OpenGL. Both have been discussed on the forums previously.