Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Is there still something auto even with fixed awbg, iso, ss?

Tue Nov 17, 2015 3:58 pm

I am experimenting with a 360° camera head based on the raspberry pi cameras. I now have an issue where we see changes in color from the camera sensors depending on the lighting in the scene, even though everything is set up as fixed. We make the mmal settings from a custom application, so there is no exact command line to show, but the relevant settings are equal to the following:
raspivid -v -t 0 -w 1440 -h 1080 -ex off -awb off -awbg 1.5,1.4 -fps 40 -ss 24830 -b 10000000
(verified by A/B testing and it looks exactly the same)

Take a look at these three images, illustrating our situation:
Image
Image
Image

What you see in these images is a composition (stitch) of the image from three cameras. You can see the blending between the images as a "Y" shape. There is no rotation or other change in the relative position of each camera on the composited image between the three. What is changed is a rotation of the camera head (the three cameras are fixed to the camera head). The black/white line is a wall corner, which is vertical in the real world. There are two white A3 papers taped to the wall.

So, the issue/problem that we see is this: as we move a camera and expose it to a differently colored environment we see a change in the color from the camera even though we set all settings to be fixed and to disregard such input.

Sometimes things like this happens because our vision system is not reliable, so we made some measurements on the white paper. For one of the cameras the paper has the following colors in the three images:
img1: 102,77,63
img2: 118,88,69
img3: 97,66,44
(r,b,g)
So it seems there really is a change in the color from the camera.

Is there something I'm not setting as fixed that can be set as fixed?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20748
Joined: Sat Jul 30, 2011 7:41 pm

Re: Is there still something auto even with fixed awbg, iso,

Tue Nov 17, 2015 5:12 pm

That might be a lens shading artefact. At present there is no way of turning off lens shading (and that will make the image look rubbish anyway).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Is there still something auto even with fixed awbg, iso,

Tue Nov 17, 2015 6:56 pm

If I've read you correctly, you're using multiple physical sensors. As well as lens shading there is generally a small variation in red and blue sensitivities between sensors. On the more advanced setups there is production line calibration of each sensor for lens shading and sensitivities, with the results being written into an EEPROM on the sensor module.
You haven't actually set the iso value with your command, but the defaults should be for x1.0 analogue and digital gains anyway.

You could add -set to your command line to get raspivid to print out shutter speed, analogue and digital gains, and red and blue gains whenever they change - in your case they shouldn't change after being initially set.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Re: Is there still something auto even with fixed awbg, iso,

Tue Nov 17, 2015 8:35 pm

Yes, you are correct - we use multiple physical sensors, connected to multiple rpi2, and the gains are fixed as if supplied with -ISO 100. Jessie with up to date rpi-update.

I actually tried to calibrate the sensors a bit like that - I gave them a uniform white thing with 5500K to look at and then I wrote a feedback loop for awbg_r and awbg_b until all the pixels on the sensor average 128,128,128 (sets ss too). I hoped those awbg_r and awbg_b values would give me a 5500K calibrated sensor all fixed, but then something changes so I see the edges (as described above).

I have a similar output to -set (notification if any setting is changed to debug-out) and all the gains and settings I know of are fixed and doesn't change.

I really hope it's not the lens shading being automatic.. I read about it in some other thread and hoped it will only do its thing if a "stat pass" is being explicitly ordered?

User avatar
jbeale
Posts: 3367
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Is there still something auto even with fixed awbg, iso,

Tue Nov 17, 2015 8:41 pm

From what has been stated here the RPi built-in lens shading cannot be turned off, no matter what settings you make. It's not much of a problem if you use the official RPi camera board with its tiny lens, but it causes the yellow/purple radial color gradient that you show in those pictures if you use, for example a camera module with standard M12 lens.

I think the ideal fix would be the ability to somehow load in custom lens shading parameters that match whatever lens you are using, however that is not a supported capability.

Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Re: Is there still something auto even with fixed awbg, iso,

Wed Nov 18, 2015 8:20 pm

Grasping at straws here, but in the pdf that is loose I read about register 0x5000 where bit 7 is said to control lens correction. I saw somebody had managed to talk to the camera over I2C0 since the videocore ISP is silent there after everything is set to manual, so maybe that would be worth trying. Is it known, though, if the lens correction is performed on the camera chip or in the VC ISP?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Is there still something auto even with fixed awbg, iso,

Wed Nov 18, 2015 8:32 pm

Lens shading correction is done in the vc isp.
The only functions used in the sensor are analogue gain and exposure time.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

poing
Posts: 1131
Joined: Thu Mar 08, 2012 3:32 pm

Re: Is there still something auto even with fixed awbg, iso,

Fri Nov 27, 2015 4:56 pm

Is the lighting really constant and not flickering at all? You seem to have a very high shutter speed so if there's any trace of flickering in the lighting it would show up between shots. Most known example is when using fluorescent tubes with shutter speeds higher than the net frequency but I've also seen effects when filming LEDs @30fps. Just a thought.

Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Re: Is there still something auto even with fixed awbg, iso,

Fri Nov 27, 2015 6:00 pm

Thats a good tip - flickering light can definitely cause issues! But in this case I'm pretty sure that's not the issue: at 40 fps a totally open (360°) shutter is 25000 μs and we're at 24830 μs so we're pretty much all open, also the 40 fps and 50 Hz mains (Sweden) should not cause major interference I think, and also I sometimes see interference in my basement but I did not see it in this case.

I can mention that I, since I wrote here last, also have tried writing an automatic color matcher that extracts the corresponding pixels from the overlapping area from all pairs of images and compares the relative "redness" and "blueness" and adjusts awbg_r and awbg_b in order to make the edges better. It ameliorates the situation a bit but is not completely satisfactory. Also, it screws up completely on the corner cases (bit of blue thing in just right spot in one image and red in the other and colors goes bananas). Hard problem to tackle :)

I.. also looked at the I2C signals.. and (of course!) 6by9 is completely correct in that all the white balancing and gains is in the vc isp and not in the ov sensor where that is all set to off. Fun to look at the signals anyway and see what goes on between vc and the sensor :)

poing
Posts: 1131
Joined: Thu Mar 08, 2012 3:32 pm

Re: Is there still something auto even with fixed awbg, iso,

Sat Nov 28, 2015 4:49 pm

OK. Still if it's important (as it seems to be) you might want to check while the lighting is provided with old fashioned light bulbs that have a glowing wolfram thread. Those will cause absolutely no flicker as the glow will not visually diminish in the time between cycles. Maybe you have some lying around.

namorK
Posts: 15
Joined: Wed Dec 02, 2015 9:34 am

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 10:41 am

bonjour

we're struggling with a pink colour cast and brighter corners due to a lens change. also, we cant't get the exposure set completely manual (camera.awb_mode = 'off', camera.awb_gains = (1.5, 1.5)).

do you know if there was an update to get rid of the lens compensation?

and what would you suggest, would it be easier to use raspivideo and a shell script?

import picamera
import time
from datetime import datetime

count = 0
while count < 999:
with picamera.PiCamera() as camera:
camera.resolution = (2592, 1944)
time.sleep(1)
camera.start_preview()
filename = '//DATA/blue_%s_auto.jpg'%datetime.now().strftime('%Y-%m-%d-%H%M%S')
camera.capture(filename)

camera.resolution = (2592, 1944)
camera.shutter_speed = 1000
camera.exposure_mode = 'off'
camera.iso = 100
camera.awb_mode = 'sunlight'
filename = '//DATA/blue_%s_fixed.jpg'%datetime.now().strftime('%Y-%m-%d-%H%M%S')
camera.capture(filename)

camera.resolution = (1920, 1080)
camera.framerate = 25
camera.sharpness = 0
camera.contrast = -10
camera.saturation = 0
camera.ISO = 100
camera.video_stabilization = False
camera.exposure_compensation = 0
camera.exposure_mode = 'off'
time.sleep(1)
camera.start_preview()
camera.awb_mode = 'off'
camera.awb_gains = (1.5, 1.5)
camera.image_effects = 'none'
camera.color_effects = None
camera.rotation = 180
camera.hflip = False
camera.vflip = False
camera.shutter_speed = 1250
filename = '//DATA/blue_%s.h264'%datetime.now().strftime('%Y-%m-%d-%H%M%S')
camera.start_recording(filename, bitrate=25000000)
camera.wait_recording(627)
camera.stop_recording()
count = count + 1

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 11:28 am

namorK wrote:we're struggling with a pink colour cast and brighter corners due to a lens change. also, we cant't get the exposure set completely manual (camera.awb_mode = 'off', camera.awb_gains = (1.5, 1.5)).
Altering the AWB settings won't affect gain or exposure time. Your code does say you are setting ISO and shutter speed, so it should be fixed.
namorK wrote:do you know if there was an update to get rid of the lens compensation?
No, there wasn't, and that probably won't help anyway as the response of your lens is unlikely to be totally uniform.
If you're taking JPEG captures, then lens shading can be forced to recompute by adding the statistics pass, but that will also recompute AWB and digital gains.
namorK wrote:and what would you suggest, would it be easier to use raspivideo and a shell script?
Up to you. They should both expose the same functionality.

You can use either -set for raspivid, or read back analogue gain, digital gain, exposure time, and AWB gains within Picamera to see what values are varying.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

namorK
Posts: 15
Joined: Wed Dec 02, 2015 9:34 am

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 11:35 am

dear 6by9

thank you very much for your quick and detailed replay!

did i get it right? using the statistics pass doesn't affect video capture. or would there be a possibility to program a white balance feature and use it before we let our balloon fly?

thanks again, roman

Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 11:46 am

If anybody is interested:
I ended up writing a sync-routine for matching the colors of the six cameras in the 360°-rig. It actually works very well now. Here is the result of the project:
http://odenvr.com/devkit/

namorK
Posts: 15
Joined: Wed Dec 02, 2015 9:34 am

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 11:55 am

dear torkel

wow, great. does the sync routine work for video, too? and would you make the routine available to us?

greetings from sunny zurich, roman

Torkel
Posts: 27
Joined: Thu Apr 16, 2015 8:43 pm

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 12:05 pm

The color-sync routine works on live video.

Sadly I can't release the source for it. It is baked into our video processing software, and that is proprietary.

But if your problem is that you have a single camera that has a color issue, then I suggest you search around the forum here for other threads. My issue in this thread is a very very subtle difference in colors between multiple cameras that is only noticeable when stitching together the images from several cameras. I know I have seen reported issues with single camera color gradient problems in other threads.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 12:07 pm

namorK wrote:did i get it right? using the statistics pass doesn't affect video capture. or would there be a possibility to program a white balance feature and use it before we let our balloon fly?
Correct. Stats pass does NOT affect video capture, only stills. When the raw frame has been captured, it is passed through the ISP to produce statistics alone, recomputes gains based on those stats, and then processes the data again to produce the output frame. It's mainly used with a flash, as that can vary all the parameters quite significantly.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

namorK
Posts: 15
Joined: Wed Dec 02, 2015 9:34 am

Re: Is there still something auto even with fixed awbg, iso,

Mon Feb 22, 2016 12:16 pm

Torkel wrote:The color-sync routine works on live video.

Sadly I can't release the source for it. It is baked into our video processing software, and that is proprietary.

But if your problem is that you have a single camera that has a color issue, then I suggest you search around the forum here for other threads. My issue in this thread is a very very subtle difference in colors between multiple cameras that is only noticeable when stitching together the images from several cameras. I know I have seen reported issues with single camera color gradient problems in other threads.
torkel

thanks again. we will try to find the mentioned posts. we have some ugly pink cast that is difficult to get rid of. and the lens correction lets the corner look brighter then the rest ;o(

it would be wonderful, if somebody would write a routine to manually white balance and lens correct with a grey card ;o)

sadly our coding skills don't allow it ;o(

Return to “Camera board”