ethanol100
Posts: 640
Joined: Wed Oct 02, 2013 12:28 pm

Re: New version of Raspistill

Mon Oct 21, 2013 10:21 pm

I think the program needs about 1 second to initialize the camera.
If I just call "time raspistill -ss 30 -t $n -o - > /dev/null", I get for real:

n=0: 1.124s
n=1000: 2.042s
n=2000: 3.038s

So it always needs ~+1s longer.

Do you see the same behaviour of frames getting darker with raspivid too?
I had tried it in bright sun light, and it seems to be relative stable. Have not tested time-lapse and have only used -awb sun.

User avatar
jbeale
Posts: 3675
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: New version of Raspistill

Mon Oct 21, 2013 10:34 pm

ethanol100 wrote:I think the program needs about 1 second to initialize the camera.
Right, but if we are using manual exposure and fixed white balance, I don't understand why it should need 1 second. There are many I2C transactions required to set up the camera from "cold start" to take a single still frame, see for example this post, but based on the I2C camera control data I logged, it looks like you can get frames after 0.3 seconds, maybe sooner with some optimizations. Of course we can use the new SIGNAL option in raspistill to get a frame with very low latency if we have the process already running, which is a great feature.

Regarding video, I tried this:

Code: Select all

raspivid -t 10000 -w 256 -h 256 -ISO 100 -awb off -ss 2000 -o test2.h264
and found during first half-second of video, the image quickly ramped down in brightness (as if it started from a default shutter and then was steered towards the commanded 2 millisecond shutter). After that the image brightness remained the same, with an exposure matching what I would have expected at that shutter speed and the illumination I had.

ethanol100
Posts: 640
Joined: Wed Oct 02, 2013 12:28 pm

Re: New version of Raspistill

Mon Oct 21, 2013 11:20 pm

Could you try a quick and dirty hack:

Code: Select all

int set_shutter=raspicamcontrol_set_shutter_speed(state.camera_component, state.camera_parameters.shutter_speed);
if(set_shutter!=0)
  fprintf(stderr, "Failed to update shutter speed\n");
directly before
"vcos_semaphore_wait(&callback_data.complete_semaphore);"
in RaspiStill.C around line 1659.

I have tried it and get relative stable exposure times, but the camera is in a completely dark room at the moment, I can't check if the frames would go darker and can only watch the exif data.

(I think it would be really useful to have an version of raspistill, with just a live preview and the possibility to change all the parameter and to observe their results in real time, like keybindings to increase or decrease shutter speed, choose between iso and awb methodes, playing with coloreffects... endless possibilities. Like v4l2ucp for v4l2 devices.)

User avatar
jbeale
Posts: 3675
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: New version of Raspistill

Tue Oct 22, 2013 1:07 am

ethanol100 wrote:Could you try a quick and dirty hack:
Looking good, I put in your code and my exposure time remains fixed now! My camera setup is basically a microscope, with illumination from a white LED controlled by the Pi GPIO. In this case I got 10 frames, I checked the first and last and they look just about identical, so I think the exposure really is fixed. Great!

Code: Select all

$ cat ftest.sh
sudo ./whiteon.sh
raspistill -t 5000 -tl 0 -w 256 -h 256 -ISO 100 -awb off -ss 8000 -o W%02d.jpg
sudo ./offwhite.sh
identify -verbose W*.jpg | grep ExposureTime

$ ./ftest.sh
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000
    exif:ExposureTime: 7956/1000000

Code: Select all

identify -verbose W01.jpg

  Image statistics:
    Overall:
      min: 0 (0)
      max: 246 (0.964706)
      mean: 105.469 (0.413603)
      standard deviation: 62.3166 (0.244379)

identify -verbose W10.jpg
    Overall:
      min: 0 (0)
      max: 248 (0.972549)
      mean: 105.486 (0.413671)
      standard deviation: 62.1076 (0.243559)
The exposure remains the same, from 1st to 10th frame. (just an out of focus image of my desktop, but nevermind that)
RPi_W01.JPG
RPi_W01.JPG (31.27 KiB) Viewed 4696 times
RPi_W10.JPG
RPi_W10.JPG (31.27 KiB) Viewed 4696 times

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 8:40 am

ethanol100 wrote:Could you try a quick and dirty hack:

Code: Select all

int set_shutter=raspicamcontrol_set_shutter_speed(state.camera_component, state.camera_parameters.shutter_speed);
if(set_shutter!=0)
  fprintf(stderr, "Failed to update shutter speed\n");
directly before
"vcos_semaphore_wait(&callback_data.complete_semaphore);"
in RaspiStill.C around line 1659.

I have tried it and get relative stable exposure times, but the camera is in a completely dark room at the moment, I can't check if the frames would go darker and can only watch the exif data.

(I think it would be really useful to have an version of raspistill, with just a live preview and the possibility to change all the parameter and to observe their results in real time, like keybindings to increase or decrease shutter speed, choose between iso and awb methodes, playing with coloreffects... endless possibilities. Like v4l2ucp for v4l2 devices.)
Hmm, looks like ss is a parameter that keep needing to be sent (rather like the exif stuff). I'l fix up the code when time allows.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 8:46 am

@jbeale: Using a microscope means you're not using the standard lens right?

The image below (click for full view) was taken with a white diffuser over the sensor (without a lens) and sunlight shining onto it. So the illumination was completely even. The top image is the jpeg output, which is gray as it should be while the shading to compensate the vignetting of the standard lens is still in place.

The bottom image is the RAW output where the shading seems to be removed (as jamesh said it would be) but there's still a distinct color difference where the original shading seems to exists in a green vs magenta color cast. Do you see that also or am I missing something? My goal is to use a non standard lens to stitch hi quality images but that won't work for now with either jpeg or raw.
raw01.jpg
raw01.jpg (15.96 KiB) Viewed 4672 times

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 8:52 am

Those results are typical for RAW with no lens shading compensation. In fact even the JPEG is a bit iffy which indicates the lens shading table isn't quite right.

You need to apply your own lens compensation to the RAW. If you have loads of memory you can work out what needs to change for each pixel from the flat field view you have, and apply that to the RAW to make it properly flat field. The way it's done in the GPU is using piecewise linear interpolation of a much smaller table, to reduce memory requirements. And it's done at 30fps!

If you do create a compensation table, please post it,it may be usable in the GPU code to improve the current table.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 9:05 am

Thanks, that makes sense. Do you or anyone else have more information on how to create such a table? I'm very willing to put a good deal of effort in this and post the results but a few pointers would help a lot.

One of the downsides using the shading with the original lens seems to be that the maximum dynamic range of the final image is reduced. The whites in the jpeg never exceed 240 where they're 'normally' (using a DSLR) 255. Also the blacks never go below 30 where they should be much closer to 0. That said I feel the overall IQ of the camera board is quite impressing for such a tiny and cheap device.

ethanol100
Posts: 640
Joined: Wed Oct 02, 2013 12:28 pm

Re: New version of Raspistill

Tue Oct 22, 2013 9:50 am

jamesh wrote: Hmm, looks like ss is a parameter that keep needing to be sent (rather like the exif stuff). I'l fix up the code when time allows.
It could also result from some autoexposure/awb going on in the background, changing exposure and whitebalance. Just resending the ss would then not be a good solution to the problem. But this is really a minor problem, taking single pictures works fine, so no hurry.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 10:02 am

poing wrote:Thanks, that makes sense. Do you or anyone else have more information on how to create such a table? I'm very willing to put a good deal of effort in this and post the results but a few pointers would help a lot.

One of the downsides using the shading with the original lens seems to be that the maximum dynamic range of the final image is reduced. The whites in the jpeg never exceed 240 where they're 'normally' (using a DSLR) 255. Also the blacks never go below 30 where they should be much closer to 0. That said I feel the overall IQ of the camera board is quite impressing for such a tiny and cheap device.
Lens shading correction by definition reduces the max dynamic range. Also, you have also encountered the 'black level', the point below which all colours are indeed black. This is an important number, and looking at the tuning, it is about 16 (on a ten bit colour). Although this number does depend on gain, temperature etc.

Are the numbers you are looking at from the JPG or the RAW btw?
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 10:03 am

ethanol100 wrote:
jamesh wrote: Hmm, looks like ss is a parameter that keep needing to be sent (rather like the exif stuff). I'l fix up the code when time allows.
It could also result from some autoexposure/awb going on in the background, changing exposure and whitebalance. Just resending the ss would then not be a good solution to the problem. But this is really a minor problem, taking single pictures works fine, so no hurry.
There are some values that get reset after a capture - this appears to be one of them. SS changing after the first frame implies this, since it should be constant, irrespective of the exposure and whitebalance.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

thegnnu
Posts: 157
Joined: Thu Oct 18, 2012 7:07 pm
Location: Bristol

Re: New version of Raspistill

Tue Oct 22, 2013 10:33 am

jamesh wrote:That's Hexxah's git hub for rpi-update, not the Raspberry Pi userland. I don't know about that, but I'd suggest trying again.

Then do the following

Code: Select all

git clone https://github.com/raspberrypi/userland.git
cd userland
git checkout master
./buildme
jamesh Thanks for info all work and raspistill is upgraded

Terry

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 10:40 am

jamesh wrote:
poing wrote:Thanks, that makes sense. Do you or anyone else have more information on how to create such a table? I'm very willing to put a good deal of effort in this and post the results but a few pointers would help a lot.

One of the downsides using the shading with the original lens seems to be that the maximum dynamic range of the final image is reduced. The whites in the jpeg never exceed 240 where they're 'normally' (using a DSLR) 255. Also the blacks never go below 30 where they should be much closer to 0. That said I feel the overall IQ of the camera board is quite impressing for such a tiny and cheap device.
Lens shading correction by definition reduces the max dynamic range. Also, you have also encountered the 'black level', the point below which all colours are indeed black. This is an important number, and looking at the tuning, it is about 16 (on a ten bit colour). Although this number does depend on gain, temperature etc.

Are the numbers you are looking at from the JPG or the RAW btw?
I was looking at the jpeg. If I desaturate both the jpeg and the RAW I get (approx)

Code: Select all

type - center - low right corner
jpeg - 140 - 215
raw - 180 - 180
jpeg (black) - 25 - 25
where middle gray is 127 so the RAW is basically too light. No doubt that is done to make the further jpeg wizardry give a better end result.

What I do not get my head around to is why the RAW gray scale is so even but the color is not. Could it be that there's some color compensation done before the data is saved to RAW?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 10:51 am

No, the RAW is straight off the sensor.

As for the range of values you get the in the JPEG, the whole of the processing pipeline has the ability to stretch and offset these numbers to make a decent image (in rgb, yuv etc), so the range you get is not very predictable. Black level, lens shading, gamma correction, awb, AGC etc.

EDIT: The raw has had no gamma correction, so the responses of the different colour colours won't be matched/linear.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
jbeale
Posts: 3675
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: New version of Raspistill

Tue Oct 22, 2013 2:03 pm

poing wrote:@jbeale: Using a microscope means you're not using the standard lens right?
Maybe I shouldn't have said 'microscope', I am just using the standard camera module with lens as-is, and added a strong closeup lens. It's not a great optical system and for flat subjects, only the centermost part of the field of view is really in focus. I was thinking about trying to replace the lens but have not yet done so.

You have proceeded past what I did with RAW and I don't understand why there would be the radial color cast that you observe, unless maybe there is some subtlety with the IR filter? Or microlens array, if any? Or maybe just a built-in manufacturing imperfection in the Bayer color filter array where the density of each R-G-B pixel's filter varies a little bit over the chip, something that would just have to be experimentally measured and removed in post-processing, as jamesh suggested.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 2:07 pm

Not sure if this is the issue, but each of the microlenses colour pigments (RGB) have different responses, and can be quite irregular. The colour correction matrix attempts to sort them out, but can never be perfect.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 2:29 pm

I found a quick and dirty solution in the form of a Lightroom plugin called 'DNG Flat Field Plug-in' (download: http://labs.adobe.com/downloads/lightro ... ml#install, description: http://labsdownload.adobe.com/pub/labs/ ... f_docs.pdf) which gives the best result so far:
rawtestOK.jpg
RAW image taken with a Nikkor 105mm f/2.8VR micro lens
rawtestOK.jpg (58.07 KiB) Viewed 4520 times
Sadly Lightroom is the type of program I don't like, but it gets the job done for now. I'll try to investigate a solution using tables, although I'm not your typical tech wizard.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 2:43 pm

What the plugin is doing is taking a calibrated image - a flat field from a light box or similar, working out a correction table, then applying it to the image. The correction table would be a colour correction per pixel (if you can afford the space) to convert the image of the flat field to an actual flat field. ie you know what colour you should have, flat grey/white, so you need to adjust the actual image to match.

Interestingly, there are some algorithms that can do the same correction without the calibration image. I have no idea how they work, but we used them on the Nokia 808 and some SS phones to avoid the calibration stage. These are not available on the Raspi as they are third party code and cost a load of money (relatively speaking)
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 3:16 pm

So in the control shot when you have pixel that should be gray (127,127,127) but has the values 127,157,127 you enter into the table a value of 0,-30,0 ? In Visual Basic I can do that so I must be able to do that in Python as well.

Then read a DNG file, correct the values with the table and re save the DNG file.

poing
Posts: 1132
Joined: Thu Mar 08, 2012 3:32 pm

Re: New version of Raspistill

Tue Oct 22, 2013 3:32 pm

jamesh wrote:Not sure if this is the issue, but each of the microlenses colour pigments (RGB) have different responses, and can be quite irregular. The colour correction matrix attempts to sort them out, but can never be perfect.
That would mean the raw color cast is sensor specific and should be corrected using an image from that specific sensor. Also I take my hat off to the people that made that color correction matrix.

User avatar
waveform80
Posts: 359
Joined: Mon Sep 23, 2013 1:28 pm
Location: Manchester, UK
Contact: Website Twitter

Re: New version of Raspistill

Tue Oct 22, 2013 3:55 pm

poing wrote:So in the control shot when you have pixel that should be gray (127,127,127) but has the values 127,157,127 you enter into the table a value of 0,-30,0 ? In Visual Basic I can do that so I must be able to do that in Python as well.

Then read a DNG file, correct the values with the table and re save the DNG file.
Yup, should be pretty easy in Python. The numpy library would be the quickest and easiest way, I suspect (it provides matrix facilities so subtracting one matrix of pixels from another is as simple as: result = matrix1 - matrix2).

I'm intending to add raw capture facilities to the picamera library in the near future and I might be tempted to sling an optional numpy interface in there; I've just finished adding manual shutter speed (just after the recent 0.5 release - it'll be in the next one). Interestingly, I haven't noticed the shutter speed varying in between image captures (I don't need to reset it between captures), but perhaps that was because I'd set the shutter speed while the preview was running? Not sure - I'll have to experiment a bit more. At the moment I'm experimenting with ISO settings to try and come up with some more sensible limits for the property setter.

Cheers,

Dave.
Author of / contributor to a few pi related things (picamera, Sense HAT emulator, gpio-zero, piwheels, etc.), and currently a software engineer at Canonical responsible for Ubuntu Server and Core on the Raspberry Pi.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 4:09 pm

poing wrote:
jamesh wrote:Not sure if this is the issue, but each of the microlenses colour pigments (RGB) have different responses, and can be quite irregular. The colour correction matrix attempts to sort them out, but can never be perfect.
That would mean the raw color cast is sensor specific and should be corrected using an image from that specific sensor. Also I take my hat off to the people that made that color correction matrix.
Actually, my phrasing was misleading. I was saying that the pigments responses vary according to R, G and B ie the curves are different shapes, and the curves can be wonky.

Of course, there is also the possibility of slightly misaligned microlenses as well....and tbh, the colour cast and lens shading (and black level) can be a bit sensor specific - which is why on more expensive devices the sensors are calibrated on the production line to take that in to account. We don't do that.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

SnowLeopard
Posts: 106
Joined: Sun Aug 18, 2013 6:10 am

Re: New version of Raspistill

Tue Oct 22, 2013 5:16 pm

jamesh wrote:There are some values that get reset after a capture - this appears to be one of them. SS changing after the first frame implies this, since it should be constant, irrespective of the exposure and whitebalance.
I see the raspicamcontrol_set_* functions are really simple, so the to-be-written raspicamcontrol_get_* equivalents should be just as simple (as in within my limited abilities).
Fetching the values after a capture should show which values do get reset and which do not...?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 26652
Joined: Sat Jul 30, 2011 7:41 pm

Re: New version of Raspistill

Tue Oct 22, 2013 5:56 pm

SnowLeopard wrote:
jamesh wrote:There are some values that get reset after a capture - this appears to be one of them. SS changing after the first frame implies this, since it should be constant, irrespective of the exposure and whitebalance.
I see the raspicamcontrol_set_* functions are really simple, so the to-be-written raspicamcontrol_get_* equivalents should be just as simple (as in within my limited abilities).
Fetching the values after a capture should show which values do get reset and which do not...?
Probably, but the reason I never wrote them is that in fact there really isn't much (read: any?) need for them. I was looking at that bit of code just now and thinking about that!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
waveform80
Posts: 359
Joined: Mon Sep 23, 2013 1:28 pm
Location: Manchester, UK
Contact: Website Twitter

Re: New version of Raspistill

Tue Oct 22, 2013 6:37 pm

SnowLeopard wrote:
jamesh wrote:There are some values that get reset after a capture - this appears to be one of them. SS changing after the first frame implies this, since it should be constant, irrespective of the exposure and whitebalance.
I see the raspicamcontrol_set_* functions are really simple, so the to-be-written raspicamcontrol_get_* equivalents should be just as simple (as in within my limited abilities).
Fetching the values after a capture should show which values do get reset and which do not...?
Actually, that's one thing the picamera library is quite useful for already: the various properties have getters and setters so, for example, you can do the following (this is from an interactive session over SSH with the current development head - the shutter_speed property isn't in the latest release but will be in the next one):

Code: Select all

>>> import picamera
>>> camera = picamera.PiCamera()
>>> camera.shutter_speed
0
>>> camera.shutter_speed = 50000
>>> camera.shutter_speed
49992
>>> camera.capture('foo.jpg')
>>> camera.shutter_speed
49992
Given it uses libmmal in basically the same way as raspistill you should get similar results (although it's possible to do things in an order that raspistill doesn't - e.g. you can start a preview and then modify settings rather than modifying them before the preview starts).


Cheers,

Dave.
Author of / contributor to a few pi related things (picamera, Sense HAT emulator, gpio-zero, piwheels, etc.), and currently a software engineer at Canonical responsible for Ubuntu Server and Core on the Raspberry Pi.

Return to “Camera board”