Raspberry Pi Forums

Hello,
I am a total beginner. These are just my first days of learning Python.
But my first impression is that the Raspberry HQ camera hasn't been implemented well. It seems to be a mess:

-Raspistill uses different automatic exposure algorithm than Python, for example different shutter speed
-Iso setting doesn't work correctly. For example choosing a lower Iso (with automatic shutter speed) can make the image look darker. Usually a lower Iso should use a lower gain and a longer exposure time without changing the brightness of the jpg. A possible reason could be the framerate, but when you adjust the framerate, iso still doesn't work correctly.
-As the Iso setting doesn't work correctly, I can not measure the brightness of the scene. There doesn't seem to be a command that does this. So it seems to be very hard to adjust the automatic(!) exposure algorithms. It also doesn't seem to be possible to get the automatic Iso value. camera.iso only gives me the value 0
-Apparently raspistill lets me set the analog gain (haven't tried yet), but in Python this doesn't seem to be possible
-I can disable noise reduction in Python, but this doesn't seem to be possible with raspistill
-the manual/documentation could be better, getting the automatic exposure time with exposure_speed only works, if I use sleep() after starting the preview. Otherwise I get the value 0.
-I have the impression that analog gain and digital gain seem to behave in a strange way. I would expect the digital gain to increase only when the analog gain is at its maximum.

What do you expect?
picamera (with one exception this year) has seen last commit in 2018:
https://github.com/waveform80/picamera

Since raspistill has what you want, why use picamera?

In case you use picamera, and nobody currently is working on making it better, you can do yourself.
I am not used to use python, but was able to add raspistill's new "--focus" feature to my picamera fork:
viewtopic.php?t=273804#p1659688
https://github.com/Hermann-SW/picamera

So your next step can be to see how I was able (with the help of forum people and 6by9) to add what I wanted.
Then create your fork of picamera and add what you miss.

What I have noticed with raspistill is that if you set a fixed analog and digital gain, the shutter speed will float for exposure, but it will not go slower than 1/16 seconds.
So that could be what you are seeing.
I modified the merlin camera script (touch camera) to give me a floating auto with fixed analog and digital gain, then set 1 stop touch values for exposure.
Thus giving max to 1/16 at fixed gains, then I fix 1/8, 1/4, 1/2 ete with touch controls.

Also why would you assume analog gain would max before digital gain is applied? Picamera doesn't do that nor do the camera manufacturers in setting up ISO values.
Whats great about the HQ camera is that we can do that. I can set up any combination I want and the best results will vary depending on the subject matter. Thus way camera manufacturers also vary analog and digital gain for most likely best calculated result. Which may not be the true best for your actual scene as not exactly what the reference was programmed to.

Yes, raspistill doesn't choose a longer exposure time than approximately 1/15s, even at Iso 100 with bad f number. So the Iso setting simply doesn't work correctly, no matter whether I use raspistill or picamera (even when I adjust the framerate with picamera).
No DSLR/mirrorless camera behaves like that. The brightness of a jpg must stay constant when you vary the Iso with automatic exposure settings (if the scene stays constant) and if you don't adjust the exposure compensation. It's the job of the Iso setting to choose an exposure time that leads to the same jpg brightness as before (if the exposure is automatic).

You have a set of exposure mode profiles:
off,auto,night,nightpreview,backlight,spotlight,sports,snow,beach,verylong,fixedfps,antishake,fireworks
Each has a set of parameters to dictate how it should alter exposure vs gain, and lso .
eg sports goes for higher gain to give a shorter exposure time.

Auto mode has a max exposure time of 63ms. If you want longer exposure times then normally you'd select night mode (exposure time up to 500ms).

ISO is implemented as an alternate set of exposure modes, and that currently has an exposure limit of 66ms. I'm just querying with others why, as that does seem quite low.

If you set gain via -ag and -dg, then the exposure mode still operates as before, but with a degree of freedom removed.

Even when I combine the night exposure mode with Iso 100, it doesn't want to choose a long exposure.
My problem is that a correctly working Iso setting would reveal the brightness of the scene (if one knows exposure time and f-number), I guess this should make it much easier to adjust the automatic exposure and gain behaviour, especially as picamera doesn't let me adjust analog and digital gain and doesn't let me read out the auto Iso value.
Even at Iso 800 the short exposure time could lead to an underexposed image in low-light conditions, this likely makes it harder to estimate the brightness of the scene. Though I am a total beginner. These are just my first impressions.

I also just noticed that the relationship between Iso and gain doesn't seem to be constant with picamera (I haven't tested raspistill)
Now I used a fixed exposure time and Iso = 1600. Picamera chose an analog gain of 9.94 and a digital gain of 2. Then I took a photo again and used an exposure time that was 10 times as long and again I used Iso 1600. Then Picamera still chose an analog gain of 9.94, but it decreased the digital gain, now the digital gain was only 1. This shouldn't happen as far as I know. When you only adjust the exposure time at the same Iso , it shouldn't decrease the gain
I also just reproduced the issue at Iso 800, seems to happen at all Isos.
So it seems that no matter whether the exposure time is automatic or not, the Iso setting doesn't seem to work correctly.

Usually, when you use a mirrorless camera or DSLR and take a photo of a constant scene with the same lens and f-number, then combinations like Iso 100 1/10s , Iso 200 1/20s , Iso 400 1/40s, Iso 800 1/80s should give you the same jpg brightness.
Iso 200 1/10s would be (1 stop) brighter than Iso 100 1/10s, just as bright as Iso 100 1/5s.
Iso 800 1/10s would be (3 stops=2^3 = 8) brighter than Iso 100 1/10s. Iso 800 1/10s must produce the same jpg brightness as Iso 100 with 0.8s exposure time or Iso 200 with 0.4s exposure time.

So due to the not correctly working Iso setting, adjusting the automatic exposure might be not very easy as picamera doesn't let me adjust the gains (the fact that I can not read out the auto iso doesn't make it easier either).

You seem to be confused on what analog and digital gain actually mean.
Analog gain is applied before the Analog to digital conversion process, ie before the AD conversion off the sensor. Digital gain is after and is best to think of it like boosting the EV exposure settings in a raw developing program.
Iso = analog gain x digital gain x 100

So for best dynamic range you want analog gain as low as possible. You could shoot -ag 1 and then boost the exposure with -dg 2 to get decent dynamic range and low noise. Setting analog gain at the maximum will drop the read noise to its minimum. So many astrophotographers will have the digital gain at 1 and the analog gain set to 16.
It depends on what you want and the advantage with the pi HQ camera module is you have full control over the values to set them where you want.
If you are outputing raw files and converting to DNG, you can easily boost an extra 4 stops from base ISO. So I kepp digital gain at 1 and under expose if needed.

dabateman wrote: ↑

Tue Aug 04, 2020 12:00 am

You seem to be confused on what analog and digital gain actually mean.
Analog gain is applied before the Analog to digital conversion process, ie before the AD conversion off the sensor. Digital gain is after and is best to think of it like boosting the EV exposure settings in a raw developing program.
Iso = analog gain x digital gain x 100

So for best dynamic range you want analog gain as low as possible. You could shoot -ag 1 and then boost the exposure with -dg 2 to get decent dynamic range and low noise. Setting analog gain at the maximum will drop the read noise to its minimum. So many astrophotographers will have the digital gain at 1 and the analog gain set to 16.
It depends on what you want and the advantage with the pi HQ camera module is you have full control over the values to set them where you want.
If you are outputing raw files and converting to DNG, you can easily boost an extra 4 stops from base ISO. So I kepp digital gain at 1 and under expose if needed.

No, I know that. As I said the relationship between Iso and gain doesn't seem to be correct here. When I adjusted the exposure time at the same Iso, Picamera automatically chose a lower total gain. This should not happen.
And no, with Picamera I can not adjust the gain as far as I know, not even indirectly by choosing the Iso as the Iso setting doesn't work correctly. And if I would use raspistill, I couldn't turn off the noise reduction as far as I know.

I also just noticed that exposure compensation doesn't work correctly either. When I use exposure_compensation=0, sleep(5), exposure_compensation = -24 , sleep(5) , exposure compensation =0, sleep(5) , the overall gain can be different (for example lower) than if I just use exposure_compensation = 0 , sleep(5) , despite the same light conditions and same (automatic) exposure time! Though this is not reproducible in all light conditions.
I just tried it again: exposure_compensation = 0 , sleep(5)
led to exposure time = 1/30s , analog gain = 6.7 , digital gain = 1, whereas exposure_compensation=0, sleep(5), exposure_compensation = -24 , sleep(5) , exposure compensation =0, sleep(5) led to 1/30s, analog gain = 5.8 digital gain = 1. I think that yesterday I had a more extreme case.

OK, so there's been a bit of cumlative racking of brains trying to remember history about some of this.

Please remember that the camera stack here was originally designed for mobile phones, not DSLRs, so generally behaviour will be closer aligned to those and choices that the mobile phone manufacturers make over how the camera works (and some of them were very exacting).
We're now moving out of those realms with people using the camera modules for far more use cases, but it's still the same underlying code with a few tweaks.

Their main criteria was that preview framerate should remain fluid, which they usually dictated to be 15fps, hence the 66ms max exposure time in most of the exposure modes. Phones are normally hand held, so going for stupidly long exposure times also made little sense due to shake blurring everything.

The switch from preview mode to capture does allow AE/AGC to recompute exposure time (and gains if applicable), so can increase the exposure time should the situation warrant it, however it is still within the parameters defined of the exposure mode, and for having fixed the ISO that still currently means max 66ms.

I'm putting together a patch that will allow the fixed ISO preview modes to run up to 100ms (10fps), and the capture up to 1s. Allowing the recomputation to push exposure time much further than that can give unpredictable results.


Exposure compensation alters the target point for AE/AGC. The algorithm is still active, so it'll change gains and exposure as it sees fit based on the scene, but again still fitting within the profiles defined of the exposure mode.

But what about the issue that a fixed Iso leads to unpredictable gains? At a fixed Iso the total gain can vary significantly, no matter whether I use raspistill or picamera. My understanding is that the total gain should be constant and always the same at a fixed Iso. This is also described by the formula that dabateman posted. Otherwise the jpg brightness would be unpredictable when you set the Iso and the exposure. This shouldn't be the case.
Picamera doesn't let me choose the gain, but choosing the total gain indirectly by setting the Iso does not work correctly due to this issue.

Looking closer, ISO does NOT lock gain during preview. It is fixed for the capture. Your tests are not taking captures.

Run "raspistill -set -ISO 100 -t 2000 -o foo.jpg" and the last log line for the capture will ALWAYS read "analog gain 592/256" in Gain for ISO 100 is calibrated as 2.317, and 2.317*256 is 593. I suspect that is then rounded to the nearest selectable analogue gain on the sensor.

Likewise, put in -ISO 200 and the capture gain is always 1187/256, -ISO 400 is 2362/256, etc.

2.317 * 256 * ISO / 100 (and rounded)

Check the ISO EXIF tag on any JPEG capture, or the MakerNote debug text, and they'll all correctly reflect the chosen ISO.

You can't tell for definite what your DSLR is doing to gain or exposure time except on the capture. As long as the exposure & gain product is the same between preview and capture then you get effectively the same picture.

6by9 wrote: ↑

Tue Aug 04, 2020 4:43 pm

Looking closer, ISO does NOT lock gain during preview. It is fixed for the capture. Your tests are not taking captures.

Yes, they are! Picamera can choose a different total gain at the same Iso. I just reproduced this again by adjusting the exposure time. At Iso 400 1/100s it chose analog gain 7.2 and digital gain 1.0. At Iso 400 1/33s it chose analog gain 5.2 and digital gain 1.0. Measured directly after the capture.

Run "raspistill -set -ISO 100 -t 2000 -o foo.jpg" and the last log line for the capture will ALWAYS read "analog gain 592/256"

I just was able to get 598/256 with raspistill -set -ISO 100 -ss 2500 -t 2000 -o foo.jpg
For Iso 200 I got 1181/256 with -ss 40000
But these are just very small differences, it seems to be much more extreme with Picamera.

As long as the exposure & gain product is the same between preview and capture then you get effectively the same picture.

No, not necessarily when I use picamera. That's why it makes things so complicated and unpredictable.

When I use exposure_compensation=0, sleep(5), exposure_compensation = -24 , sleep(5) , exposure compensation =0, sleep(5) , the overall gain can be different (for example lower) than if I just use exposure_compensation = 0 , sleep(5) , despite the same light conditions and same (automatic) exposure time! Though this is not reproducible in all light conditions.
I just tried it again: exposure_compensation = 0 , sleep(5)
led to exposure time = 1/30s , analog gain = 6.7 , digital gain = 1, whereas exposure_compensation=0, sleep(5), exposure_compensation = -24 , sleep(5) , exposure compensation =0, sleep(5) led to 1/30s, analog gain = 5.8 digital gain = 1

No captures mentioned in there. I can only work with the information you provide.

You haven't stated where you have read analog gain and digital gain from.
EXIF or Makernote are the only really valid places, as preview will have resumed if you're reading it back dynamically with Picamera.

I haven't looked at the documentation regarding exif yet. I think the exposure compensation issue could be a different issue than the ISO / gain issue. All my tests were without exif. I tested exposure_speed , analog_gain , digital_gain either directly before the capture or directly after capture.

Today I used this code in order to measure the Iso/gain issue with Picamera:
If Picamera could not read out the gains that are used for the capture, then I guess, this could make things even more complicated.

Please remember that we did not write Picamera, so it may do things differently to the demo applications we have written (raspistill etc)

But it sounds like 6by9 has hit the nail on the head- you need to check the EXIF for the values you are looking for - we cannot guarantee that Picamera will report the values you think that its reporting.

I think your problem is Picamera. I don't think it knows the HQ camera at all. I think HermanSW is correct and that is the problem.\
I am reading through the code and IMX477 may need to be added to lines 223. to add the sensor modes.
I have added it and will test if better.

But if you read through to line 2950 it indicates in the code that the preview shutter speed limit can't be slower than 33, 333usec due to the framerate attribute. This is even worst than the 1/16 auto limit of raspistill. I have tested this in auto and it does seem to be limited to 1/30 seconds as the slowest shutter speed.

You can set analog gain and digital gain with Picamera though. I have used Bill Williams PiCameraApp GUI to do that.

I think you can turn off noise reduction in raspistill. In the newest firmware you can even turn off the HQ calibration to eliminated all potential of star eating issue. I would stick to raspistill until Picamera can be correctly updated.

dabateman wrote: ↑

Wed Aug 05, 2020 3:55 pm

I think your problem is Picamera. I don't think it knows the HQ camera at all. I think HermanSW is correct and that is the problem.\
I am reading through the code and IMX477 may need to be added to lines 223. to add the sensor modes.
I have added it and will test if better.

I've already corrected HermannSW and asked him to stop spreading FUD. I'll ask you to do the same.

Picamera doesn't need updating for the HQ camera - if it desperately did, then we'd have arranged it before launch.
Since the IMX219 (V2) camera module was introduced, Picamera has used a call to the firmware to query the max resolution of the sensor, and that is about all it needs.
There are a few potential cosmetic updates that would make it nicer, but NONE are essential for the main functionality.

dabateman wrote:But if you read through to line 2950 it indicates in the code that the preview shutter speed limit can't be slower than 33, 333usec due to the framerate attribute. This is even worst than the 1/16 auto limit of raspistill. I have tested this in auto and it does seem to be limited to 1/30 seconds as the slowest shutter speed.

Read the comment.

If you ask a sensor to capture at 30fps (which is what the framerate parameter will do), then you can't expose for longer than the frame time. Simple physics - the sensor can't be exposing two different frames simultaneously.
HOWEVER that's related to video record normally where you want a fixed frame rate. Preview and stills normally doesn't care and allows frame rate to vary fairly widely as required by AE. That's normally denoted by the frame rate request to the firmware being set to 0fps, and then there is a frame rate range parameter to allow further control of that.

Yes,
Fully agree. I figured it out reading through all the code for PiCameraApp, which is what I was using to access Picamera, as its a GUI.
It calls up picamera.
PiCameraApp needed to be edited. It has a fixed max resolution set, by editing in the modes for the HQ camera and changing the max resolution (again this is only in PiCameraApp) then it works.
It has a frame rate that limits exposure settings but default can be changed. It has 1/6, to allow for 6 seconds maximum exposure time. I will have to see if I can edit this as well.

You are correct there isn't a problem with picamera. As long as you stick to HQ settings. Its correctly doing what it should.

This is what was using and wasn't expecting it to have fixed settings:
https://github.com/Billwilliams1952/PiCameraApp

I will not try to spread false information. All this is extremely new to me. I am a hardware guy not software. So reading through the code has helped.

I haven't found a way to add analog gain and digital gain values to the exif data with Picamera. So I wasn't able to confirm the preview/exif theory.

You can set analog gain and digital gain with Picamera though

As far as I know this isn't possible. The Picamera documentation also says that this is not possible. You can adjust the Iso, but this leads to unpredictable results as I have mentioned in my previous posts. Even if it actually chose the same gain at the same Iso (what it doesn't seem to do if you only look at the values that analog_gain provides), you still get unpredictable results when you write an algorithm that adjusts the Iso automatically because Picamera can not output the automatic Iso value and it also can not output the correct gain values (if the preview/exif theory is correct). Adjusting the exposure time automatically (depending on the light conditions) leads to unpredictable results, too because the HQ camera can not tell me the correct exposure in low-light conditions due to the Iso 100 shutter speed bug.

I think you can turn off noise reduction in raspistill.

As far as I know there is no way to turn off noise reduction with raspistill, if you save as a jpg. That's why I currently definitely prefer Picamera.


I have found another issue, which affects raspistill and picamera:
When you turn off auto white balance, but use the same awb gains that auto white balance chose, the colors are different. It looks less saturated than the auto white balance, but it's not just saturation. When you increase the saturation in the manual white balance mode, you still don't get exactly the same colors. Adjusting the awb gains doesn't help either. The color reproduction of the manual white balance mode (auto white balance turned off) was definitely more accurate (if you increase the saturation due to the low default saturation in the manual white balance mode) despite the same awb gains.

I just was able to get the gain in the exif data under "Copyright". I used camera.analog_gain and IFD0.Copyright and then I looked at the metadata.
I still get very different gain values at Iso 100. For example analog gain 2.6 , digital gain 1.0 and then (when I adjust the exposure time, the exposure time was compatible with the framerate) analog gain 1.59 , digital gain 1.0 . Both with camera.iso = 100

I think this measurement is still wrong. I thought that maybe exif_tags is read again in the moment the camera captures the image. Though I haven't found a different solution. The Pi camera exif data don't contain any MakerNote data. Anyway this seems to confirm that analog_gain and digital_gain output useless data , if Picamera does choose the same gain at the same iso (what I don't know at the moment).

I have also tested Iso 200. I got analog gain 5.28 , digital gain 1.0 and then I got (again at iso 200, but different exposure time) analog gain 2.6 , digital gain 1.0

Though I think I can confirm that exposure compensation does not work 100% correctly with Picamera. Choosing a low exposure compensation and then again exposure_compensation = 0 leads to images that are slightly darker compared to when you don't adjust exposure_compensation.

So you've read back the gains for the last preview frame (we've already said preview frames do not have fixed gains) and written them to an EXIF tag. How does that help?

The easiest way to read the debug is "strings foo.jpeg | grep exp", which will give you a line like It is stored under the Makernote tag, but as Makernote has no standard encoding most tools don't know what to do with it.

EXIF tags aren't written if you've used "use_video_port=true" in your capture call - that doesn't actually do a still capture, just encodes a video frame.

6by9 wrote: ↑

Sat Aug 15, 2020 6:53 am

So you've read back the gains for the last preview frame (we've already said preview frames do not have fixed gains) and written them to an EXIF tag. How does that help?

The easiest way to read the debug is "strings foo.jpeg | grep exp", which will give you a line like

Code: Select all

dev=-1 mlux=-1 exp=199998814 ag=256 focus=255 gain_r=3.226 gain_b=1.519 greenness=0 ccm=7706,-3062,-544,-144,5866,-1622,542,-1294,4854,0,0,0 md=0 tg=0 0 oth=0 0 b=0 f=0 0 fi=0 ISP Build Date: May 20 2020, 15:43:01 VC_BUILD_ID_VERSION: 4dad1450c8ad35f26f65cc4be82d87ead2bd7e21 (tainted) VC_BUILD_ID_USER:  VC_BUILD_ID_BRANCH: 2711_2 

It is stored under the Makernote tag, but as Makernote has no standard encoding most tools don't know what to do with it.

EXIF tags aren't written if you've used "use_video_port=true" in your capture call - that doesn't actually do a still capture, just encodes a video frame.

Thank you, this worked, no exif viewer could see the data. Yes, I realized that my measurement was not better than my previous measurement. I thought that maybe Picamera.capture() opens the Py file and looks for all exif definitions again maybe...

So now my first impression is that the analog gain seems to be constant at the same Iso.

Now I think I realize what the issue really is: The preview does not work correctly!
Sometimes the preview ignores your iso and lowers the preview gain in order to give you a preview that isn't overexposed.
But sometimes it doesn't ignore the iso (for example high Iso) and can produce an overexposed preview although it could lower the gain. This seems to affect raspistill and Picamera.
It seems to be unpredictable which preview brightness the preview wants to achieve. Therefore when I look at the analog gain and digital gain values during the preview, it might be unpredictable for me to calculate which gain and exposure combinations are possible in order to get the correct jpg brightness! This leads to inconsistent results! Though maybe the preview doesn't suffer from this issue, if shutter speed and iso are not set.
Though even then you would still have the exposure compensation issue, which I have already mentioned:
When you decrease the exposure compensation value and then chose again exposure_compensation = 0 and wait several seconds, then the preview and the jpg are slightly darker than a preview/jpg where you didn't adjust the exposure compensation at all. Again this leads to inconsistent results! I just noticed that sometimes it takes approximately 8 seconds from exposure compensation = -15 to exposure compensation = 0, but in some other cases (for example something like exposure compensation = -15 , then exposure compensation = 15, then exposure compensation = 0) it doesn't find the correct exposure compensation = 0 setting at all, not even after 8 seconds, doesn't make sense in my opinion.
As the exposure compensation does not work correctly, I could maybe write my own exposure compensation algorithm, but then I have the issue that I can not control the preview, so I might not be able to preview the adjustments.
Hopefully, the libcamera implementation won't suffer from the same issues.

Another thing I noticed: The exif data lists a tg value, which stands for "total digital gain" according to
viewtopic.php?t=194093 . I noticed that the tg values of Picamera photos can change a lot at the same Iso, for example at Iso 100 with ag=592 I was able to get tg values between 8000 and 185. But the overall gain shouldn't change at the same Iso. Then I compared the jpgs with raspistill with the same Iso and same exposure time and fortunately the brightness differences were not large (although raspistill always reported a digital gain of 256/256), once there wasn't a brightness difference at all. But mostly the Picamera photos were slightly brighter.