User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

High Quality Camera RAW Examples

Fri May 08, 2020 9:11 pm

Hello

As I await the arrival of my module to arrive, does anybody have any test images with RAW they've shot ( or are willing to shoot ) with the bayer image attached? Using the new High Quality Camera Module

Using raspistill command:

Code: Select all

raspistill --raw -o image.jpg

ethanol100
Posts: 636
Joined: Wed Oct 02, 2013 12:28 pm

Re: High Quality Camera RAW Examples

Sat May 09, 2020 9:42 am

I took one example and uploaded it to github.
This is taken with a Canon 50mm lens and a 3d printed C-to-EF mount adapter.

I used the command you specified. Please let me now, if you need something else, like flips/rotation or similar.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High Quality Camera RAW Examples

Sat May 09, 2020 10:47 am

I'll try to dig out my updated version of dcraw which handles 12bit Bayer :-)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: High Quality Camera RAW Examples

Sat May 09, 2020 10:54 am

6by9 wrote: I'll try to dig out my updated version of dcraw which handles 12bit Bayer :-)
That would be awesome! :D
Working on updating my python utility to be able to work with the new camera module.
Any idea as for the specific offsets or header lengths to be aware of? I'm working in reverse from the code examples and explanation given in the picamera bayer documentation. ( i.e. look for the BRCM, etc.. ) plugging in the new sensor dimensions and higher bit depth.
ethanol100 wrote: I took one example and uploaded it to github.
This is taken with a Canon 50mm lens and a 3d printed C-to-EF mount adapter.

I used the command you specified. Please let me now, if you need something else, like flips/rotation or similar.
This is much appreciated, Thank you! I'm also looking forward to giving this camera a whirl with my various EF lenses ;)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High Quality Camera RAW Examples

Sat May 09, 2020 2:15 pm

schoolpost wrote:
Sat May 09, 2020 10:54 am
6by9 wrote: I'll try to dig out my updated version of dcraw which handles 12bit Bayer :-)
That would be awesome! :D
Working on updating my python utility to be able to work with the new camera module.
Any idea as for the specific offsets or header lengths to be aware of? I'm working in reverse from the code examples and explanation given in the picamera bayer documentation. ( i.e. look for the BRCM, etc.. ) plugging in the new sensor dimensions and higher bit depth.
It appears I'd already commited the code for parsing raw12 (and 14, 16, and 8) back in July 2017
The offsets are

Code: Select all

        const long offsets[] = {
                //IMX477 offsets
                3375104,  //2028x1080
                4751360,  //2028x1520
                18711040, //4056x3040
                1015808,  //1012x760
but I'm now having to work out why I'm getting totally black images. Unlikely to be before Monday as it doesn't appear to be an obvious one.

Pushed to https://github.com/6by9/dcraw, but giving black images for me. If I take the raw off the end (eg tail -c 18711040 imx477_md3.jpg >imx477_md3.raw) and process that through dcraw, then it comes out OK, so it's something about the detection.

Edit: Issue found. Part of the standard dcraw code was checking

Code: Select all

  if (!strcmp(make,"Sony") && raw_width > 3888)
    black = 128 << (tiff_bps - 12);
Seeing as we met the two criteria (at least for the 12MP mode) we got our black level set to a daft value, and everything came out black.
I've not pushed that change yet as it has potential to mess other things up.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High Quality Camera RAW Examples

Sat May 09, 2020 2:46 pm

OK, told a lie. Fixed, and now has a sensible black level set based on the sensor (and mode in the case of imx477).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ethanol100
Posts: 636
Joined: Wed Oct 02, 2013 12:28 pm

Re: High Quality Camera RAW Examples

Sat May 09, 2020 3:33 pm

Thank you, I can confirm your version of dcraw works fine now.

User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: High Quality Camera RAW Examples

Sun May 10, 2020 12:21 am

Little bit of trial and error but eventually we get the final result. Reading the 12bit raw bayer, packing it into DNG format with most of the necessary tags and metadata using a python script, here is what we get! :)

Now colors need work, for my script PyDNG ( https://github.com/schoolpost/PyDNG ) I've been using CCM that I've basically just scoured the internet for and found to work well on the older camera modules. ( nothing scientific or accurate about that )

Libcamera does provide some of calibration data as apart of the library, including CCM's under various illuminants. So maybe those can be used here? I've actually already tried that with the IMX219 and using those CCM values into the ColorMatrix1 and ColorMatrix2 tags used by the DNG format, but no avail the colors appeared way off.

I'll be upgrading PyDNG soon for those who would like to test or help tweak.

Image

xkubazz
Posts: 19
Joined: Sun May 10, 2020 3:18 pm

Re: High Quality Camera RAW Examples

Sun May 10, 2020 3:41 pm

I found that Photoshop's "Auto Tone" function works really well on RPi HQ Camera RAW files.

JPEG straight out of camera:
Image

RAW after conversion to 16-bit TIFF using 6by9's fork of dcraw:
Image

TIFF after applying "Auto Tone" in Photoshop:
Image

It is a great staring point for applying further colour grading. It is not perfect solution as I want command line (and preferably open source) tool to do this to streamline my workflow, but I didn't find anything that can produce such good results so far.
If you have any ideas what could work well, please share.
Astrophotography using Raspberry Pi HQ Camera
https://terramex.neocities.org/astro/

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 46
Joined: Tue Jan 07, 2020 9:15 am

Re: High Quality Camera RAW Examples

Sun May 10, 2020 8:19 pm

Libcamera does provide some of calibration data as apart of the library, including CCM's under various illuminants. So maybe those can be used here? I've actually already tried that with the IMX219 and using those CCM values into the ColorMatrix1 and ColorMatrix2 tags used by the DNG format, but no avail the colors appeared way off.
If you look in those JSON camera tuning files that we ship with libcamera you should be able to find both the gains you need to white balance (WB) the image correctly, and the colour matrix to make the colours look right.

For the WB gains, search for "ct_curve". That field gives a list of numbers where each triple consists of: the colour temperature (K), the grey response of the sensor for red light (so it's reciprocal is the red gain you need) and the response for blue light (so again the reciprocal gives the blue gain).

For colour matrices, search for "ccms". Again these are listed by colour temperature.

User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: High Quality Camera RAW Examples

Sun May 10, 2020 10:23 pm

therealdavidp wrote:
Sun May 10, 2020 8:19 pm
Libcamera does provide some of calibration data as apart of the library, including CCM's under various illuminants. So maybe those can be used here? I've actually already tried that with the IMX219 and using those CCM values into the ColorMatrix1 and ColorMatrix2 tags used by the DNG format, but no avail the colors appeared way off.
If you look in those JSON camera tuning files that we ship with libcamera you should be able to find both the gains you need to white balance (WB) the image correctly, and the colour matrix to make the colours look right.

For the WB gains, search for "ct_curve". That field gives a list of numbers where each triple consists of: the colour temperature (K), the grey response of the sensor for red light (so it's reciprocal is the red gain you need) and the response for blue light (so again the reciprocal gives the blue gain).

For colour matrices, search for "ccms". Again these are listed by colour temperature.
Thanks for the explanation! these calibration values and obtaining them and using them correctly isn't exactly trivial.

I've reffered back to articles like these to help make some sense of it: http://www.odelama.com/photo/Developing ... nd_Part-2/

From the article above, the DNG format benefits from having these bits of calibration/metadata information:

CalibrationIlluminant1 https://www.awaresystems.be/imaging/tif ... nant1.html
CalibrationIlluminant2 https://www.awaresystems.be/imaging/tif ... nant2.html
ColorMatrix1 https://www.awaresystems.be/imaging/tif ... trix1.html
ColorMatrix2 https://www.awaresystems.be/imaging/tif ... trix2.html
ForwardMatrix1
ForwardMatrix2
ReductionMatrix1https://www.awaresystems.be/imaging/tif ... trix1.html
ReductionMatrix2 https://www.awaresystems.be/imaging/tif ... trix2.html
CameraCalibration1 https://www.awaresystems.be/imaging/tif ... tion1.html
CameraCalibration2 https://www.awaresystems.be/imaging/tif ... tion2.html

The CCM I find in the JSON files maps pretty straight forward, CCM->ColorMatrix1 and I speicify the Calibration illuminant that most closely matches the color temperature for that CCM. repeat for ColorMatrix2

As for how I can use the rest of the values, to fill out the rest of those DNG tags I've listed above isn't exactly clear. It's not requred that you have all those tags for a workable DNG raw image file, but whatever information I attach to the DNG to help the software interpet the RAW values the better it is, I sense.

What I've done in the meantime is snoop the DNG metadata for a Mavic Air Drone that uses the simillar IMX477 and used its metadata for the time being and I found the colors do look alright more or less. But then again I dont think "looks good" is a very scientific approach so I'd really dial in these values properly.

Original JPEG
Image

CCM's used from DJI Drone DNG file
Image

CCM's from the IMX477 JSON
Image

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 46
Joined: Tue Jan 07, 2020 9:15 am

Re: High Quality Camera RAW Examples

Mon May 11, 2020 8:03 am

CCM's from the IMX477 JSON
Hmm, the image you include there really looks to me like no CCM has been applied at all. It has that de-saturated look that is typical if you don't correct the sensor's RGB values. The trouble is, without knowing exactly what processing has gone on, it's hard to understand the results.

The one thing I can tell you, though, are the main steps involved in the ISP as regards colour processing. We have:

1. Black level removal.
2. Colour shading correction (actually this is a very small effect with imx477 as there is minimal colour shading, though it's quite a big effect on imx219).
3. White balancing. This applies gains to the red and blue channels.
4. Colour matrix.
5. Gamma transform.

There is other processing, of course, but these are really the ones that affect the final colours. In the libcamera world you can alter or disable any of these stages and see the effect just by editing the JSON file, though not when the imaging pipeline is under the control of the proprietary GPU firmware.

User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: High Quality Camera RAW Examples

Tue May 12, 2020 6:23 pm

As a sanity check, I'm looking for clarification on bitpacking scheme for the 12 bit RAW image data.

My understanding currently puts 2 pixels of image data into 3 bytes in this order:

AAAAAAAA BBBBBBBB AAAABBBB

However given the nature of the least significant bits and their influence on the image I could be wrong and it could really be this:

AAAAAAAA BBBBBBBB BBBBAAAA

Which is the correct packing? Thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8667
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High Quality Camera RAW Examples

Tue May 12, 2020 7:59 pm

Aiui the second is correct
https://www.kernel.org/doc/html/v4.17/m ... gb12p.html

This is where I'm hoping I have dcraw correct.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Jack Hogan
Posts: 9
Joined: Wed May 13, 2020 2:08 pm
Contact: Website

Re: High Quality Camera RAW Examples

Wed May 13, 2020 2:31 pm

Hi all, new member and pi noob, really excited about the new interchangeable lens development.

Just wanted to let whoever was interested know that a Compromise Color Matrix is in the MakerNote of every -r raw jpg. It takes black subtracted demosaiced data from white balanced camera space to sRGB linearly, so first the raw data needs to be white balanced with the r and b gains (also in the MakerNote), then the matrix applied, then gamma.

The matrix appears to vary based on the illuminant, so they must be doing some interpolating/tweaking before writing it to the the file. It is in dcraw format, just divide by 10000 and normalize. This is what a MakerNote looks like:

Code: Select all

Maker Note: ev=-1 mlux=-1 exp=36156 ag=256 focus=255 gain_r=3.175 gain_b=1.726 greenness=0 ccm=8450,-3678,-672,-672,6402,-1630,326,-1982,5758,0,0,0 md=0 tg=257 257 oth=257 262 b=0 f=257 262 fi=0 ISP Build Date: Feb 12 2020, 12:39:13 VC_BUILD_ID_VERSION: 53a54c770c493957d99bf49762dfabc4eee00e45 (clean) VC_BUILD_ID_USER: dom VC_BUILD_ID_BRANCH: master 
Using Matlab to extract the raw data this is the result after rendering, with a touch of local contrast and sharpening:

Image

It's in Adobe RGB, so if your browser is not properly color managed the tones will be off. Chrome seems to show it well.

Jack
PS: I can confirm that the order of the 4 LSBs is the opposite of what one would initially think.
Last edited by Jack Hogan on Thu May 14, 2020 12:14 pm, edited 2 times in total.

Jack Hogan
Posts: 9
Joined: Wed May 13, 2020 2:08 pm
Contact: Website

Re: High Quality Camera RAW Examples

Thu May 14, 2020 7:52 am

Hi,

To clarify my previous post, the black level should be subtracted from 12-bit unpacked raw data before applying white balance gains, demosaicing, the matrix and gamma. The black level is around 256.0-257.3 12-bit raw DN, depending on analog gain (to get analog gain, divide ag by 256 in the MakerNote). With a gain of 1 the black level is about 256.25, the average in the available 1-16 gain range is 256.664 DN, both figures with my unit at room temperature
schoolpost wrote:
Tue May 12, 2020 6:23 pm
As a sanity check, I'm looking for clarification on bitpacking scheme for the 12 bit RAW image data.

My understanding currently puts 2 pixels of image data into 3 bytes in this order:

AAAAAAAA BBBBBBBB AAAABBBB

However given the nature of the least significant bits and their influence on the image I could be wrong and it could really be this:

AAAAAAAA BBBBBBBB BBBBAAAA

Which is the correct packing? Thanks!

AAAAAAAA BBBBBBBB BBBBAAAA, this is how I do it in Matlab, should be self explanatory:

Code: Select all

s.cfa = zeros(c*r,1,'uint16');
    s.cfa(1:2:end) = bitshift(bin(1:3:end),4) + bitand(bin(3:3:end),15);
    s.cfa(2:2:end) = bitshift(bin(2:3:end),4) + bitshift(bin(3:3:end),-4);
    s.cfa = reshape(s.cfa,[],r)';                   %12 bit raw in uint16
I originally did it the other way around, which resulted in some funny looking black frame histograms, now they look right.

Jack

meantux
Posts: 1
Joined: Thu Apr 25, 2019 4:01 am

Re: High Quality Camera RAW Examples

Sat May 23, 2020 6:00 am

Hi, I'm experimenting with PyDNG

Context: I'm looking for a better solution to scan films (my project is the development of the Gugusse Roller)

Camera: the Raspberry Pi HQ camera.
Samples:
20 frames (dng): www.deniscarl.com/samples/exposedok.zip
20 frames (dng, from same film but slightly overexposed): www.deniscarl.com/samples/saturationSample.zip

As a photographer, I'm used to post-process raws with lightroom, and because of my film scanner project I also have some experience of post-processing 12 bits tifs with DaVinci Resolve.

When I post-process the DNGs from PyDNG coming out of the HQ camera, like in 'cam.capture("00000.jpg", bayer=True)', then use the RPICAM2DNG.convert("fn.jpg", process=False, compress=False) I don't feel like I'm post-processing raws.. when I try to blow out the dark areas, they pixelize rapidly, much like jpegs... when I reduce the gain to recuperate the saturated areas, I'm not recuperating anything.... Maybe we get more bits, more values between the dark and the light, but we're not getting more dynamic range, it's not extending more in the dark and more in the light...

That's what I feel.... Am I doing something wrong?... is there a step that I missed to set my sensor in 12bits..... is there a step to set the sensor in some sort of d-log mode?

Here some samples coming from another camera (arducam 10MP monochrome camera, captured in 12 bits, trichromy and re-assembled with ridiculously simple opencv calls) 10 frames as 16bits tifs, www.deniscarl.com/samples/ArducamTrichr ... 12Bits.zip
With these I can recuperate a bit of data lost in the dark and in the clear.. it feels more like working with 12 bits...

Help me make the HQ Raspberry Pi camera work, I'll rather use it then having to go back to the slower usb arducam kit....

Thank you for your attention.

xkubazz
Posts: 19
Joined: Sun May 10, 2020 3:18 pm

Re: High Quality Camera RAW Examples

Sat May 23, 2020 11:46 am

When I post-process the DNGs from PyDNG coming out of the HQ camera (...) I don't feel like I'm post-processing raws
I'm not PyDNG author, but tried using it and noticed that it outputs 8-bit DNGs by default. Didn't have time to look at sources and just switched to dcraw.

Code: Select all

./dcraw -T -6 -W -r 3.0 1.0 1.49 1.0 -g 1 1 filename.jpg
Gives you 16-bit linear TIFF with 12-bit data inside.
Removing -W it will auto adjust exposure, removing -g 1 1 will apply default gamma curve, adding -a will auto adjust white balance.

Used it on JPGs captured with raspistill with --raw option.
Astrophotography using Raspberry Pi HQ Camera
https://terramex.neocities.org/astro/

User avatar
schoolpost
Posts: 32
Joined: Sun Feb 19, 2017 10:47 am
Location: Canada
Contact: Website

Re: High Quality Camera RAW Examples

Thu May 28, 2020 1:53 am

meantux wrote:
Sat May 23, 2020 6:00 am
Hi, I'm experimenting with PyDNG

Context: I'm looking for a better solution to scan films (my project is the development of the Gugusse Roller)

Camera: the Raspberry Pi HQ camera.
Samples:
20 frames (dng): www.deniscarl.com/samples/exposedok.zip
20 frames (dng, from same film but slightly overexposed): www.deniscarl.com/samples/saturationSample.zip

As a photographer, I'm used to post-process raws with lightroom, and because of my film scanner project I also have some experience of post-processing 12 bits tifs with DaVinci Resolve.

When I post-process the DNGs from PyDNG coming out of the HQ camera, like in 'cam.capture("00000.jpg", bayer=True)', then use the RPICAM2DNG.convert("fn.jpg", process=False, compress=False) I don't feel like I'm post-processing raws.. when I try to blow out the dark areas, they pixelize rapidly, much like jpegs... when I reduce the gain to recuperate the saturated areas, I'm not recuperating anything.... Maybe we get more bits, more values between the dark and the light, but we're not getting more dynamic range, it's not extending more in the dark and more in the light...

That's what I feel.... Am I doing something wrong?... is there a step that I missed to set my sensor in 12bits..... is there a step to set the sensor in some sort of d-log mode?

Here some samples coming from another camera (arducam 10MP monochrome camera, captured in 12 bits, trichromy and re-assembled with ridiculously simple opencv calls) 10 frames as 16bits tifs, www.deniscarl.com/samples/ArducamTrichr ... 12Bits.zip
With these I can recuperate a bit of data lost in the dark and in the clear.. it feels more like working with 12 bits...

Help me make the HQ Raspberry Pi camera work, I'll rather use it then having to go back to the slower usb arducam kit....

Thank you for your attention.

Having reviewed your frames, I'm able to recover a fair amount of highlight detail. Even so with the shadows, maybe less than with the highlights but still able to recover some detail. ( Using Adobe Camera RAW )

There is no processing happening with PyDNG, simply unpacking the 12-bit RAW data and placing into a DNG file ( with relevant metadata tags ) these metadata tags have been setup to the best of my abillity but this is where further improvements with PyDNG can be realized. ( DNG outlines lots of tags you can add that helps the RAW interpreter better process your image )

Your comparsion with the HQ cam and the arducam isn't apples to apples....I take it with a monochrome sensor and tricrhomatic process you gain full resolution color images per channel theryby gaining alot of color and image fidelity over that of a bayer sensor....If I've understood correctly.

Return to “Camera board”