RAW output information

[6by9]

I've just been searching for what data is available for OV5647 in the public domain and stumbled across https://android.googlesource.com/kernel ... 5647_reg.c which includes "/*2608*1952 Reference Setting 24M MCLK 2lane 280Mbps/lane 30fps*/"

I've not analysed it, and I very much doubt they really get 5MP @ 30fps as the sensor can't do it, but the resolution implies it isn't cropping out the masked pixels.

[blindbloom]

I assumed the masked sensor data was being included in the RAW file in my post of June 10, where I thought 2608 sensors per row, with 8 masked sensors on each side (8 + 2592 + 8 = 2608). But others have since said that the "padding" in the byte stream between rows was uninitialized memory (i.e. garbage). If you want to see a sample, here is one (jbeale's RAW-GMB-Nov2014.jpg):

http://www.brucelindbloom.com/index.htm ... sorNew.png

This is a 16 bit per pixel grayscale PNG image (about 6MB) showing the sensor data. It is 2608 x 1956 pixels with 8 masked columns on the left and right and 6 masked rows at the top and bottom. Look closely at the "masked" sensors around the edge and you will see that they are mostly black, but there are places where it looks like garbage. Using these for black-level compensation would be dangerous because of the garbage. Therefore, I am proceeding with the understanding (hope!) that black level has already been applied or that the data was captured in a way that does not require this compensation.

[jbeale]

Therefore, I am proceeding with the understanding (hope!) that black level has already been applied or that the data was captured in a way that does not require this compensation.

Before any heads explode... we are told that black level subtraction has not been done. It's the raw data (apart from the edge pixel garbage). "Does not require compensation" is in the eye of beholder, but it is what it is.

By the way, I didn't realize blindbloom was *the* Bruce Lindbloom with the amazing color science web page. Well worth a look for those interested!

[jamesh]

jbeale is right - the raw is straight off the sensor (not including the borders - they are not loaded from the sensor), so you will need to do a lot of the correction implied by the two lists above to get a decent image.

[bablokb]

I have stated many times that the Bayer order is present in the raw header, but nobody seems to be bothered to investigate. Try about byte 68 into the raw header. The enum values are even in the userland repo https://github.com/raspberrypi/userland ... pes.h#L144 Process that correctly and you can then even deal with deliberate flips.

Hm, I did investigate. When I dump the header, I see only hex zero at about byte 68, regardless of vertical or horizontal flips.

I use the following script to dump the header (you can pass the filename and optionally an offset into the header. As a third parameter you can pass the length, which defaults to the complete header size).

Bernhard

Code: Select all

#!/bin/bash

RAWBLOCKSIZE=6404096
HEADERSIZE=32768

infile="$1"
offset="${2:-0}"
length="${3:-$HEADERSIZE}"

filesize=$(stat -c "%s" "$infile")
let start=filesize-RAWBLOCKSIZE+offset

dd if="$infile" iflag=skip_bytes,count_bytes \
                skip=$start count=$length 2>/dev/null | hexdump -C

[6by9]

Hm, I did investigate. When I dump the header, I see only hex zero at about byte 68, regardless of vertical or horizontal flips.

I use the following script to dump the header (you can pass the filename and optionally an offset into the header. As a third parameter you can pass the length, which defaults to the complete header size).

I corrected myself:
viewtopic.php?f=43&t=44918&start=150#p773451

Sorry, I gave a duff offset before for the Bayer order - the structure I was looking at was embedded within another. It's more likely to be around byte 244 into the header.

blindbloom had picked up on this at viewtopic.php?f=43&t=44918&start=150#p773622 and wasn't complaining with an offset of 244 (though his code only handles 2 of the 4 options), so 244 is probably right. I haven't checked in an actual file.

edit: checked actual files and something isn't right there, so apologies. This does stir recollections of an issue we had before, so I'll investigate.

[bablokb]

I corrected myself:
viewtopic.php?f=43&t=44918&start=150#p773451

Sorry, I gave a duff offset before for the Bayer order - the structure I was looking at was embedded within another. It's more likely to be around byte 244 into the header.

blindbloom had picked up on this at viewtopic.php?f=43&t=44918&start=150#p773622 and wasn't complaining with an offset of 244 (though his code only handles 2 of the 4 options), so 244 is probably right. I haven't checked in an actual file.

edit: checked actual files and something isn't right there, so apologies. This does stir recollections of an issue we had before, so I'll investigate.

I also checked my files, and I see byte 242 (offset 241) as a candidate:
normal orientation: 0
horizontal flip: 1
vertical flip: 2
hf+vf: 3

I only have 4 files (they are on github:https://github.com/bablokb/raspiraw/tree/master/images) to check, so this is not a scientific proof.

BTW my script is wrong (off by one), it should be

Code: Select all

let start=filesize-RAWBLOCKSIZE-1+offset

[6by9]

Lawyers be damned. The slightly sanitised start of the struct of interest is:

Code: Select all

   uint8_t name[32]; /**< short text description of the mode, e.g. "720P" */
   uint16_t width;               /**< width of frame in pixels */
   uint16_t height;              /**< height of frame in pixels */
   uint16_t padding_right;       /**< additional non-data pixels to the right of the frame */
   uint16_t padding_down;        /**< additional non-data pixels below the frame */
   uint32_t 
   uint32_t 
   uint32_t 
   uint32_t 
   uint32_t 
   uint16_t 
   uint16_t 
   uint16_t transform;     /**< Possible transformations in this mode / user requested transformations */
   uint16_t format;       /**< Image format of the frame. */
   uint8_t bayer_order;   /**< If a Bayer format frame, the Bayer order in the frame. */
   uint8_t bayer_format;  /**< If a Bayer format frame, the packing mode of the data. */

name will be "2592x1944" and 23 NULLs.
width is 2592, or 0xa20
height is 1944, or 0x798.
No padding required on this sensor.
So I think you're looking at transform at offset 241. That will change, but does not represent the base Bayer order.
bayer_order is the correct field to look at, and should be an enum value from interface/vctypes/vc_image_types.h

Code: Select all

typedef enum
{ //defined to be identical to register bits
   VC_IMAGE_BAYER_RGGB     = 0,
   VC_IMAGE_BAYER_GBRG     = 1,
   VC_IMAGE_BAYER_BGGR     = 2,
   VC_IMAGE_BAYER_GRBG     = 3
} VC_IMAGE_BAYER_ORDER_T;

My captures all had it set to 1, or VC_IMAGE_BAYER_GBRG which is the setting before the transform. The layer above the camera driver corrects the order based on the flips and then tells the sensor the flips to apply.

As I said, this does stir memories from when the team trying to do automated camera tuning were analysing raws, and the setting wasn't as required. I'm just checking history on that branch to see if we are missing some diffs. Commit:

SW-11720: Write correct camera mode from write_raw_md_stage
Write the camera mode passed in switch_mode and not camera
calibration metadata.

looks highly plausible.
Sorry for sending people on a wild goose chase - I sometimes forget quite how many things never made it onto the Pi branch I'll look at getting that commit cherry-picked and released.

[6by9]

Patches found and applied to my local tree. Having done a set of captures I can do:

Code: Select all

pi@raspberrypi ~ $ ./raw_header.sh no_flip.jpg 244 16
00000000  01 03 0a 00 00 00 00 00 
pi@raspberrypi ~ $ ./raw_header.sh h_flip.jpg 244 16
00000000  02 03 0a 00 00 00 00 00  02 00 00 00 10 00 00 00  |................|
pi@raspberrypi ~ $ ./raw_header.sh v_flip.jpg 244 16
00000000  00 03 0a 00 00 00 00 00  02 00 00 00 10 00 00 00  |................|
pi@raspberrypi ~ $ ./raw_header.sh hv_flip.jpg 244 16
00000000  03 03 0a 00 00 00 00 00  02 00 00 00 10 00 00 00  |................|

So:
- no flips is GBRG
- h flip is BGGR
- v flip is RGGB
- h&v flip is GRBG
which all looks correct to me. I've got a small set of patches to push to Dom, so I'll roll these in as well.

If someone were really feeling like updating dcraw, then they can now handle all the correct modes. You've also now got the location for the width and height in the height, so you could even get it to work with captures where the mode has been forced. Just remember that the line stride is ALIGN_UP(ALIGN_UP(width,32) * 10/8, 32) for Bayer raw 10 (other formats are available but aren't used on OV5647 driven by the GPU).

[blindbloom]

This is all great information! In most (all?) DSLR raw formats, the Bayer order is encoded as four numbers, for example 0, 1, 1, 2 means RGGB. For RPi RAW, it is an enumerated type! This enumeration also works for the "old firmware" images (I know, who cares?). In an attempt to document the known parts of the header, I have updated my example:

Code: Select all

RawDecodeExample()
{
	static const char rawFilename[] = "D:/RaspberryPi/RAW-GMB-Nov2014.jpg";	// from jbeale's site
	//static const char rawFilename[] = "D:/RaspberryPi/RAW0029.jpg";	// from jbeale's site
	static const int knSizeX = 2592;
	static const int knSizeY = 1944;
	static const int knEndOfRowPadding = 24;	// padding at end of each row
	static const int knBytesPerSet = 5;
	static const int knPixelsPerSet = 4;
	static const int knStrideY = knBytesPerSet * knSizeX / knPixelsPerSet + knEndOfRowPadding;	// in bytes
	static const int knHeaderSize = 32768;
	static const int knEndOfImagePadding = 8 * knStrideY;
	static const int knRawDataSize = knHeaderSize + knSizeY * knStrideY + knEndOfImagePadding;

	// offsets into RAW header, relative to its start
	static const int knIdOffset = 0;				// char[4]
	static const int knSensorDataOffset = 8;		// uint32_t
	static const int knVersionOffset = 16;			// char[128?]
	static const int knNameOffset = 176;			// char[32]
	static const int knWidthOffset = 208;			// uint16_t
	static const int knHeightOffset = 210;			// uint16_t
	static const int knPaddingRightOffset = 212;	// uint16_t
	static const int knPaddingDownOffset = 214;		// uint16_t
	static const int knTransformOffset = 240;		// uint16_t
	static const int knImageFormatOffset = 242;		// uint16_t
	static const int knBayerOrderOffset = 244;		// BYTE
	static const int knBayerFormatOffset = 245;		// BYTE
	static const int knXOffset = 420;				// char[4]

	// open the raw file
	FILE* fd = NULL;
	fopen_s(&fd, rawFilename, "rb");

	// seek to the header
	fseek(fd, -knRawDataSize, SEEK_END);

	// read the beginning of the header
	static const int knPartialHeaderSize = 512;
	BYTE auHeader[knPartialHeaderSize];
	fread(auHeader, sizeof(BYTE), sizeof(auHeader), fd);

	// get the offset to the sensor data
	UINT uOffset =
		(static_cast<UINT>(auHeader[knSensorDataOffset + 3]) << 24) +
		(static_cast<UINT>(auHeader[knSensorDataOffset + 2]) << 16) +
		(static_cast<UINT>(auHeader[knSensorDataOffset + 1]) << 8) +
		(static_cast<UINT>(auHeader[knSensorDataOffset]));

	// extract encoded metadata
	int nEncodedWidth = (static_cast<int>(auHeader[knWidthOffset + 1]) << 8) + static_cast<int>(auHeader[knWidthOffset]);
	int nEncodedHeight = (static_cast<int>(auHeader[knHeightOffset + 1]) << 8) + static_cast<int>(auHeader[knHeightOffset]);
	int nEncodedPaddingRight = (static_cast<int>(auHeader[knPaddingRightOffset + 1]) << 8) + static_cast<int>(auHeader[knPaddingRightOffset]);
	int nEncodedPaddingDown = (static_cast<int>(auHeader[knPaddingDownOffset + 1]) << 8) + static_cast<int>(auHeader[knPaddingDownOffset]);
	int nEncodedTransform = (static_cast<int>(auHeader[knTransformOffset + 1]) << 8) + static_cast<int>(auHeader[knTransformOffset]);
	int nEncodedImageFormat = (static_cast<int>(auHeader[knImageFormatOffset + 1]) << 8) + static_cast<int>(auHeader[knImageFormatOffset]);
	BYTE uBayerOrder = auHeader[knBayerOrderOffset];
	BYTE uBayerFormat = auHeader[knBayerFormatOffset];

	// make sure the header looks ok, as far as we're able to check it
	assert(uOffset == knHeaderSize - 4);	// offset - 4 of sensor data, relative to the start of the header
	assert(strncmp(reinterpret_cast<const char*>(&auHeader[knIdOffset]), "BRCM", 4) == 0);
	assert(strncmp(reinterpret_cast<const char*>(&auHeader[knVersionOffset]), "ov5647 version 0.1", 18) == 0);
	assert(strncmp(reinterpret_cast<const char*>(&auHeader[knNameOffset]), "2592x1944", 9) == 0);
	assert(strncmp(reinterpret_cast<const char*>(&auHeader[knXOffset]), "xxxx", 4) == 0);
	assert(nEncodedWidth == knSizeX);
	assert(nEncodedHeight == knSizeY);
	assert(uBayerOrder <= 3);

	// print some metadata
	fprintf(stdout, "Image Width =  %d\n", nEncodedWidth);
	fprintf(stdout, "Image Height =  %d\n", nEncodedHeight);
	fprintf(stdout, "Padding Right = %d\n", nEncodedPaddingRight);
	fprintf(stdout, "Padding Down = %d\n", nEncodedPaddingDown);
	fprintf(stdout, "Transform = %d (0x%04x)\n", nEncodedTransform, nEncodedTransform);
	fprintf(stdout, "Image Format = %d (0x%04x)\n", nEncodedImageFormat, nEncodedImageFormat);
	fprintf(stdout, "Bayer Order = %s\n",
		(uBayerOrder == 0) ? "RG/GB" :
		(uBayerOrder == 1) ? "GB/RG" :	// new firmware
		(uBayerOrder == 2) ? "BG/GR" :	// old firmware
		(uBayerOrder == 3) ? "GR/BG" : "unrecognized");
	fprintf(stdout, "Bayer Format = %d (0x%02x)\n", uBayerFormat, uBayerFormat);
	fflush(stdout);

	// seek to the sensor data
	fseek(fd, uOffset + 4 - knPartialHeaderSize, SEEK_CUR);

	// create an image for the Bayer data
	GenImage image(knSizeX, knSizeY, keGray);

	for (int nY = 0; nY < knSizeY; nY++)
	{
		for (int nX = 0; nX < knSizeX;)
		{
			BYTE auData[knBytesPerSet];	// every set of five bytes encodes four, 10-bit pixels (5 x 8 bits = 4 * 10 bits = 40 bits)
			fread(auData, 1, knBytesPerSet, fd);

			// compute the 10-bit sensor values, in the range [0, 1023]
			UINT uV0 = (static_cast<UINT>(auData[0]) << 2) + (static_cast<UINT>(auData[4] & 0x03) >> 0);
			UINT uV1 = (static_cast<UINT>(auData[1]) << 2) + (static_cast<UINT>(auData[4] & 0x0C) >> 2);
			UINT uV2 = (static_cast<UINT>(auData[2]) << 2) + (static_cast<UINT>(auData[4] & 0x30) >> 4);
			UINT uV3 = (static_cast<UINT>(auData[3]) << 2) + (static_cast<UINT>(auData[4] & 0xC0) >> 6);

			// write the sensor values, scaling from the range [0, 1023] to [0.0, 1.0]
			image.Write(nX++, nY, uV0 / 1023.0f);
			image.Write(nX++, nY, uV1 / 1023.0f);
			image.Write(nX++, nY, uV2 / 1023.0f);
			image.Write(nX++, nY, uV3 / 1023.0f);
		}

		// seek to the first active pixel of the next row
		fseek(fd, knEndOfRowPadding, SEEK_CUR);
	}

	// save the active sensor image
	image.SaveToFile(L"D:/RaspberryPi/Bayer.png", ke16BitsPerChannel);

	fclose(fd);
	return;
}

Additionally, here is a dump of the first 256 bytes of jbeale's "new firmware" image header, with added comments. Values are expressed as decimal, hex, binary and ASCII, as shown by column headers. Variable names from 6by9's very helpful information.

Code: Select all

RAW-GMB-Nov2014.jpg (jbeale)
Off Dec Hx Binary   ASC DataType VariableName   Comment
--- --- -- -------- --- -------- -------------- ---------------------
  0  66 42 01000010 'B'
  1  82 52 01010010 'R'
  2  67 43 01000011 'C'
  3  77 4d 01001101 'M'
  4 110 6e 01101110 'n'
  5   0 00 00000000
  6   0 00 00000000
  7   0 00 00000000
  8 252 fc 11111100
  9 127 7f 01111111
 10   0 00 00000000
 11   0 00 00000000
 12   0 00 00000000
 13   0 00 00000000
 14   0 00 00000000
 15   0 00 00000000
 16 111 6f 01101111 'o'
 17 118 76 01110110 'v'
 18  53 35 00110101 '5'
 19  54 36 00110110 '6'
 20  52 34 00110100 '4'
 21  55 37 00110111 '7'
 22  32 20 00100000 ' '
 23 118 76 01110110 'v'
 24 101 65 01100101 'e'
 25 114 72 01110010 'r'
 26 115 73 01110011 's'
 27 105 69 01101001 'i'
 28 111 6f 01101111 'o'
 29 110 6e 01101110 'n'
 30  32 20 00100000 ' '
 31  48 30 00110000 '0'
 32  46 2e 00101110 '.'
 33  49 31 00110001 '1'
 34   0 00 00000000
 35   0 00 00000000
 36   0 00 00000000
 37   0 00 00000000
 38   0 00 00000000
 39   0 00 00000000
 40   0 00 00000000
 41   0 00 00000000
 42   0 00 00000000
 43   0 00 00000000
 44   0 00 00000000
 45   0 00 00000000
 46   0 00 00000000
 47   0 00 00000000
 48   0 00 00000000
 49   0 00 00000000
 50   0 00 00000000
 51   0 00 00000000
 52   0 00 00000000
 53   0 00 00000000
 54   0 00 00000000
 55   0 00 00000000
 56   0 00 00000000
 57   0 00 00000000
 58   0 00 00000000
 59   0 00 00000000
 60   0 00 00000000
 61   0 00 00000000
 62   0 00 00000000
 63   0 00 00000000
 64   0 00 00000000
 65   0 00 00000000
 66   0 00 00000000
 67   0 00 00000000
 68   0 00 00000000
 69   0 00 00000000
 70   0 00 00000000
 71   0 00 00000000
 72   0 00 00000000
 73   0 00 00000000
 74   0 00 00000000
 75   0 00 00000000
 76   0 00 00000000
 77   0 00 00000000
 78   0 00 00000000
 79   0 00 00000000
 80   0 00 00000000
 81   0 00 00000000
 82   0 00 00000000
 83   0 00 00000000
 84   0 00 00000000
 85   0 00 00000000
 86   0 00 00000000
 87   0 00 00000000
 88   0 00 00000000
 89   0 00 00000000
 90   0 00 00000000
 91   0 00 00000000
 92   0 00 00000000
 93   0 00 00000000
 94   0 00 00000000
 95   0 00 00000000
 96   0 00 00000000
 97   0 00 00000000
 98   0 00 00000000
 99   0 00 00000000
100   0 00 00000000
101   0 00 00000000
102   0 00 00000000
103   0 00 00000000
104   0 00 00000000
105   0 00 00000000
106   0 00 00000000
107   0 00 00000000
108   0 00 00000000
109   0 00 00000000
110   0 00 00000000
111   0 00 00000000
112   0 00 00000000
113   0 00 00000000
114   0 00 00000000
115   0 00 00000000
116   0 00 00000000
117   0 00 00000000
118   0 00 00000000
119   0 00 00000000
120   0 00 00000000
121   0 00 00000000
122   0 00 00000000
123   0 00 00000000
124   0 00 00000000
125   0 00 00000000
126   0 00 00000000
127   0 00 00000000
128   0 00 00000000
129   0 00 00000000
130   0 00 00000000
131   0 00 00000000
132   0 00 00000000
133   0 00 00000000
134   0 00 00000000
135   0 00 00000000
136   0 00 00000000
137   0 00 00000000
138   0 00 00000000
139   0 00 00000000
140   0 00 00000000
141   0 00 00000000
142   0 00 00000000
143   0 00 00000000
144   3 03 00000011
145 246 f6 11110110
146   0 00 00000000
147   0 00 00000000
148 224 e0 11100000
149   1 01 00000001
150   0 00 00000000
151   1 01 00000001
152 255 ff 11111111
153   0 00 00000000
154   0 00 00000000
155   0 00 00000000
156   0 00 00000000
157   0 00 00000000
158   0 00 00000000
159   0 00 00000000
160 192 c0 11000000
161  12 0c 00001100
162   0 00 00000000
163   0 00 00000000
164   0 00 00000000
165   0 00 00000000
166   0 00 00000000
167   0 00 00000000
168   0 00 00000000
169   0 00 00000000
170   0 00 00000000
171   0 00 00000000
172   0 00 00000000
173   0 00 00000000
174   0 00 00000000
175   0 00 00000000
176  50 32 00110010 '2'	uint8_t  name[32]       short text description of the mode
177  53 35 00110101 '5'
178  57 39 00111001 '9'
179  50 32 00110010 '2'
180 120 78 01111000 'x'
181  49 31 00110001 '1'
182  57 39 00111001 '9'
183  52 34 00110100 '4'
184  52 34 00110100 '4'
185   0 00 00000000
186   0 00 00000000
187   0 00 00000000
188   0 00 00000000
189   0 00 00000000
190   0 00 00000000
191   0 00 00000000
192   0 00 00000000
193   0 00 00000000
194   0 00 00000000
195   0 00 00000000
196   0 00 00000000
197   0 00 00000000
198   0 00 00000000
199   0 00 00000000
200   0 00 00000000
201   0 00 00000000
202   0 00 00000000
203   0 00 00000000
204   0 00 00000000
205   0 00 00000000
206   0 00 00000000
207   0 00 00000000
208  32 20 00100000 ' '	uint16_t width          width of frame in pixels (low byte, high byte)
209  10 0a 00001010
210 152 98 10011000     uint16_t height         height of frame in pixels (low byte, high byte)
211   7 07 00000111
212   0 00 00000000     uint16_t padding_right  additional non-data pixels to the right of the frame
213   0 00 00000000
214   0 00 00000000     uint16_t padding_down   additional non-data pixels below the frame
215   0 00 00000000
216  27 1b 00011011     uint32_t
217 127 7f 01111111
218   0 00 00000000
219   0 00 00000000
220 176 b0 10110000     uint32_t
221   7 07 00000111
222   0 00 00000000
223   0 00 00000000
224 128 80 10000000     uint32_t
225  76 4c 01001100 'L'
226 206 ce 11001110
227   0 00 00000000
228   0 00 00000000     uint32_t
229   1 01 00000001
230   0 00 00000000
231   0 00 00000000
232   0 00 00000000     uint32_t
233  15 0f 00001111
234   0 00 00000000
235   0 00 00000000
236   1 01 00000001     uint16_t
237   0 00 00000000
238   1 01 00000001     uint16_t
239   0 00 00000000
240   0 00 00000000     uint16_t transform      possible transformations in this mode / user requested transformations
241   0 00 00000000
242  33 21 00100001 '!' uint16_t format         image format of the frame
243   0 00 00000000
244   1 01 00000001     uint8_t  bayer_order    if a Bayer format frame, the Bayer order in the frame
245   3 03 00000011     uint8_t  bayer_format   if a Bayer format frame, the Bayer format in the frame
246  10 0a 00001010
247   0 00 00000000
248   0 00 00000000
249   0 00 00000000
250   0 00 00000000
251   0 00 00000000
252   2 02 00000010
253   0 00 00000000
254   0 00 00000000
255   0 00 00000000

[6by9]

Patches to fix the raw header sent to Dom for release. I guess it'll just need to be tested to see if the old vs new firmware will do the right thing with this bug.

edit Patches released. sudo rpi-update to get them.

[Schwabenchris]

Hey Folks,

I was taking dark images to measure dark noise of the sensor. The histogram of the images isn't really Poisson alike, what I would expect.

I'm importing them as dng with Matlab, they are scaled up to fit in an 16bit unsigned integer array.
What you see is the histogram of and dark shot, cut from 0:2^16 to 0:4000, cause the rest ist almost 0.

It has a steps, can't explain them with circuit design or physical effects yet. Any chance this is a software error?

[blindbloom]

Thanks for sharing this interesting histogram.

The RPi camera captures 10-bits per sensor element. My interpretation of the stepped appearance (each step is four lines wide in the histogram) is that the two LSBs of each sample is uniform noise. Only the eight MSBs contain valid image data. For example, the 10-bit values {0, 1, 2, 3} are equally likely to appear, as are {4, 5, 6, 7} and so on.

Proving the cause might be tricky, but if you could provide us with an original raw capture file from the RPi, we could determine whether the problem is already there rather than being introduced by a downstream process (e.g. DNG conversion, Matlab, etc.).

[Schwabenchris]

Hey,

Thanks for your fast reply.

Jpeg+Raw:
http://s000.tinyupload.com/index.php?fi ... 9363156724

DNG-file using raspiraw from https://github.com/illes/raspiraw:
http://s000.tinyupload.com/index.php?fi ... 2141524478

DNG-file using rpi2dng from https://github.com/bablokb/raspiraw:
http://s000.tinyupload.com/index.php?fi ... 7905265107


Matlab code to import the dng-file:

Code: Select all

t = Tiff('filename','r');
offsets = getTag(t,'SubIFD');
setSubDirectory(t,offsets(1));
cfa = read(t);
close(t);
figure(1);
histogram(cfa(:),0:4000);

[sharix]

I get the same histogram in Mathematica (rpi2dng raw file):

[blindbloom]

I took a close look at the Jpeg+Raw file and plotted a histogram of the lowest 32 codes (out of 1024) for each of the Bayer filters (G1, B, R, G2). From these histograms, I conclude that over this range of codes:

• Bits 0 and 1 for red and blue are approximately uniformly distributed random numbers (white noise).

• Bit 0 for G1 and G2 is an approximately uniformly distributed random number (white noise).

• Bit 1 of G1 and G2 is more likely to be 0 than 1.

I find this both surprising and unsettling. Considering that these codes represent scene-referred values (linear), and that they are at the dark end of the scale, shadow detail will suffer.

Can anyone explain this? I wonder if this characteristic is common to all RPi cameras or if there is something wrong with this particular one.

Corrected G1 and R graphs.

darkshot+raw.png (11.28 KiB) Viewed 5105 times

[jbeale]

Nice work analyzing this. Without more analysis my immediate guess is that the OV5647 camera module hardware simply does not deliver true 10-bit ADC performance, despite the nominal 10-bit data format. Looking at raw images they have always seemed very noisy to me, it is only after the heavy processing in the RPi image chain that they look good.

[blindbloom]

I agree that noise is an issue and a problem. But Schwabenchris was concerned about the non-Poisson shape. There is something else going on here besides noise that is causing the "stepped" look. So I was wondering if this is a one-off, or a characteristic of all RPi cameras. If anyone can repeat the "darkshot+raw" test on a different camera, I will do a similar analysis.

[jbeale]

I have taken a dark frame with "raspistill -ISO 100 -r -o raw1.jpg" and it is about 6.5 MB. Let me know how to get it to you (eg. send email in PM?)
This was not actually the official RPi camera, rather the Adafruit "spy cam" on a small flex cable but it is the same Omnivision sensor.

[blindbloom]

Here is the same plot of jbeale's image. Very different from the other. The stepped look is not present. Since this is a different camera manufacturer (but using the same sensor), I guess we still cannot be sure what the cause of the stepping is.

Corrected G1 and R graphs.

RPI-raw1.png (11.94 KiB) Viewed 5105 times

[Schwabenchris]

I was using ISO 800 and a shutter speed of 5s.

Just got the info, my sensor was previously used to measure it's noise behavior over a temperature range from 283,15 K to 343,15 K. But that's still inside the specifications of the OV5647

[jbeale]

Sounds like "ISO 800" may be implemented as a digital gain, rather than a pre-ADC analog gain? But blindbloom's plots from my dark frame still look strange to me- I would expect something like a gaussian shape, but from the histogram, it's almost as if the two LSBs have switched places. Just for fun... if you swap the order of b0 and b1 for each channel does it look any better?

[blindbloom]

I don't think that will help. If you think about the effect of swapping b0 and b1 of each pixel, you get this mapping:

00 -> 00 (no change)
01 -> 10
10 -> 01
11 -> 11 (no change)

If you look at any of the graphs (jbeale's) there are eight columns that contain anything significant. Let's number them 0 through 7. Swapping b0 and b1 will swap columns (1 & 2) and columns (5 & 6). The other columns will stay the same. This will not help make the histograms look any better. Or am I misunderstanding you?

[6by9]

It's a sensor designed for mobile phones and costing a couple of dollars! How much can you really complain?!
I very much doubt there is anything too obscure really going on - why would OVT do bother? There's nothing in it for them.

Analogue gain (as set via ISO) should be just that, not digital gain post ADC. Why would they do that on only the green channel anyway?

Has anyone checked the processing path once it is on the Pi in detail? PEBKAC is very common answer. I'd suggest taking the rawcam stuff to grab a direct raw. If in doubt, get the peripheral to unpack from raw10 to raw16 so you don't have to prat around with getting it wrong in your software.
I'd also say avoid the really long exposure time mode - OV did acknowledge that it was pushing the boundaries a little, although should work without issue.

[blindbloom]

Ok. I figured this all out. The (mostly undocumented) raw data is appended to the end of the JPEG file, as described elsewhere. The pixels are 10-bits each and four sequential pixels are encoded in five sequential bytes. This is the correct format:

Decoding.png (16.32 KiB) Viewed 5097 times

When decoded this proper way, the "dark shot" images we've been working with make much nicer histograms like this:

EncodingFix.png (7.5 KiB) Viewed 5097 times

The Mathematica and Matlab decoders should be changed accordingly. Maybe Schwabenchris can contact them with this info?