narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

running raspiraw with v2.1 camera

Thu Aug 16, 2018 9:09 pm

Raspiraw is outputting data, but the data is sparse. For example here is a hexdump excerpt:

00c32d0 1010 1110 102e 1010 1e11 1010 1110 102e
00c32e0 1010 2e11 1110 1110 1022 1010 2e11 1010
00c32f0 1110 101d 1011 e210 1010 1010 10ee 1011
00c3300 2211 1110 1110 1023 1011 e210 1110 1110
00c3310 1023 1010 ea10 1110 1010 10f3 1011 e610
00c3320 1110 1010 10d2 1011 e310 1110 1010 10e6
00c3330 1010 ee10 1110 1010 10f3 1011 5210 1010
00c3340 1010 1040 1010 5410 1010 1010 1015 1010
00c3350 5510 1010 1010 1005 1010 0110 1010 1010
00c3360 1004 1010 0010 1010 1010 1054 1010 6410
00c3370 1010 1010 1014 1010 0010 1010 1010 1055
00c3380 1010 5510 1010 1010 1085 1010 5410 1010


This data was grabbed through tools/1280x720 from the https://github.com/6by9/raspiraw repository and a well-lit scene. The scripts run fine without any dropped frames. Other scripts in the directory produce similar results (e.g. 640x480).

When I try to demosaic the raw frames I get black bars in the image -- that is, you can see a periodic(ish) 0x10 in the raw data, so I don't think it's an issue of slicing the data incorrectly. (I've tried the code https://github.com/6by9/dcraw as well as my own code as a sanity check.) Other than the black bars, you can make out the scene, so that's good.

I thought it might be 10-bit data, but it doesn't quite make sense. The file sizes imply 8-bit data (raw file sizes are width*height).

I'm using a v2.1 camera board revision a020d3.

I'm just looking for some hints (please!)

thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: running raspiraw with v2.1 camera

Fri Aug 17, 2018 7:09 am

A bug had crept in to the firmware which resulted in the buffer sizes being incorrectly computed. First affected release was 30th July, fixed in the 8th Aug release. That probably explains the incorrectly sized buffers.

The data certainly appears to be raw 10 where you get the 8 most significant bits of 4 pixels, and then the remaining least significant bits for those pixels packed in to one byte. It generally appears as a column of noise every 5th pixel if you view as 8bpp mono.
dcraw should have been able to deal with the images provided it had the header at the start - I believe HermanSW's scripts save that to a separate file.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Fri Aug 17, 2018 7:34 pm

Nailed it --- updating the firmware fixed it. Nice!

I've been having fun playing with this -- the framerates are impressive and the image quality is impressive as well. :)

Is it normal for raspiraw to hang on occasion? In general, the higher the fps, the more likely it will hang. When raspiraw does hang, it puts the camera into a defunct state (raspiraw, raspistill, raspivid will all hang) until I reboot. The frames look good for many of the higher framerates though.

For example if I run this:
raspiraw -md 7 -t 1000 -ts tstamps.csv -hd0 hd0.32k --height 480 --top 0 --vinc 17 --fps 200 -sr 1 -g 240 -e 200 -o /dev/shm/out.%04d.raw 2>/dev/null >/dev/null
The frames are good quality, no artifacts, but it hangs every 20 or so invocations. Is this a matter of keeping things within bounds?

thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: running raspiraw with v2.1 camera

Fri Aug 17, 2018 8:20 pm

There is a known issue with rawcam that under some conditions it ends up trying to return a buffer twice which results in the firmware deliberately aborting.
I'm looking at it at the moment in the office - it's almost certainly a race condition hiding in there, but it's not obvious where at the moment.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Tue Aug 21, 2018 12:36 pm

Hi 6x9, many thanks for looking into this. I'm excited to use this for our application. Do you have a ETA?

thanks!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5805
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: running raspiraw with v2.1 camera

Tue Aug 21, 2018 4:30 pm

narragansett wrote:
Tue Aug 21, 2018 12:36 pm
Hi 6x9, many thanks for looking into this. I'm excited to use this for our application. Do you have a ETA?
My recommendation would be to use the V4L2 driver rather than rawcam, but doing so requires a bit more know-how to set up the sensor driver. The V4L2 driver is the one that will get more support attention than rawcam.

As it happens I think I've found the bug in rawcam. I'm putting together the required patches for review, so should hit an rpi-update soon (normal warnings apply about stability and bugs).

If it is of interest, I pushed a basic AWB algorithm implementation to https://github.com/6by9/raspiraw/tree/temp_awb yesterday.
I might see if I can do a basic AE/AGC loop too, but that gets more involved due to the latency in the sensor for adopting new exposure and gain settings, and having to clip to the requested frame rate, etc.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Tue Aug 21, 2018 5:02 pm

I'm interested in the fast framerates that raspiraw provides. I hope to get 400-500 fps, reduced resolution with enough lighting and v2 camera. I can't see raw formats supported with the current v4l2 driver (v4l2-ctl -d /dev/video0 --list-formats)

Please ping back when your patch is ready :)

thanks!

HermannSW
Posts: 787
Joined: Fri Jul 22, 2016 9:09 pm

Re: running raspiraw with v2.1 camera

Wed Aug 22, 2018 12:54 pm

> I hope to get 400-500 fps
>
You definitely need raspiraw because anything that runs through Raspberry GPU is limited to <202fps due to GPU not being capable of processing higher than that -- 6by9 stated that GPU bug will not be fixed.

You can get [email protected] or [email protected] with v2 camera: You can get same framerates with 640x352 or 640x448 FOV by using the corresponding "..._s" mode that scans only every other line (and duplicates lines in post processing if needed). High framerates mean low shutter times, so you will need bright light:
https://www.raspberrypi.org/forums/view ... 8#p1320034
Image


P.S:
Outside of your requirements you can get 1007fps(!) for 640x75 with v2 camera as well (played at 1fps):
https://www.raspberrypi.org/forums/view ... 5#p1310445
Image
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/fork-raspiraw      https://github.com/Hermann-SW/userland
https://github.com/Hermann-SW/wireless-control-Eachine-E52-drone      https://twitter.com/HermannSW

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Wed Aug 22, 2018 5:09 pm

Hi Hermannsw,
Thank you for the information :) I was inspired by your forum messages. 1000 fps is an incredible result!

I noticed that raspiraw supports registers for the omnivision camera , but doesn't appear to support the sony sensor, in which case the -vinc and -hinc will only work with the v1 camera.

Have you been able to get the sony sensor to skip lines so that the higher framerates can cover more of the image?

thanks!

HermannSW
Posts: 787
Joined: Fri Jul 22, 2016 9:09 pm

Re: running raspiraw with v2.1 camera

Thu Aug 23, 2018 10:46 am

narragansett wrote:
Wed Aug 22, 2018 5:09 pm
I noticed that raspiraw supports registers for the omnivision camera , but doesn't appear to support the sony sensor, in which case the -vinc and -hinc will only work with the v1 camera.
Please see my "fork-raspiraw" link in signature, that contains most recent features.
You can find the v2 tools not integrated yet into that fork in v2 directory of code drop here:
https://www.raspberrypi.org/forums/view ... 4#p1320044
Have you been able to get the sony sensor to skip lines so that the higher framerates can cover more of the image?
YES!

From this posting where I did measure the propeller speed of RC airplane as 20140rpm with v2 camera 1007fps video:
https://www.raspberrypi.org/forums/view ... 7#p1319617


This was video obtained with "raspiraw.drop.5_26_18/v2/640x75" tool at 1007fps:
Image


And this with "raspiraw.drop.5_26_18/v2/640x150_s" tool at 1007fps as well, FOV doubled:
Image


P.S:
I will continue to work on integration of v2 and v1 tools from that code drop into fork-raspiraw, but I am having too many projects right now. Still trying to find RC airplane lost 20m high in group of trees, search&rescue small camera drone lost while searching for airplane in 5-10m high trees, did reverse engineer Eachine E52 FPV camera wireless protocol and was able stream E52 camera video to Raspberry Pi -- but now I got 2nd E52 drone and that has different protocol (video is transferred via UDP and not via TCP as for 1st drone, seems to be a newer firmware, protocol is understood, and I learnt how to send single SYN against port that responds [RST, ACK]), ...
bookmark list: https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/fork-raspiraw      https://github.com/Hermann-SW/userland
https://github.com/Hermann-SW/wireless-control-Eachine-E52-drone      https://twitter.com/HermannSW

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Thu Aug 23, 2018 2:22 pm

Very nice -- thank you for the links Hermannsw! :)

I'll enjoy playing with this and seeing how the gain and exposure affect things.

thanks!

narragansett
Posts: 19
Joined: Tue Aug 07, 2018 12:34 am

Re: running raspiraw with v2.1 camera

Thu Aug 23, 2018 11:02 pm

I'm not sure if this is the best place to post, but I gave the new firmware a whirl (https://drive.google.com/file/d/1pfj5OY ... czj9r/view) and it works great! Muchas gracias 6x9!

Return to “Camera board”