falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Long exposure with raspiraw

Thu Jun 14, 2018 7:45 pm

Hello all,

I've been trying to take long exposure (>1s) captures with the raspiraw program by @6by9, but I don't really know in which mode should I put the camera. Ideally, I'd like to capture full resolution (2592x1944) raw images (without Bayer interpolation) with the sensor's default color depth (which is 10bit for OV5647).

I'm setting the "exposure (-e)" parameter to its maximum value (2^20-1), but this doesn't seem to work and it has different behaviours depending on the "mode" parameter.

I've tried the following commands:

- ./raspiraw -md 0 -g 1 -o /dev/shm/out%05d.raw -t 20000 -sr 1 -e 1048575 -ts timestamps.txt
This seems to capture one image every 64ms. This is based on the 'delay' column of the timestamps.txt file. (also, I'm not sure if this is the time between images or the real exposure time).

- ./raspiraw -md 3 -g 1 -o /dev/shm/out%05d.raw -t 20000 -sr 1 -e 1048575 -ts timestamps.txt
In this case the 'delay' column oscillates between 1s, 2s, 3s... sometimes even more..

According to the table of modes in http://picamera.readthedocs.io/en/lates ... nsor-modes, I need to put the sensor in mode 3, where the lower frame rate is 1/6s, which is equivalent to 6s exposure time, so I was expecting to get that exposure time when setting mode 3 and the maximum "-e" parameter... but doesn't seem to be the case...

Any advise on how to get maximum exposure time with raspiraw?

BTW, I'm using a Raspberry Pi 3 with the camera module V1 (OV5647).

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Mon Jun 18, 2018 1:59 pm

Hello,

I've been playing around a little bit more and I think mode 3 is not working properly. I've taken a couple of images with the following parameters and most of them look like if they were out of sync.

Code: Select all

./raspiraw -md 3 -g 1 -o /dev/shm/test%05d.raw -t 60000 -sr 1 -ts timestamps.txt -eus 5999884 -f 0.166666667
And these are the images (they were rescaled from 2592x1944 to 1024x768):
test11.jpg
test11.jpg (103.15 KiB) Viewed 421 times
The attachment test11.jpg is no longer available
The exposure parameter (5999884us) is the same that appears in the mmal log when I take picture with raspistill and a shutter speed of 6s.

I wasn't able to find a similar problem by reading the forum since most of the time people want to take fast captures with very high fps.
So, do you have any idea of what could be wrong?

Thanks in advance!!
Attachments
test22.jpg
test22.jpg (108.95 KiB) Viewed 421 times

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Tue Jun 19, 2018 3:02 pm

Hello again!

So far I was able to take snapshots with an integration time of almost 4s (based on what it is said on timestamps file and on the brightness of the images). I used the following command line:

Code: Select all

./raspiraw -md 3 -g 1 -o /dev/shm/out%05d.raw -t 120000 -sr 1 -ts timestamps.txt -r "380E,7F;380F,FF;3500,0F;3501,FF;3502,FF;3036,50"
I'm setting the registers TIMING_VTS to 0x7FFF, EXPOSURE to 0x0FFFFF and SC_CMMN_PLL_MULTIPLIER to 0x50. I ended up setting these registers by hand because I think the -e and -f parameters are not handle right by raspiraw for long exposure times. (If I set -e to its maximum then the VTS is not set.. or If I set -f to small values then the VTS register is set to weird values.)

Looking at the differences between mode 2 and mode 3 I've found that the register 0x3036 (SC_CMMN_PLL_MULTIPLIER) was being written to 0x34 while on mode 3, but it had the default value (0x69) on mode 2. So I started to play with this register and the best I could get was 4s of Tint with 0x50. Lower values on this PLL register give longer integration times, but then some images are out of sync, so I think I should modify something about the timings in the VCOS/mmal/something... I really don't have any idea of what I am doing with this register, so any advice will be appreciated.

Another this that is weird is that if I set the register TIMING_VTS to 0x8FFF I get black images only. But if I set it to 0xFFFF it works again with a bit longer integration time. So, it seems to be a bit there that does something else... a debug bit?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5216
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Long exposure with raspiraw

Tue Jun 19, 2018 4:43 pm

falcald wrote:
Tue Jun 19, 2018 3:02 pm
Looking at the differences between mode 2 and mode 3 I've found that the register 0x3036 (SC_CMMN_PLL_MULTIPLIER) was being written to 0x34 while on mode 3, but it had the default value (0x69) on mode 2. So I started to play with this register and the best I could get was 4s of Tint with 0x50. Lower values on this PLL register give longer integration times, but then some images are out of sync, so I think I should modify something about the timings in the VCOS/mmal/something... I really don't have any idea of what I am doing with this register, so any advice will be appreciated.
If you have a look at how a phase locked loop is implemented then it may make more sense - https://en.wikipedia.org/wiki/Phase-loc ... ck_diagram.
The external oscillator on the camera board is 25MHz, so the PLL is scaling that up to produce the pixel clock that drives almost the whole sensor.
Reduce the multiplication factor and the pixel clock goes slower, hence how we managed to get it up to 6secs. I hit the same issues that trying to take it slower just started failing - I don't think I've ever seen a spec for what is expected to work or not.
falcald wrote:Another this that is weird is that if I set the register TIMING_VTS to 0x8FFF I get black images only. But if I set it to 0xFFFF it works again with a bit longer integration time. So, it seems to be a bit there that does something else... a debug bit?
Read the data sheet and VTS is reported as being 10 bits of resolution, so anything above 0x3FF is toggling bits that are denoted as "debug mode". All bets are off if you toggle them.
raspiraw should be removing any bits above bit 9, but that code needs some careful examination as it may well be flawed (we found some odd things going on, but haven't had a chance to fully debug).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Tue Jun 19, 2018 8:16 pm

Hi 6by9!
If you have a look at how a phase locked loop is implemented then it may make more sense - https://en.wikipedia.org/wiki/Phase-loc ... ck_diagram.
The external oscillator on the camera board is 25MHz, so the PLL is scaling that up to produce the pixel clock that drives almost the whole sensor.
Reduce the multiplication factor and the pixel clock goes slower, hence how we managed to get it up to 6secs. I hit the same issues that trying to take it slower just started failing - I don't think I've ever seen a spec for what is expected to work or not.
When I said that I didn't know what I was doing I was referring to which clock I was scaling or what was the effect in the chip (I know what a PLL is, but thanks anyway :-) ). Now I know that this is the pixel clock scaling, thanks! So, if I change this scaling factor I don't have to change any other timing registers on the chip, or should I?
Read the data sheet and VTS is reported as being 10 bits of resolution, so anything above 0x3FF is toggling bits that are denoted as "debug mode". All bets are off if you toggle them.
You are right.. but I get only 244ms of exposure if I use 0x03FF (In fact 0x07FF, because it doesn't work with 0x03FF.. I get black images), whereas when I set it to 0x7FFF the exposure time is almost 4s and, I don't know why, it works... Maybe with a better datasheet I would be able to solve this mystery...
raspiraw should be removing any bits above bit 9, but that code needs some careful examination as it may well be flawed (we found some odd things going on, but haven't had a chance to fully debug).
I'm using the -r parameter to write the VTS register so I think this doesn't apply here, or does it?

Anyway, I've found this fork of raspiraw https://github.com/tmorley2000/raspiraw ... 712a8423c3 which in that commit states that it enables the possibility of long exposure captures.. I've tried it and it didn't work on mode 3 (but I have to test it touching the PLL register yet. I'm going to do that as soon as I finish writing). What I found interesting is that he changes the HTS and VTS registers depending on the exposure and fps parameters. So, what exactly are these registers? Could you give me a hint? (The datasheet that I have doesn't say anything).

Thanks again for your help!

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Wed Jun 20, 2018 10:09 pm

falcald wrote:
Tue Jun 19, 2018 8:16 pm
Anyway, I've found this fork of raspiraw https://github.com/tmorley2000/raspiraw ... 712a8423c3 which in that commit states that it enables the possibility of long exposure captures.. I've tried it and it didn't work on mode 3 (but I have to test it touching the PLL register yet. I'm going to do that as soon as I finish writing).
It works in mode 3 with the register 0x3036 (SC_CMMN_PLL_MULTIPLIER) as low as 0x50.. with lower values I started to get out of sync images.

Anyway, I couldn't get exposure times in the order of seconds. So far the best was my command line with magic parameters.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5216
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Long exposure with raspiraw

Thu Jun 21, 2018 9:32 am

Having a quick look at our driver, it allows a max integration time of 32767 (0x7FFF), so 10 bits looks like a datasheet inaccuracy.
Your 0x8FFF is therefore actually 0x0FFF, whereas 0xFFFF is 0x7FFF.

Please note that VTS (Vertical Total Size, aka FRM_LENGTH on imx219) will only alter the framerate, not the exposure time. Frame rate is controlled by adding extra blanking lines to the frame, and that is what VTS is setting. Obviously this can not go below the height of the frame.
Exposure time is then set as described in https://picamera.readthedocs.io/en/late ... -operation as the number of lines between resetting a row and reading it out. That has to be less than VTS.
Note too that there is an HTS register that allows padding to be added horizontally too.

I can't remember the full detail of what I was changing PLLs to get up to 6s exposure (it was back in July 2014), but I seem to recall I was trying to stick with simple multiply/divide by 2 values. Then again perhaps not as the time per line value is 183789ns vs 32503ns normally, which is x5.65. (183789*32767 = 6.022s, 32503*32767 = 1.065s, so those match up with the achieved min frame rates for modes 3 & 2 respectively)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Thu Jun 21, 2018 3:32 pm

6by9 wrote:
Thu Jun 21, 2018 9:32 am
Having a quick look at our driver, it allows a max integration time of 32767 (0x7FFF), so 10 bits looks like a datasheet inaccuracy.
Your 0x8FFF is therefore actually 0x0FFF, whereas 0xFFFF is 0x7FFF.
"This datasheet is wrong".. that was the first thing I thought. Thanks for the confirmation. It would be great if you can change it in the ov5647_modes.h file of raspiraw:

Code: Select all

   .vts_reg_num_bits =     15,      // total vertical size [14:8] and [7:0] (ov5647 datasheet is wrong)
6by9 wrote:
Thu Jun 21, 2018 9:32 am
Please note that VTS (Vertical Total Size, aka FRM_LENGTH on imx219) will only alter the framerate, not the exposure time. Frame rate is controlled by adding extra blanking lines to the frame, and that is what VTS is setting. Obviously this can not go below the height of the frame.
Great! I think that I understand better these HTS and VTS registers now.
6by9 wrote:
Thu Jun 21, 2018 9:32 am
Exposure time is then set as described in https://picamera.readthedocs.io/en/late ... -operation as the number of lines between resetting a row and reading it out. That has to be less than VTS.
Are you sure it has to be less than the VTS? Why is the EXPOSURE register (0x3500-0x3502) 20 bits long then? (Another mistake in the datasheet?)
What is its unit? "line_time" or "pixel_time"? (for example, if I set EXPOSURE to 10, is it 10*line_time or 10*pixel_time?).

For what I've seen, setting the VTS and HTS to their maximum values and increasing the value of the EXPOSURE beyond 0x007FFF (maximum VTS) and up to 0x0FFFFF increases the image brightness, so I infer 0x0FFFFF is the maximum integration time, i.e. the maximum time between reset and read.
6by9 wrote:
Thu Jun 21, 2018 9:32 am
I can't remember the full detail of what I was changing PLLs to get up to 6s exposure (it was back in July 2014), but I seem to recall I was trying to stick with simple multiply/divide by 2 values. Then again perhaps not as the time per line value is 183789ns vs 32503ns normally, which is x5.65. (183789*32767 = 6.022s, 32503*32767 = 1.065s, so those match up with the achieved min frame rates for modes 3 & 2 respectively)
May I ask you how did you get those times per lines? I imagine it depends on the pixel clock, the PLL setting (which doesn't work below 0x50 in my setup) and the HTS.

Thanks again for your help. I think I'm almost done with this!

HermannSW
Posts: 699
Joined: Fri Jul 22, 2016 9:09 pm

Re: Long exposure with raspiraw

Thu Jun 21, 2018 7:30 pm

falcald wrote:
Thu Jun 21, 2018 3:32 pm
May I ask you how did you get those times per lines? I imagine it depends on the pixel clock, the PLL setting (which doesn't work below 0x50 in my setup) and the HTS.
6by9 is bound by NDAs, so he cannot tell you anything that is not publically available somewhere. You get those line time by formula "maximal framerate = 1,000,000,000 / (vts * line_time_ns)". You can see here for modes 7, 6, 5 and 4 of ov5647_modes.h of raspiraw:

Code: Select all

$ bc -ql
1000000000/(484*21665)
95.36652215459676173437
1000000000/(484*31749)
65.07655996974200267646
1000000000/(754*23216)
57.12697910706418513162
1000000000/(996*23216)
43.24672916337991525024
Btw, you can lookup all details in several hundreds of pages v1 and v2 camera specs youself:
v1: https://cdn.sparkfun.com/datasheets/Dev ... 7_full.pdf
v2: viewtopic.php?t=177308
--> Raspberry camera / gstreamer / raspivid / raspiraw (bookmark list):
https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/fork-raspiraw      https://github.com/Hermann-SW/userland      https://twitter.com/HermannSW

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Thu Jun 21, 2018 9:18 pm

HermannSW wrote:
Thu Jun 21, 2018 7:30 pm
falcald wrote:
Thu Jun 21, 2018 3:32 pm
May I ask you how did you get those times per lines? I imagine it depends on the pixel clock, the PLL setting (which doesn't work below 0x50 in my setup) and the HTS.
6by9 is bound by NDAs, so he cannot tell you anything that is not publically available somewhere. You get those line time by formula "maximal framerate = 1,000,000,000 / (vts * line_time_ns)". You can see here for modes 7, 6, 5 and 4 of ov5647_modes.h of raspiraw:

Code: Select all

$ bc -ql
1000000000/(484*21665)
95.36652215459676173437
1000000000/(484*31749)
65.07655996974200267646
1000000000/(754*23216)
57.12697910706418513162
1000000000/(996*23216)
43.24672916337991525024
Btw, you can lookup all details in several hundreds of pages v1 and v2 camera specs youself:
v1: https://cdn.sparkfun.com/datasheets/Dev ... 7_full.pdf
v2: viewtopic.php?t=177308
Hi HermannSW! Thanks, I have that datasheet, but it has some (important) inaccuracies and missing information.

Regarding the line time. Ok, that's how to get it in an "experimental" way (i.e. I measure the fps and knowing the vts I can get line_time_ns). There should be a "theoretical" way involving pixel clk, PLL, etc... anyway I can live without knowing it.

Thanks again for your help!

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5216
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Long exposure with raspiraw

Thu Jun 21, 2018 9:38 pm

falcald wrote:
Thu Jun 21, 2018 9:18 pm
Hi HermannSW! Thanks, I have that datasheet, but it has some (important) inaccuracies and missing information.

Regarding the line time. Ok, that's how to get it in an "experimental" way (i.e. I measure the fps and knowing the vts I can get line_time_ns). There should be a "theoretical" way involving pixel clk, PLL, etc... anyway I can live without knowing it.

Thanks again for your help!
When under NDA with Omnivision you get a nice spreadsheet which you bung all the numbers in and the line timings fall out the other end :D
If you looked through the datasheet for multipliers and dividers then you can probably work it out for yourself based on the numbers provided.

Units for VTS and exposure are always in lines. I was pretty sure that exposure had to be less than VTS, but you do appear to be correct that exposure is 20 bits vs VTS at 15 bits, so what use are those top 5 bits? I can't say without delving back into code, and probably shouldn't anyway due to NDAs.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Fri Jun 22, 2018 1:35 pm

6by9 wrote:
Thu Jun 21, 2018 9:38 pm
When under NDA with Omnivision you get a nice spreadsheet which you bung all the numbers in and the line timings fall out the other end :D
If you looked through the datasheet for multipliers and dividers then you can probably work it out for yourself based on the numbers provided.

Units for VTS and exposure are always in lines. I was pretty sure that exposure had to be less than VTS, but you do appear to be correct that exposure is 20 bits vs VTS at 15 bits, so what use are those top 5 bits? I can't say without delving back into code, and probably shouldn't anyway due to NDAs.
I feel like talking to a monk very far away in the mountains :lol:
It seems that I'll need to do a bit of research and a couple of experiments more with some try&error.

Again thanks for your help!

falcald
Posts: 9
Joined: Thu Jun 14, 2018 7:22 pm

Re: Long exposure with raspiraw

Mon Jun 25, 2018 5:30 pm

Hi! Just for the record:

The datasheet of another sensor of the family (https://cdn.sparkfun.com/datasheets/Sen ... asheet.pdf) says:
The exposure value in registers 0x3500~0x3502 is in units of line*16 - the low 4 bits (0x3502[3:0]) is the fraction of line, the maximum value in {0x380E + 0x380F} + {0x350C, 0x350D} is in unit of line
Which I understand means that the EXPOSURE register is in "1/16 of line" units, and that the 4 LSB bits are fractions of line. (http://forums.xilinx.com/xlnx/attachmen ... _V1.11.pdf says something similar).

In those datasheets the "manual exposure control" section wasn't removed like in the OV5647 preliminary datasheet.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 10 guests