Frame synchronisation

[anykey]

Well there are a lot more people wanting sync control, but...

guys stop wining... instead of complaining, make the best use of what is available today. If you want all your features go buy a ready many unit for that (~$1000/unit).

Buy you can also already use the raspberry pi with camera today and have very accurate sync for photography! I have build multiple 3D scanner setups with 100+ raspberry units per scanner and have millisecond accurate sync!

Sure, I can not use flash, but using permanent lighting, but get great results.

check it out: www.pi3dscan.com

[xpeace]

Could you give us some insight on the ms accurate capture process used, instead of making plain advertisement (for I have to admit a cool product). Btw wouldn't the cheaper A model be sufficient for your capture task ?

(Set to 128/128) I can run a FullHD GTK3 Gui as well as a opencv3 frame capture using the raspicam library on the A model. Btw what do you use as power unit for 100 raspberries ?

[6by9]

Seeing as he is using the LAN to trigger all the captures and handle the image storage, I'd say a model B(+) with LAN built in makes a more sensible choice over a model A without.

I have no involvement in that project, but if you have a fixed exposure time (or frame rate, though raspistill doesn't expose that by default), then the startup time for each instance of raspistill will be nigh on identical. Trigger that via multicast over the LAN and I can see the image synchronisation being within a couple of ms.

[n8ers]

Thanks for looking into this 6by9, it seemed too good to be true that the OV5647 would have an Electronic Global Shutter and that they wouldn't advertise and exploit that heavily. IMO, they should not advertise a "global shutter", they should advertise "global reset", as they have distinct meanings.

EGS sensors are becoming cheap now though; for example, the Aptina MT9V034 (sensor alone) is about 15USD here. I'd love to see someone build a Raspberry Pi interface for it, given the machine vision needs of the community.

Anyway, thanks again.

[Borisnov]

Might the Raspberry Pi Camera Rev.C from Arducam offer a spark of hope?
http://www.arducam.com/raspberry-pi-cam ... rformance/

it provides the FREX and STROBE signals which can be used for multi-camera synchronize capture with proper camera driver firmware.

[6by9]

Might the Raspberry Pi Camera Rev.C from Arducam offer a spark of hope?
http://www.arducam.com/raspberry-pi-cam ... rformance/

it provides the FREX and STROBE signals which can be used for multi-camera synchronize capture with proper camera driver firmware.

No, as that is not the official Raspberry Pi camera board.
Plus, as I have already documented, the FREX control mode requires an external shutter to do something useful, so you still don't have the necessary hardware.
The strobe line has some potential uses for signalling exposure timings, but there are too many variations of when it is triggered to make a generic solution.

Of course feel free to pick up on viewtopic.php?f=43&t=109137 and write your own driver with the datasheet (that is only available under NDA).

[Borisnov]

That is stretching my abilities a little to far, i'm afraid. I start to get some picture, but it's pretty lowres, Thanks for the quick reply.

[ivannaz]

Regarding this idea of software frame synchronization, I see that I could start the cameras from a LAN trigger and perhaps achieve a ms level sync.

But if we have continuous image acquisition afterwards, is it possible to induce a small camera delay in one of the raspberries, should the program in this particular unit finds is it too much advanced compared to a reference unit?

Through the current mmal calls, I mean. I am not asking for GPU code changes. The idea here would be to keep an array of raspberries more or less (+/- 1ms) in sync, so if we start them in sync, and a few of them slowly skew due to small clock difference, we could compensate during acquisition.

We can, of course, stop everything and start again, but will loose a few frames in this process. Is there an alternative besides the already mentioned concept of hardware hacking the clocks?

[6by9]

Regarding this idea of software frame synchronization, I see that I could start the cameras from a LAN trigger and perhaps achieve a ms level sync.

But if we have continuous image acquisition afterwards, is it possible to induce a small camera delay in one of the raspberries, should the program in this particular unit finds is it too much advanced compared to a reference unit?

Through the current mmal calls, I mean. I am not asking for GPU code changes. The idea here would be to keep an array of raspberries more or less (+/- 1ms) in sync, so if we start them in sync, and a few of them slowly skew due to small clock difference, we could compensate during acquisition.

We can, of course, stop everything and start again, but will loose a few frames in this process. Is there an alternative besides the already mentioned concept of hardware hacking the clocks?

You can try tweaking MMAL_PARAMETER_VIDEO_FRAME_RATE on the respective devices. It allow steps of 1/256fps, although there is some rounding in the sensor driver too due to the way framerate control is done in the sensor. Have fun writing the control loop!

[ivannaz]

Thank you. I will try it and report here the result.

[sinc]

The FREX pin is a global reset of all pixels. Line 1 will then start reading out at the end of the configured exposure time, but readout is the same as in rolling shutter mode. If you don't have an external shutter, then the lower lines will have a steadily increasing exposure time as the frame is read out.
Therefore FREX is not a feasible feature to use with the standard Pi Camera.

While it's certainly true that you can get some additional and uneven exposure, where this comes in handy is when the flash illumination is significantly greater than the ambient light. OV FREX, or Global Reset can be used quite effectively, especially with Xenon, but even with LED depending on the lighting dynamics. In darker ambient condition, it can be utilized to take very well exposed photos.

[prutchi]

Hi!

I recently purchased an Arducam Rev.C OV5647 camera for the Raspberry Pi. I can confirm that it works just as the official Raspberry Pi camera, but also has a separate set of pads with pinouts for the STROBE and FREX signals. However, Arducam does not yet have a way for activating the firmware register that enables these lines.

Has anyone been able to find a way of turning on STROBE and FREX functionality?

Cheers,

David

[6by9]

I recently purchased an Arducam Rev.C OV5647 camera for the Raspberry Pi. I can confirm that it works just as the official Raspberry Pi camera, but also has a separate set of pads with pinouts for the STROBE and FREX signals. However, Arducam does not yet have a way for activating the firmware register that enables these lines.

Has anyone been able to find a way of turning on STROBE and FREX functionality?

Google "ov5647 datasheet" and you'll get the hit http://www.seeedstudio.com/wiki/images/ ... 7_full.pdf
Pick up the rawcam stuff from viewtopic.php?f=43&t=109137&sid=132519e ... 14ddc90a80, read the datasheet to determine which mode of operation you want, and edit the register set appropriately. Don't complain when it doesn't do what you want.

Following Broadcom's shuttering of the mobile baseband division, the only person really left updating the camera firmware is me, and that is on a voluntary and "spare time" basis.
As the STROBE and FREX lines are not exposed on the Foundation's Pi camera board there is little motivation in me adding that sort of functionality to the firmware.

[jbeale]

I recently purchased an Arducam Rev.C OV5647 camera for the Raspberry Pi.

Where did you purchase this? Do you know where I can get one (shipped to me in USA) ? The US-based Arducam distributor, Marlin P. Jones http://www.mpja.com/ does not list this particular board.

[6by9]

As the STROBE and FREX lines are not exposed on the Foundation's Pi camera board there is little motivation in me adding that sort of functionality to the firmware.

After a thought, I'll relent a little.
Iff someone else takes the rawcam stuff and datasheet to put together a register set that is actually useful, I will look into integrating that into the main firmware under a config.txt option. (Even that may be tricky as there aren't hooks for per frame updates).
Over to you as the community if you want this.

[jbeale]

If I can get obtain a board with STROBE / SYNC outputs, I plan to look into this. I would like to reliably know at the millisecond level when the start of frame is. This can be useful even with the rolling shutter.

[NTE12]

How do I feed an external clock signal to the camera? Do I physically have to remove the Y1 or is it enough to cut the power supply line on the camera's flat cable (=remove supply from Y1) and bring new clock to the little round test point next to Y1?

Hello Kozuch,
I am also interested in following this approach for camera synchronization. Did you find out what you have to do to disable the clock?
Also, is there a command for controlling the camera's clock? That would prevent one from having to touch the hardware.

[mitchb]

6by9: "Iff someone else takes the rawcam stuff and datasheet to put together a register set that is actually useful"

I would be interested in support for STROBE and FREX, but I'm not sure what would be useful to you to implement that. Maybe the following is adequate. Since my main interest is STROBE, that is all I have here. Maybe it would be more manageable to implement just STROBE support first and FREX support later.

What I (and probably others) would like is to have STROBE pulse during each frame during the time illumination should occur. Four strobe modes are available - xenon, LED1, LED2 and LED3. LED3 is not useful since it just outputs a delayed version of the strobe request bit (0x3B00[7]). The xenon flash mode sets STROBE for a short (1-4 row periods) pulse at the start of the third frame after the strobe request. To me, this is not that useful and also it could be derived from the LED1 or LED2 modes, so I will focus on them. Also xenon mode requires that the strobe request be actively set and reset in order to generate new pulses.

LED1 and LED2 modes are the same except that, in LED2, AWB is on. In these modes, STROBE is set when illumination should occur for the third frame following strobe request. If strobe request is left set, STROBE will continue to be asserted at the appropriate illumination time with a programmable number of frames skipped between the frames [1]. I do not know whether 0 skipped frames are possible. 1 skipped is possible since that is shown in the datasheet (Figure 4-11). The number of skipped frames is set in registers LED_ADD_ROW (0x3A1C, 0x3A1D where 0x3A1C is led_add_row[15:8] and 0x3A1D is led_add_row[7:0])

Summary - To enable STROBE high when illumination should occur and AWB off:

LED_ADD_ROW (0x3A1c:0x3A1D) = 0x0000 (if it works) or 0x0001 // number of frames skipped between STROBEs
STROBE_CTRL (0x3B00) = 0x01 // [7] = strobe request off, [1:0] = mode LED1
STROBE_CTRL (0x3B00) = 0x81 // [7] = strobe request on, [1:0] = mode LED1

[1] LED1 & LED2 mode - multiple pulse output (OV5647 datasheet Sect. 4.9.1)

Mitch

[6by9]

Except AWB and AGC are not run on the OV5647, they are run on the Broadcom ISP. So what is the OV5647 going to try and do to AGC during the flash images?

And on capture the Broadcom code is almost always going to stop and restart the sensor. It then expects the first frame to be corrupt (as it normally is), but if the OV5647 has dropped the frame, the first frame isn't corrupt, and the Broadcom code will drop the wanted frame. So that mechanism would need reworking.

This is why I have passed the issue over to the community.
If and only if someone comes up with a proven workable register set, then I will investigate further and look at integrating it. Just reciting chunks from the datasheet is not sufficient.

There is already the GPIO based flash driver (viewtopic.php?f=43&t=83484) which will almost provide what you want. It will assert a GPIO for the duration of the drop frame and the required frame (mainly as the GPU doesn't have accurate information as to when exposure starts), and drops it when the frame end occurs for the wanted frame. [1]

(Just looking at figure 4-10 of the OV datasheet, it almost looks like in LED 1 and LED2 the strobe line is set for LESS than the full exposure time - it comes on as the last line starts exposing, and goes off as the first line starts readout. Either that is a badly drawn diagram, or don't go trying to do flash captures with an exposure time less than the readout time. More fun things for the community to check out!)

[1] It also does a preflash to calibrate the AGC and flash power (should the flash driver have intensity control, which this doesn't) that will give a pulse on the GPIO, though I may look into whether I can shift that to an extra GPIO so that the main capture flash is on a unique GPIO.

[Kozuch]

I had a conversation with Lee who designed the ArduCAM cameras - they have an interesting boards now called Arducam Mini which uses OV sensors too. Those are SPI cameras that can connect to RPi too. Both the sensors they use have the FREX feature and Lee is willing to enable the feature on the 5 Mpix camera that exposes a FREX pin. However, he has exactly the same problem as we have - he needs help with camera register settings. So if anyone is willing to help with registers for FREX on OV5642 help is welcome. You can contact Lee at admin<thisisatsign>arducam.com.

[6by9]

Those are SPI cameras that can connect to RPi too. Both the sensors they use have the FREX feature and Lee is willing to enable the feature on the 5 Mpix camera that exposes a FREX pin. However, he has exactly the same problem as we have - he needs help with camera register settings. So if anyone is willing to help with registers for FREX on OV5642 help is welcome. You can contact Lee at admin<thisisatsign>arducam.com.

Do they not have contacts at Omnivision? If they're shipping OV modules I would have expected them to have, and OV do have Apps Engineers there just to support customer projects.

[jbeale]

My experience is that OV is selective about who they support. They were not interested in talking with me when I worked for a fairly small ($100M) company. OTOH these guys https://embeddedmicro.com/products/camera-shield.html are a small startup. I met them at MakerFaire and they got OV support at least at the level of basic register sets; so maybe it's who you know, or the story you tell?

[DaveSemm]

I just received my OV5647 Camera Board /w CS mount Lens fully compatible with Raspberry Pi, and rigged up a simple motion detecting camera with an old Pi B board. The images are usually wonderfully sharp, but sometimes the main subject is blurred, because, after all, the shot is triggered by motion...

So I was just about to ask how I can make use of that enticing looking Strobe connection to trigger a LED flash (or iR flash), when I came across this thread and noticed it is getting a bit heated...

Has anyone made progress with a simple way to add flash support for this camera? Or is that not going to happen?

BTW I also just now installed the new Jessie version of Raspbian and am very impressed with how it has resolved many of the old issues. Maybe it is also more flash friendly?

[6by9]

Has anyone made progress with a simple way to add flash support for this camera? Or is that not going to happen?

How about the flash driver? viewtopic.php?f=43&t=83484
It doesn't use the strobe pin, but provides the option to trigger an external light source for stills captures with appropriate exposure compensation (requires a preflash).

[DaveSemm]

Has anyone made progress with a simple way to add flash support for this camera? Or is that not going to happen?

How about the flash driver? viewtopic.php?f=43&t=83484
It doesn't use the strobe pin, but provides the option to trigger an external light source for stills captures with appropriate exposure compensation (requires a preflash).

Thanks! I will try it out!