Pidestrian
Posts: 3
Joined: Mon Mar 30, 2015 10:43 am

Synchronize PiCam Frame capture with GPIO Output

Mon Mar 30, 2015 1:01 pm

Hi,

I have a problem regarding the synchronized capture of images from the PiCam Module. :?:

Goal:
Capture still images (single shot) and trigger each frame separately.
Sychronize each frame capture with a GPIO output signal (LED).
Time offsets between frames should be below the ms range.

Testing setup:
using RaspberryPi B+ 1.2,wiringpi lib,
V4L (http://www.jayrambhia.com/blog/capture-v4l2/),
RaspiCam C++ API (http://www.uco.es/investiga/grupos/ava/node/40),
raspberrypi/userland/MMAL libs (example code from RaspiStill), openCV
4 LEDs are connected to GPIOs, set as Outputs, triggered one after another.

fixed parameters: resolution 640x480, RGB (CV_8UC3)
varied parameters: shutter time, led delay (delay between LED_ON and frame capture, and LED_OFF/LED_ON)
retrieve delay, ( between frame capture and reading the frame from the framebuffer, depends on the implementation)

What I expected:
the frames are triggered shortly after the gpio output is set, each frame shows exactly one LED at maximum brightness.

What I noticed:
frames are not synchronized. the delay is important, a higher led delay results in a higher chance to get a frame with only one led lit.
This of course partly depends on my setup. However, sometimes images show multiple leds lit at the same time, questioning the blocking-characteristics of some function calls.
(frame capture and readout overlap, 2 frames combined to one image => magic light aka lit surface with all LEDs off ;) )
Also changing the retrieve delay, while keeping all other parameters constant, results in different frames (leds in different states, or completely off)

Summary:
So, basically, V4l, RaspiCam_Still_Cv and raspistill access the Picam in video stream mode.
So far, i found no implementation to really trigger only one still frame.
Instead a random frame is retrieved from an video stream as "still" image.
(otherwise the retrieve delay or multiple readout of a frame from the camera frame buffer would have no effect)

I looked into the mmal implementation of the raspistill demo app.
(I also tried to take code samples and rewrite it, got an ENOSPEC error, when enabling the mmal default camera component)
There is a capture-port defined for the default camera component, however, even this capture port (not the video output port) seems to take only one frame from an intern video stream.

By varying the parameters i eventually found values to almost synchronize the frames.
However, these depend on the used resolution and other factors, thus are likely to change randomly if other settings are used.

Any idea how to actually trigger only one distinct frame with the PiCam?
Is it even possible?
I could imagine, this might be interesting for various applications.
If someone managed to do this, please add a small code example.

thx for your help. :|

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9069
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Synchronize PiCam Frame capture with GPIO Output

Mon Mar 30, 2015 2:00 pm

Numerous issues with what you are attempting.
  • The sensor has a rolling shutter, so the image is not all being exposed at the same time. If the LEDs are being detected by different lines in the sensor then there will be a time offset between them
  • The sensor readout is independent of the ISP, so there is a variable latency between the end of the sensor readout and when the YUV frame is completed.
  • Delivering frames to the client is independent of both the sensor and the ISP.
  • There are about 6 different readout modes from the sensor for different framerate and resolution combinations. The delay between the line which "sees" your LEDs and the end of the frame will vary due to that
So basically I think you are going to struggle to get the sort of synchronisation you're after.

If you're really after a single still capture with synchronisation, then have a look at http://www.raspberrypi.org/forums/viewt ... 43&t=83484
There is an LED flash driver built in to the firmware. That will control one GPIO for the capture, although please note that it will trigger a preflash sequence as it is attempting to compute exposure time and gains for the capture, and it will be enabled for 2 frame periods due to the rolling shutter.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Pidestrian
Posts: 3
Joined: Mon Mar 30, 2015 10:43 am

Re: Synchronize PiCam Frame capture with GPIO Output

Tue Mar 31, 2015 9:11 am

Hi,

thank you for your fast answer.
I'll look into the new Driver.

SimplyOm
Posts: 5
Joined: Wed Jun 01, 2016 8:01 am

Re: Synchronize PiCam Frame capture with GPIO Output

Fri Jun 03, 2016 10:40 am

Hi Pidestrian,
I just read the post you made a year back and realised that I have almost similar objective in my project too. I am also facing problems while trying to synchronise frames with GPIO triggers.
Besides, the frame rate itself is not stable. Even the loop containing only cam.grab() shows latency upto 15 ms.
I just wanted to know were you able to synchronise the frames later using changes in the driver or how far could you reach in your objective?

Return to “Camera board”