6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Hardware camera sync pulses

Mon Aug 07, 2017 4:00 pm

There have been various requests over the years for output sync pulses for frame synchronisation. Generally it's been stated that doing such things isn't trivial, and it wasn't. However, as of the 22nd July firmware, there is support for repurposing the camera LED GPIO to change state on frame start and frame end interrupts.

Rather than adding mounds of new plumbing I've altered the handling of the disable_camera_led property in config.txt:
If 0, the camera LED GPIO goes high when the camera is powered up, and goes off when it is powered down.
If 1, the camera LED GPIO will always remain low.
If 2, the camera LED GPIO will go low on a frame start, and high on frame end.
If 3, the camera LED GPIO will go high on a frame start, and low on frame end.
Frame start = interrupt raised when the end of the first line in the frame is received from the sensor.
Frame end = interrupt raised when the end of the last line in the frame is received from the sensor.
Interrupt latency is normally sub-microsecond on the VPU, so the GPIO timing is pretty tight.

The one major proviso is that the GPIO selected MUST be one of the directly connected GPIOs on the SoC, and that is not the default situation on a Pi3 (it connects to an I2C GPIO expander).
All is not lost though, as on any Pi you can reconfigure the GPIO used for the camera LED via the dt-blob.bin file as detailed in https://www.raspberrypi.org/documentati ... uration.md and https://www.raspberrypi.org/documentati ... -camera.md. Maintaining a dt-blob.bin is a little bit of a pain when new Pi variants are released, but that isn't that frequent, and I'm afraid is just one of those things.

Please read waveform80's description of how the sensor actually works before requesting changes - http://picamera.readthedocs.io/en/lates ... -operation. The frame start/end are the only real events that are available at the SoC, hence that is what is exposed. Start/end of exposure is not visible outside of the sensor.
The period of the pulse will vary with the frame rate, and the mark/space ratio of the sync pulse will vary with the mode - that is to be expected.
Last edited by 6by9 on Sun Aug 05, 2018 1:05 pm, edited 1 time in total.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pibumble
Posts: 5
Joined: Fri Dec 30, 2016 9:28 pm

Re: Hardware camera sync pulses

Tue Nov 14, 2017 5:34 pm

Try as I might I still don't understand anything of the device tree.
A have been at it for some hours now and this is not the first time.
I tried Google and found more people who try to explain but they all are the same:
It is like learning C and the manual starts with talking about pointers, structures and unions
consisting of other pointers to structures, references and labels all at the same time.
I now admire Kernighan even more for explaining a complex language so well and starting so simple.

Which means I still have not got the faintest idea how to move the camera led on the Pi-3.
I looked at the computer module dt-blob-cam1.dts and it has a clear entries.

under pin_config:
[email protected] { function = "output"; termination = "no_pulling"; }; // GPU WILL USE THIS PIN FOR CAMERA 0 LED

under pin_defines:
[email protected]_0_LED {
type = "internal";
number = <2>;
};

but the bcm2710-rpi-3-b.dts has neither a pin_config entry nor a pin_defines one!

Can somebody just tell me what text to add where to get the camera LED on GPIO pin X?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Tue Nov 14, 2017 5:48 pm

dt-blob.dts/bin is NOT the kernel device tree, it just reuses the syntax and compiler.

Start with https://github.com/raspberrypi/firmware ... t-blob.dts which is the source for the blob that is built in to the firmware.
For a Pi3, find the pins_3b2 section (line 1157).
Line 1226 is [email protected]_0_LED. Amend that section appropriately. "external" means it is on the GPIO expander which starts at GPIO 128. You probably want to reset it to "internal" if you want to use one of the normal GPIO lines.
You also need to define your chosen GPIO as an output with no pulling in the "[email protected]" list.

Compile as per https://www.raspberrypi.org/documentati ... uration.md

Code: Select all

sudo dtc -I dts -O dtb -o /boot/dt-blob.bin dt-blob.dts
and copy the resulting dt-blob.bin file to /boot on the target Pi.
Reboot.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pibumble
Posts: 5
Joined: Fri Dec 30, 2016 9:28 pm

Re: Hardware camera sync pulses

Wed Nov 15, 2017 9:52 am

Thank you.
That is the sort of answer a noob like me understands: 'recipe type'.
Do A followed by B then mix in C and cook for D minutes.

I managed to move the camera led to GPIO 21 (at the corner of the connector so it is easier to hang a scope probe off.)
The line goes high and low.
When I add disable_camera_led=1 The line no longer goes high or low.
However disable_camera_led=2 or 3 give no change : The line no longer goes high or low.

Which made me think I have not got the right firmware
uname -a produces:
Linux raspberrypi 4.4.50-v7+ #970 SMP Mon Feb 20 19:18:29 GMT 2017 armv7l GNU/Linux
So that is from Feb and your fix is in July

Followed the instructions how to get latest release from here: https://www.raspberrypi.org/documentati ... pdating.md
(Which takes a while to complete)
Now uname gives: Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux
Which is STILL before your build of 22 July.

That is how far my knowledge goes.
What do I need to do to pull in the latest MMAL version?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Wed Nov 15, 2017 10:22 am

pibumble wrote:
Wed Nov 15, 2017 9:52 am
Followed the instructions how to get latest release from here: https://www.raspberrypi.org/documentati ... pdating.md
(Which takes a while to complete)
Now uname gives: Linux raspberrypi 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux
Which is STILL before your build of 22 July.

That is how far my knowledge goes.
What do I need to do to pull in the latest MMAL version?
Almost certainly you're still running a Raspbian Jessie release, not Stretch which was released in August. Run "cat /etc/os-release" to confirm - it'll contain either PRETTY_NAME="Raspbian GNU/Linux 8 (jessie)", or PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)" as the first line. Jessie no longer gets significant updates.

Unless you have a very good reason to stick with Jessie, then I'd suggest you download a fresh image from https://www.raspberrypi.org/downloads/raspbian/ and start with that.
(Iff you have a very good reason to stick with Jessie, then you can use rpi-update to grab the absolute latest kernel and firmware, however those releases have less testing that the main Raspbian updates, so issues can sometimes creep in).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pibumble
Posts: 5
Joined: Fri Dec 30, 2016 9:28 pm

Re: Hardware camera sync pulses

Wed Nov 15, 2017 11:39 am

all's well that ends well.

I got beautiful pulses on my scope.
Thank you.

Now for the difficult part (as from here I have to do it myself) play with the results.

Kozuch
Posts: 62
Joined: Sun Aug 10, 2014 10:35 am

Re: Hardware camera sync pulses

Thu Apr 19, 2018 2:36 pm

Nice to hear we've got a precise frame signal from the camera! You do not mention any camera modes so I guess this will work in ANY camera mode (video, stills), am I right? I also guess this may be working regardless of other camera settings (shutter speed, ISO etc.). Would you confirm please?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Thu Apr 19, 2018 3:14 pm

Kozuch wrote:
Thu Apr 19, 2018 2:36 pm
Nice to hear we've got a precise frame signal from the camera! You do not mention any camera modes so I guess this will work in ANY camera mode (video, stills), am I right? I also guess this may be working regardless of other camera settings (shutter speed, ISO etc.). Would you confirm please?
Correct. This is called from the frame start and frame end interrupt handling of the CSI2 receiver, so it is active for every frame going in any mode with any settings. You will have very minor interrupt latency jitter on the signal, but the VPU is pretty good in that regard (certainly better than the ARMs!)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Kozuch
Posts: 62
Joined: Sun Aug 10, 2014 10:35 am

Re: Hardware camera sync pulses

Wed Apr 25, 2018 11:05 pm

Do the sync pulses respect the framerate setting done by --framerate/-fps parameters of RaspiVidYUV? If I request a framerate of 5 Hz, does the sensor run at 5 Hz (and hence give 5 Hz sync pulses) or is the framerate lowered in software with the sensor running at a higher framerate?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Thu Apr 26, 2018 6:26 am

Kozuch wrote:
Wed Apr 25, 2018 11:05 pm
Do the sync pulses respect the framerate setting done by --framerate/-fps parameters of RaspiVidYUV? If I request a framerate of 5 Hz, does the sensor run at 5 Hz (and hence give 5 Hz sync pulses) or is the framerate lowered in software with the sensor running at a higher framerate?
Last line of the first post
The period of the pulse will vary with the frame rate, and the mark/space ratio of the sync pulse will vary with the mode - that is to be expected.
Frame rate control (on the Pi at least) is always done by varying the blanking period on the sensor. Some systems vary the PLL setup to change the pixel clock, but that is significantly more involved.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fishbeetle
Posts: 6
Joined: Tue Apr 03, 2018 4:26 pm

Re: Hardware camera sync pulses

Tue May 15, 2018 1:23 pm

Hi There,
I'm trying to use the LED flash sync feature in continuous grab mode on a Pi3B with a v2 camera. I followed the instructions from above and was able to get the flash pulse(s) for single grabs but in continuous grab mode (with exposure_mode off) I get no pulses. Should this possible?
---
EDIT
Subsequently, I re-tried disable_camera_led=2 in config.txt & re-compiling an edited dt-blob file to map IO17 to camera-0-LED and it works
perfectly.
---
Thanks again.
Steve.
Last edited by fishbeetle on Tue May 15, 2018 3:42 pm, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Tue May 15, 2018 2:53 pm

fishbeetle wrote:
Tue May 15, 2018 1:23 pm
I'm trying to use the LED flash sync feature in continuous grab mode on a Pi3B with a v2 camera. I followed the instructions from above and was able to get the flash pulse(s) for single grabs but in continuous grab mode (with exposure_mode off) I get no pulses. Should this possible?
Thanks for your time.
single grab and continuous grab mode? Do you mean the capture and capture_continuous methods from Picamera?
I'm also concerned you've used the word flash. The usage with the simple flash driver as documented here is not the same as the hardware camera sync pulses. Which one are you using?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fishbeetle
Posts: 6
Joined: Tue Apr 03, 2018 4:26 pm

Re: Hardware camera sync pulses

Tue May 15, 2018 8:34 pm

Thanks for responding, yes I should have said capture_continuous using PiCamera in Python. To be precise, I want to drive an IR illuminator with the pulse signal, if not directly then via some custom pulse shaping network. Any advise would be much appreciated. I'll be using the camera in fully manual mode, shutter speed, gain (ISO) set by software and at a fixed frame rate of 3 - 6 fps.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Wed May 16, 2018 9:51 am

fishbeetle wrote:
Tue May 15, 2018 8:34 pm
Thanks for responding, yes I should have said capture_continuous using PiCamera in Python. To be precise, I want to drive an IR illuminator with the pulse signal, if not directly then via some custom pulse shaping network. Any advise would be much appreciated. I'll be using the camera in fully manual mode, shutter speed, gain (ISO) set by software and at a fixed frame rate of 3 - 6 fps.
OK, I'm still not clear whether you're using the sync pulses described here by setting disable_camera_led=2 or 3, or using the flash driver as described in the Picamera docs.
I have the impression it's the flash driver - that is known to have limitations if exposure mode is off and burst mode (it can't do the precalcs for flash intensity and adjusting the exposure/gain based on the effect the preflash has on the scene).

If you're wanting to drive an IR illuminator off the sync pulses described here then I think you're a little stuck. They trigger on the frame start and end interrupts at the SoC, so the end of exposure of the first and last lines. The SoC has no notion of when the frame exposure starts other than running a timer from either frame start or end. With a rolling shutter the frame exposures will often overlap too, so you never get to turn the illumination off.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fishbeetle
Posts: 6
Joined: Tue Apr 03, 2018 4:26 pm

Re: Hardware camera sync pulses

Wed May 16, 2018 2:59 pm

OK, couldn't get the flash driver to work in capture_continuous, but setting disable_camera_led=3 does generate the pulses I need. And driving the IR illuminator directly with these pulses works like a dream. I'm using exposure_mode=off and a fixed shutter speed of just 1-2ms and at 6 frames per second I'm able to save a huge amount of power in the nominally 40W illuminator - about 85% in fact. So now I have all I need, thank you again for you help and thank you to all involved in the v2 camera product - astonishing performance for the price and great support too. Five stars.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Wed May 16, 2018 3:20 pm

fishbeetle wrote:
Wed May 16, 2018 2:59 pm
OK, couldn't get the flash driver to work in capture_continuous, but setting disable_camera_led=3 does generate the pulses I need. And driving the IR illuminator directly with these pulses works like a dream. I'm using exposure_mode=off and a fixed shutter speed of just 1-2ms and at 6 frames per second
I don't know which mode you're running, but typically on IMX219 a line takes ~19us to read out. If you're turning on the illumination at the frame start interrupt then the top line will have had no extra illumination, and on a 1ms exposure time line 52 (1000us / 19us) is just starting to expose. Therefore there will be a gradual increase in brightness between lines 1 and 52.
That may be acceptable in your situation, but I wanted to flag it up.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fishbeetle
Posts: 6
Joined: Tue Apr 03, 2018 4:26 pm

Re: Hardware camera sync pulses

Wed May 16, 2018 5:08 pm

Indeed so, and to do the job properly I would have to generate a modified pulse stream with leading edges one exposure-time earlier than that of the frame sync pulses from the IO pin. This would be a straightforward matter with a simple PIC processor but, as it happens, it's not a problem in my application. Thanks again for the clarification.

raspi.td
Posts: 1
Joined: Sun May 20, 2018 8:53 am

Re: Hardware camera sync pulses

Sun May 20, 2018 9:07 am

Some noob questions which might be also relevant for other people:
Is it possible to record this values to a file? So record a table with frame start and frame end times?
Is it possible to use this information to record the exact time periode during which an still image was taken?
If I use capture_continuous in PiCamera as described above, does than each pulse equal one image and can I solow down the capture frequency to corespond with my sd card speed?

Sorry for the many questions. I looked around a lot but could not find an answer for those questions and I'm still not enough into the subject to understand it by my own.

Thank you for any answers or hints, Tobias

Kozuch
Posts: 62
Joined: Sun Aug 10, 2014 10:35 am

Re: Hardware camera sync pulses

Mon May 21, 2018 8:01 pm

6by9 wrote:
Tue Nov 14, 2017 5:48 pm
Line 1226 is [email protected]_0_LED. Amend that section appropriately.

Could someone please provide me with a sample part of https://github.com/raspberrypi/firmware ... t-blob.dts for Pi 3 v1.2 (pins_3b2) if I want to map the camera led to GPIO21 (pin #40) to get the sync pulses there? In [email protected]_0_LED section should I just change type to "internal" and number to 21?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5954
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Hardware camera sync pulses

Tue May 22, 2018 2:43 pm

Kozuch wrote:
Mon May 21, 2018 8:01 pm
6by9 wrote:
Tue Nov 14, 2017 5:48 pm
Line 1226 is [email protected]_0_LED. Amend that section appropriately.

Could someone please provide me with a sample part of https://github.com/raspberrypi/firmware ... t-blob.dts for Pi 3 v1.2 (pins_3b2) if I want to map the camera led to GPIO21 (pin #40) to get the sync pulses there? In [email protected]_0_LED section should I just change type to "internal" and number to 21?
Yes, exactly that.

Code: Select all

            [email protected]_0_LED {
               type = "internal";
               number = <21>;
};
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

fishbeetle
Posts: 6
Joined: Tue Apr 03, 2018 4:26 pm

Re: Hardware camera sync pulses

Wed May 23, 2018 5:26 pm

>If I use capture_continuous in PiCamera as described above, does than each pulse equal one image and can I solow down the capture frequency to corespond with my sd card speed?

Yes and Yes.

re time-stamping for the frame-sync pulses - I don't know of a way to do this but if it's possible I would be interested too.

Steve.

gius
Posts: 8
Joined: Thu Jun 07, 2018 10:04 pm

Re: Hardware camera sync pulses

Wed Aug 01, 2018 8:58 am

Hi,
I am a member of a no profit educational association.
We have a project of taking pano pictures of places of historic and artistic interest. In most of scenes we can not use a single camera because of moving objects/subjects in the scene.
We are planning to built a set of cams to take panoramic pictures, not video.
We are taking into consideration a set of rpi's and camera modules 5mp to shoot simultaneously (with a delay of 1, 2 milliseconds) ideally "pushing a button", by way of proper hardware connections and adequate software.

I read the following topics, not exactly on this matter, regarding synchronization in some way:
viewtopic.php?t=84383#p791456
viewtopic.php?f=43&t=48238&start=75

As far as I may understand (no knowledge in electronics) conclusion is that it is not possible to synchronize the set I described to take pictures at the same time.

I am writing to check, to be sure, I understood correctly.

Thank you.

Giuseppe

Kozuch
Posts: 62
Joined: Sun Aug 10, 2014 10:35 am

Re: Hardware camera sync pulses

Wed Aug 01, 2018 3:28 pm

Sync of single photo should be possible (long term video is not, both cameras will drift, because they each have separate oscilator). See here for more info: https://www.raspberrypi.org/blog/frozen ... pberry-pi/

You will have a setup similar to the one liked, but probably fewer cameras.

gius
Posts: 8
Joined: Thu Jun 07, 2018 10:04 pm

Re: Hardware camera sync pulses

Wed Aug 01, 2018 6:41 pm

Thank you for hint.
good to hear it would be possible.
I do not understand how it works and I can not find any explanation on that page. I will try to ask the guys at piface...
We would need a set of 12, or maybe more.., cams; if that is possible any suggestions from anyone who knows how to proceed would be much appreciated

gordon77
Posts: 3501
Joined: Sun Aug 05, 2012 3:12 pm

Re: Hardware camera sync pulses

Thu Aug 02, 2018 8:08 am

6by9 wrote:
Mon Aug 07, 2017 4:00 pm
There have been various requests over the years for external triggers for frame synchronisation. Generally it's been stated that doing such things isn't trivial, and it wasn't. However, as of the 22nd July firmware, there is support for repurposing the camera LED GPIO to change state on frame start and frame end interrupts.....



I don't quite see how this externally triggers cameras, it appears it just shows you when a picture is being taken. Anyone got any details how it can be used to trigger cameras?

Return to “Camera board”