xtal256
Posts: 14
Joined: Sat Aug 26, 2017 8:34 am

How to get lowest latency with PiCamera

Mon May 13, 2019 11:35 am

Hello,

I have a Raspberry Pi 3 and a Pi camera (specifically, the mini "spy" variant) which captures video and outputs to a HDMI connected LCD screen. I'd like to also save the video to file, but showing it on screen is a must.
I am looking to get the highest framerate and lowest latency possible.

Currently, I use the following setup:

Code: Select all

camera = picamera.PiCamera()
camera.exposure_mode = 'sports'  # To reduce motion blur
camera.framerate = 49  # Highest supported by mode 5
camera.start_preview(fullscreen=True)
...
That framerate is set based on the documentation, which is the highest framerate at a decent resolution (mode 5: 1296x730).
Apparently, the spy camera only comes in a V1 module, not V2. (camera.revision returns 'ov5647').

However, latency is still a big problem. When I swipe my hand in front of the camera, there is a noticeable delay before it shows on the screen. I'd say about 200ms.

Is there any better settings I could use? Might it be possible to get better performance by delving into the picamera's more detailed API, or even MMAL?
I have seen some blogs and tutorials use "picamera.array", but I'm not sure if it would be of any benefit to me.


Has anyone had any experience with this, or have any tips?

thanks
Last edited by xtal256 on Tue May 14, 2019 10:05 am, edited 1 time in total.

Andyroo
Posts: 2880
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: How to get lowest latency with PiCamera

Mon May 13, 2019 1:05 pm

I would have a hunt through the posts of HermannSW via memberlist.php?mode=viewprofile&u=198596

He is the master of high speed photography on the Pi.
Need Pi spray - these things are breeding in my house...

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How to get lowest latency with PiCamera

Mon May 13, 2019 1:34 pm

Andyroo wrote:
Mon May 13, 2019 1:05 pm
I would have a hunt through the posts of HermannSW via memberlist.php?mode=viewprofile&u=198596

He is the master of high speed photography on the Pi.
But not for real-time display. He's capturing the raw Bayer data and post-processing it.

I've been through where any latency comes from numerous times in the past. Memory says I've had frames H264 encoded <120ms from receiving the first line of pixels. Display is generally less than that.
What refresh rate is your monitor in? 30Hz vs 60Hz will be adding an extra 16ms. "tvservice -s"

PiCamera is using MMAL underneath, but it'd be worth running just "raspivid -t 0" as a native C user of MMAL.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

xtal256
Posts: 14
Joined: Sat Aug 26, 2017 8:34 am

Re: How to get lowest latency with PiCamera

Tue May 14, 2019 6:32 am

Hi 6by9,

Are you saying that you have made blogs/posts on latency in the past? Could you provide some links?

My monitor is 1920x1080 @ 60Hz. In order to get the Pi to output at the correct mode when the screen is turned off, I have set the following video config (after making sure the monitor supported that mode):

Code: Select all

hdmi_group=1
hdmi_mode=16

I will see if "raspivid -t 0" can give better results than the Python code, but I'd really like to use the overlay features of the PiCamera's Python API.

xtal256
Posts: 14
Joined: Sat Aug 26, 2017 8:34 am

Re: How to get lowest latency with PiCamera

Tue May 14, 2019 10:44 am

After reading the doco in more detail, it appears I have been doing it wrong.

I thought that just setting the framerate would be enough for it to detect that it needs to use sensor mode 5. But it may have still been at 1920x1080 resolution and using a different sensor mode.

I have now forced it to use mode 5 like so:

Code: Select all

camera = picamera.PiCamera(sensor_mode=5)
#camera.exposure_mode = 'sports'  # To reduce motion blur. Does not seem to make much difference.
camera.framerate = 49  # Highest supported by mode 5
and am now seeing much lower latency.

That being said, I am still aiming for the lowest possible. So if anyone has any tips I'd much appreciate it.

xtal256
Posts: 14
Joined: Sat Aug 26, 2017 8:34 am

Re: How to get lowest latency with PiCamera

Mon May 20, 2019 4:28 am

6by9 wrote:
Mon May 13, 2019 1:34 pm
I've been through where any latency comes from numerous times in the past.
Hi 6by9. I'd like to learn more about latency on the picamera. Could you please link me to some of these discussions you mention? When I search your posts for the term "latency", I get 170 results. Are there any posts in particular that you are thinking of?

Also, a related question. Does the camera preview bypass the CPU and send the captured frames directly to the display using the GPU? Or is it necessary for some processing to happen on the CPU?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6650
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: How to get lowest latency with PiCamera

Mon May 20, 2019 9:42 am

xtal256 wrote:
Mon May 20, 2019 4:28 am
6by9 wrote:
Mon May 13, 2019 1:34 pm
I've been through where any latency comes from numerous times in the past.
Hi 6by9. I'd like to learn more about latency on the picamera. Could you please link me to some of these discussions you mention? When I search your posts for the term "latency", I get 170 results. Are there any posts in particular that you are thinking of?
https://www.raspberrypi.org/forums/view ... 8#p1179652
https://www.raspberrypi.org/forums/view ... 0#p1004792
xtal256 wrote:Also, a related question. Does the camera preview bypass the CPU and send the captured frames directly to the display using the GPU? Or is it necessary for some processing to happen on the CPU?
Yes. If you read the code for any of the raspicam apps you'll find that it uses a mmal_connection with the MMAL_CONNECTION_FLAG_TUNNELLING flag set between all components. This keeps the connection on the GPU, therefore the ARM doesn't need to get involved.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.


Return to “Camera board”