pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

help! long exposure night photos timing out - SOLVED

Mon Jul 18, 2016 8:46 am

I'm using a simple python3 script to capture a sequence of images at night with a camera v2 on a pi zero. It keeps failing after just 2 or 3 pics. Decreasing the shutter time helps (but I get a black picture). Increasing the delay between calls to capture helps a little - but still fails eventually. I have increased GPU memory split - no change.

The script appears to fail more quickly during the day (but pictures are grossly overexposed of course)

This is the failure:

Code: Select all

Traceback (most recent call last):
  File "repeattest.py", line 13, in <module>
    camera.capture(fn)
  File "/usr/lib/python3/dist-packages/picamera/camera.py", line 1382, in capture
    'Timed out waiting for capture to end')
picamera.exc.PiCameraRuntimeError: Timed out waiting for capture to end
>>> 
and here is the script: (based on the low light example in http://picamera.readthedocs.io/en/relea ... -low-light

Code: Select all

from picamera.camera import PiCamera
from fractions import Fraction
from time import sleep
#camera = PiCamera(resolution=(3280, 2464), framerate=Fraction(1, 10))
camera = PiCamera(resolution=(1640, 1232), framerate=Fraction(1, 10))
camera.shutter_speed = 1000000
camera.awb_mode='cloudy'
camera.exposure_mode = 'off'
camera.iso=100
for i in range(20):
    fn='nightz%02d.jpg' %i
    print(fn)
    camera.capture(fn)
    sleep(1)
Last edited by pootle on Wed Jul 20, 2016 8:51 am, edited 1 time in total.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7533
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 9:34 am

You've asked for 1/10th fps, so 10s per frame, regardless of shutter speed.

Code: Select all

camera = PiCamera(resolution=(1640, 1232), framerate=Fraction(1, 10))
The PiCamera timeout appears to be 30s.
On capture the current frame is always completed which may take up to the frame period of 10s. It then reprograms the sensor, starts it back up again, drops the first frame when it comes in 10s later as it is always corrupt, and then processes the next frame to make your capture. (up to 10)+10+10 = pretty close to 30s under some circumstances.

After the capture, assuming burst mode is not requested, the sensor switches back to preview which involves the same process, so another 20s sequence as the current frame is already completed. Issue another capture request during that and you're almost certainly going to go over the 30s timeout.

Perhaps a request to waveform80 to make the timeout equivalent to 5 or 6 frame periods would be reasonable - https://github.com/waveform80/picamera/issues
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 10:21 am

Wow that's quite a long winded process! Is there any way to capture a sequence of stills with a much smaller gap between frames - or do I have to use the video port for that?

I was aiming to use the camera for meteor showers, so the smaller the gap between exposures the better.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7533
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 10:37 am

video port, or burst mode flag.

If you think about it from a system level then you have to give some answer to what to do after completing the capture. You have an active viewfinder output and nothing else - make a decision.
Normally returning to viewfinder is the correct answer. For the use case where the user still has their finger on the shutter button then you set burst mode before starting the capture first capture, and release it when the button is released, and that then gives the GPU enough information to change behaviour.

Other approaches would be to run preview at full resolution by forcing the sensor mode to mode 2. That reduces the frame rate during preview (and remember that those frames are what AGC and AWB adapt based on) to a max of 15fps, but should also remove the need to stop the sensor at all. The V1.3 OV5647 sensor effectively did that anyway as only one mode supported exposure times >1sec, so asking for a very low frame rate automatically removed the mode switch.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

gordon77
Posts: 4295
Joined: Sun Aug 05, 2012 3:12 pm

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 10:46 am

Could you use raspistill and set max -ss at 6 seconds and set -bm, burst mode.

Could you increase the iso you only have 100 set. ?

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 11:33 am

iso 100 would be too low - here is one I took last night and the overall exposure is just about right as the sky glow is just visible near the zenith.

full size http://imgur.com/a/GL04j

Burst mode looks promising - I have done a sequence of 10 OK - since the frame rate is so low I don't have a problem with bandwidth to disc. Also as the pi is headless, I haven't got preview turned on either.

pfile
Posts: 11
Joined: Tue May 21, 2013 3:40 am

Re: help! long exposure night photos timing out

Mon Jul 18, 2016 6:09 pm

yeah i used iso800 + --ss 5900000 to make this video (omnivision board + 120 degree fisheye). there is really bad light pollution here so i'm not sure how that exposure would work under truly dark skies...

https://www.youtube.com/watch?v=Q6tN2D_9QmE

the two starbursts at the side appear to be some kind of hotspots on the sensor. they can be removed by calibrating the raw images with darks, however, the sensor appears to be very sensitive to temperature. the darks i made during the day inside a clothes dresser overcorrect the nighttime images. i am not sure if those patterns appear in the bias frames. perhaps only bias calbration would be necessary to remove them.

skypi
Posts: 111
Joined: Sat Aug 09, 2014 11:48 pm

Re: help! long exposure night photos timing out

Tue Jul 19, 2016 11:14 pm

pootle wrote:I'm using a simple python3 script to capture a sequence of images at night with a camera v2 on a pi zero. It keeps failing after just 2 or 3 pics. Decreasing the shutter time helps (but I get a black picture). Increasing the delay between calls to capture helps a little - but still fails eventually. I have increased GPU memory split - no change.
just had that on a pi3 with a v2 camera

camera.capture(capstream,'rgba',use_video_port=True,resize=(wtest,htest))
File "/usr/lib/python2.7/dist-packages/picamera/camera.py", line 1382, in capture
'Timed out waiting for capture to end')
picamera.exc.PiCameraRuntimeError: Timed out waiting for capture to end

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

long exposure night photos sorted out

Wed Jul 20, 2016 8:39 am

I've fixed the problem by using capture_sequence in burst mode - 6b9's explanation above pretty clearly explained the reasons, so I start the camera at .7 secs exposure (purely arbitrary value that will work...) and then in an iterator I up the exposure time (currently to 9 seconds for a V2 camera) after a couple of photos. After that things all work fine.

I've had no problems with this (from the little use so far) in spite of the rather dire warnings in help(PiCamera.capture_sequence)!

I'm planning to use this to see if I can get a few meteor photos when the Delta Aquarids and Perseids turn up over the next few weeks.

I have a (so far) very rudimentary python3 script to do this if anyone is interested.

weathernick
Posts: 6
Joined: Thu Jun 16, 2016 2:56 pm

Re: help! long exposure night photos timing out - SOLVED

Wed Jul 20, 2016 10:54 pm

pootle wrote:I've fixed the problem by using capture_sequence in burst mode - 6b9's explanation above pretty clearly explained the reasons, so I start the camera at .7 secs exposure (purely arbitrary value that will work...) and then in an iterator I up the exposure time (currently to 9 seconds for a V2 camera) after a couple of photos. After that things all work fine.

I've had no problems with this (from the little use so far) in spite of the rather dire warnings in help(PiCamera.capture_sequence)!

I'm planning to use this to see if I can get a few meteor photos when the Delta Aquarids and Perseids turn up over the next few weeks.

I have a (so far) very rudimentary python3 script to do this if anyone is interested.
How long does it take to process a 9 second exposure? I have been trying to solve this problem with night exposures taking too long and I am thankful I found this forum. I think I need to use burst mode to speed things up. right now I can take a 3 second exposure every 15 seconds. I am hoping to do better with using burst mode.

Thanks,

Nickolas

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: help! long exposure night photos timing out - SOLVED

Fri Jul 22, 2016 6:45 am

I'm currently taking 9 second exposure pics every 10 seconds and saving as a png, but not doing any other processing on them. I'm just about to try setting up a separate process to monitor the pictures - proabably not every one though (at least not on a pi Zero).

15 seconds is extraordinarily long for 3 seconds exposures. Are you running a shell script? I am using Python. You definitely need to use burst mode.

gordon77
Posts: 4295
Joined: Sun Aug 05, 2012 3:12 pm

Re: help! long exposure night photos timing out - SOLVED

Wed Jul 27, 2016 4:51 pm

pootle wrote:I'm currently taking 9 second exposure pics every 10 seconds and saving as a png, but not doing any other processing on them. I'm just about to try setting up a separate process to monitor the pictures - proabably not every one though (at least not on a pi Zero).

15 seconds is extraordinarily long for 3 seconds exposures. Are you running a shell script? I am using Python. You definitely need to use burst mode.
If you run raspistill as a subprocess then you should be able to process a picture whilst it takes the next, as long as the processing doesn't take longer than 9 seconds!.

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: help! long exposure night photos timing out - SOLVED

Wed Jul 27, 2016 5:18 pm

gordon77 wrote:If you run raspistill as a subprocess then you should be able to process a picture whilst it takes the next, as long as the processing doesn't take longer than 9 seconds!.
Its actually the time between exposures I am trying to minimise - a 10 second exposure pic every 10.small seconds. I am analysing them out of process as well, but the only way I've found to take stills with very little gap between is capture_continuous in burst mode.

Iain

gordon77
Posts: 4295
Joined: Sun Aug 05, 2012 3:12 pm

Re: help! long exposure night photos timing out - SOLVED

Wed Jul 27, 2016 6:23 pm

Have you tried this example ....? From here http://picamera.readthedocs.io/en/relea ... amera.html

Code: Select all

import io
import time
import picamera 
with picamera.PiCamera() as camera: 
   stream = io.BytesIO()
   for foo in camera.capture_continuous(stream, format='jpeg'): 
# Truncate the stream to the current position (in case # prior iterations output a longer image)
       stream.truncate()
       stream.seek(0)
       if process(stream): 
          break   
If so how do you read the pictures it takes to process them?

pootle
Posts: 340
Joined: Wed Sep 04, 2013 10:20 am
Location: Staffordshire
Contact: Website

Re: help! long exposure night photos timing out - SOLVED

Thu Jul 28, 2016 7:56 am

gordon77 wrote:Have you tried this example ....? From here http://picamera.readthedocs.io/en/relea ... amera.html

If so how do you read the pictures it takes to process them?
I did look at that, but I am nowhere near having an i/o bandwidth problem so I just write the pics to a scratch folder (could use ramdisk if needed but not so far) and analyse them out of process

gordon77
Posts: 4295
Joined: Sun Aug 05, 2012 3:12 pm

Re: help! long exposure night photos timing out - SOLVED

Thu Jul 28, 2016 8:19 am

Thanks, I do similar at present writing to /run/shm

Return to “Camera board”