Page 5 of 5

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 12:51 pm
by ethanol100
Ok, with your command I get similar timings. But with setting "-tl" and a non zero value for "-t" I get this:

Code: Select all

 raspistill -tl 6000 -fp --nopreview -bm -v -set -ISO 100 -awb off -awbg 1.1,1 -t 60000 -ss 6000000 -o t_%03d.jpg

create the following files:

Code: Select all

-rw-r--r-- 1 pi 175084 2014-09-11 14:42:11.914278294 +0200 t_001.jpg
-rw-r--r-- 1 pi 181050 2014-09-11 14:42:17.934695175 +0200 t_005.jpg
-rw-r--r-- 1 pi 184483 2014-09-11 14:42:23.955112053 +0200 t_006.jpg
-rw-r--r-- 1 pi 184941 2014-09-11 14:42:29.975528930 +0200 t_007.jpg
-rw-r--r-- 1 pi 187798 2014-09-11 14:42:35.995945803 +0200 t_008.jpg
-rw-r--r-- 1 pi 190114 2014-09-11 14:42:42.016362675 +0200 t_009.jpg
-rw-r--r-- 1 pi 192838 2014-09-11 14:42:48.026778851 +0200 t_010.jpg
-rw-r--r-- 1 pi 194460 2014-09-11 14:42:54.057196411 +0200 t_011.jpg
Images 5 to 11 are all 6 sec apart. There is only a delay after the first capture which raspistill reports with "mmal: Skipping frame 2 to restart at frame 5".

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 5:19 pm
by fehlfarbe
Geat! The >>non zero value for "-t"<< was the problem :)

Is there a way to send the last timelapse image file to a running process, e.g. a python script that creates a running average?

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 6:11 pm
by ethanol100
For post processing it is easier to not use jpg but the raw(yuv or rgb) data using raspiyuv. You can use

Code: Select all

raspiyuv [options you wish] -o - | ./someProgram
to pipe the image to a program.

For yuv or rgb you know how much bytes are send for each image.
(for yuv you get Y=h*b bytes then U=h*b/4 bytes and finally V=h*b/4 bytes,
for rgb you get R=h*b bytes, then G=h*b bytes and finally B=h*b bytes["-rgb switch in raspiyuv"])

You could read one image from stdin a time into a buffer and then use numpy's function "frombuffer()" to create the pixel data array.
With this array you could process as you wish.

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 6:44 pm
by 6by9
ethanol100 wrote:For yuv or rgb you know how much bytes are send for each image.
(for yuv you get Y=h*b bytes then U=h*b/4 bytes and finally V=h*b/4 bytes,
for rgb you get R=h*b bytes, then G=h*b bytes and finally B=h*b bytes["-rgb switch in raspiyuv"])
RGB will be byte interleaved, ie pixel 1 R, pixel 1 G, pixel 1 B, pixel 2 R, pixel 2 G, pixel 2 B, etc. (or I may have the order wrong and it is B, G, R, B, G, R, etc).
There are also RGB565 and RGBX32 formats that can be requested via MMAL but not currently from raspistill/vid.

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 7:04 pm
by ethanol100
You are right. There are h*b "RGB" tupple.
A nice feature for the rgb888 format is that you can easily "convert" it to a PPM image by adding a header

Code: Select all

P6
2592 1944
255
to it (if h=2592 and w=1944).
i.e. with

Code: Select all

(echo -e "P6\n2592 1944\n255"; cat image.rgb ) >image.ppm
Then most programs can open the image.

Edit: the reason for the longer delay between frames with "-t 0" is a bug in raspistill/raspiyuv.
If we use -tl and then -t the code

Code: Select all

      case CommandTimeout: // Time to run viewfinder for before taking picture, in seconds
      {
         if (sscanf(argv[i + 1], "%u", &state->timeout) == 1)
         {
            // Ensure that if previously selected CommandKeypress we don't overwrite it
            if (state->timeout == 0 && state->frameNextMethod != FRAME_NEXT_KEYPRESS)
               state->frameNextMethod = FRAME_NEXT_FOREVER;

            i++;
         }
         else
            valid = 0;
         break;
      }
will switch the frameNextMethod to FRAME_NEXT_FOREVER which results in a fixed 10s timeout.
I thought this has been fixed long time ago...(ok, only in raspivid...)
Perhaps this would be better?

Code: Select all

if (state->timeout == 0 && state->frameNextMethod == FRAME_NEXT_SINGLE)

Re: long exposure not working on B+

Posted: Thu Sep 11, 2014 7:55 pm
by fehlfarbe
Thank you very much!

Now, everything is working :)
I'm calling

Code: Select all

raspiyuv -rgb -tl 6000 -fp --nopreview -bm -ISO 800 -awb off -awbg 1,1 -t 999999999 -ss 6000000 -o - | ./process.py
And my process.py looks like this:

Code: Select all

#!/usr/bin/env python
import time, sys
import numpy as np
from PIL import Image

width = 2592
height = 1944

if __name__ == '__main__':

	while True:
		t0 = time.time()
		buf = sys.stdin.read(width*height*3)
		t1 = time.time()
		print "%f" % (t1-t0)
		img = Image.frombuffer('RGB', (width, height), buf, 'raw', 'RGB', 0, 1)
		#img.save("%d.jpg" % time.time())
EDIT: changed "raspistill" command to "raspiyuv"

Re: long exposure not working on B+

Posted: Sun Sep 21, 2014 3:00 pm
by Mark_PH
Hello,
I just tried the above command and get the error message:

Code: Select all

raspistill -rgb -tl 6000 -fp --nopreview -bm -ISO 800 -awb off -awbg 1,1 -t 999999999 -ss 6000000 -o - | ./process.py
Invalid command line option (-rgb)
0.000339
Traceback (most recent call last):
  File "./process.py", line 16, in <module>
    img = Image.frombuffer('RGB', (width, height), buf, 'raw', 'RGB', 0, 1)
  File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1858, in frombuffer
    return fromstring(mode, size, data, decoder_name, args)
  File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1797, in fromstring
    im.fromstring(data, decoder_name, args)
  File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 594, in fromstring
    raise ValueError("not enough image data")
ValueError: not enough image data
Seems the rgb option is not working? Are you using a special version of raspistill? I tried to use raspistillyuv instead of raspistill, but raspistillyuv is not installed. I did a rpi-update today.

Best regards,
Mark

Re: long exposure not working on B+

Posted: Sun Sep 21, 2014 3:19 pm
by ethanol100
It's raspiyuv.

Re: long exposure not working on B+

Posted: Sun Sep 21, 2014 3:31 pm
by jbeale
The program is called raspiyuv, not raspistillyuv. This command works for me:

Code: Select all

raspiyuv -rgb -tl 6000 -fp --nopreview -bm -ISO 800 -awb off -awbg 1,1 -t 999999999 -ss 6000000 -o - | ./process.py

Re: long exposure not working on B+

Posted: Sun Sep 21, 2014 4:11 pm
by Mark_PH
Thank you, it is working now.

Re: long exposure not working on B+

Posted: Sun Sep 21, 2014 4:52 pm
by fehlfarbe
Sorry, this was my fault. Now I changed my last post to "raspiyuv".

Re: long exposure not working on B+

Posted: Wed Sep 24, 2014 8:09 pm
by fehlfarbe
Another question:
I'm using raspiyuv -rgb and I'm reading the stdout in python with

Code: Select all

width = 2592
height = 1944
buf = sys.stdin.read(width*height*3)
width*height*3 = 15116544 bytes
But there is more data in stdin buffer. When I'm reading the whole buffer I get 15178752 bytes for one image. The difference (62208 bytes) is added at the beginning to the next image I'm reading from buffer. So every image in a sequence is shifted by some rows.
How comes the 15178752 bytes for an RGB image? Is it maybe active pixels + inaktive pixels?

Re: long exposure not working on B+

Posted: Wed Sep 24, 2014 8:16 pm
by ethanol100
The image is 1952 instead of 1944 in height, because of the need to be multiple of 16. The Additional rows are junk just added to fullfill the x16 format.

Re: long exposure not working on B+

Posted: Thu Sep 25, 2014 7:15 pm
by fehlfarbe
Thanks it's working now with a height of 1952 :)

Re: long exposure not working on B+

Posted: Wed Nov 05, 2014 11:46 am
by caracol_uk
Would it be possible to revisit whether exposures longer than 6s are possible? I realise that this may no longer be feasible if those able to make progress on this no longer have access to the GPU source code. Just thought I'd check what the current situation is and whether there is anyone with GPU source code access that wouldn't mind taking a look. Any increase at all would be good, though the closer we can get to the 20s limit the better :-).

Simon

Re: long exposure not working on B+

Posted: Wed Nov 05, 2014 1:08 pm
by 6by9
caracol_uk wrote:Would it be possible to revisit whether exposures longer than 6s are possible? I realise that this may no longer be feasible if those able to make progress on this no longer have access to the GPU source code. Just thought I'd check what the current situation is and whether there is anyone with GPU source code access that wouldn't mind taking a look. Any increase at all would be good, though the closer we can get to the 20s limit the better :-).
Very unlikely to happen I'm afraid. I believe almost all of those who were involved in the imaging side of the Pi now have new jobs elsewhere.
Whilst the Pi Foundation have made some arrangements with a couple of us, it is very much a "spare time" thing now rather than being able to get away with doing a bit on company time.

The issue with 20s exposures was that it just didn't seem to produce any data that the receiver could pick up. It'd probably be a case of getting a scope out and doing some low level tinkering to see what was happening actually on the line, and it may be that the receiver simply can not go that slowly. Seeing as I got the register settings via email from Omnivision and I no longer have access to my Broadcom email, we'd probably need to go back to OV to get them to send over the register settings again too! Some things are never simple.

Re: long exposure not working on B+

Posted: Mon Nov 10, 2014 10:54 am
by _russel
Hi Guys,

Have you had a chance to look at slightly longer exposures since the last post?

Thx

Russel

Re: long exposure not working on B+

Posted: Mon Nov 10, 2014 8:51 pm
by 6by9
_russel wrote:Hi Guys,

Have you had a chance to look at slightly longer exposures since the last post?l
The last post was 5 days ago and had the comment "very unlikely to happen I'm afraid", and that was the first update since September.
Do you really expect the situation to have changed in those 5 days when it hadn't done in the previous 2 months?

It is still very unlikely to happen.

Re: long exposure not working on B+

Posted: Mon Nov 10, 2014 10:12 pm
by jbeale
You can get almost the same effect already as a true 20 sec. exposure, by taking four 5-second exposures and adding them together.

Re: long exposure not working on B+

Posted: Mon Nov 10, 2014 11:42 pm
by riklaunim
jbeale wrote:You can get almost the same effect already as a true 20 sec. exposure, by taking four 5-second exposures and adding them together.
Not always. 4 frames will only increase SNR by 2x (sqrt). Also if the signal at 5 sec exposure is below noise level it won't work.

Re: long exposure not working on B+

Posted: Sat Dec 06, 2014 4:03 pm
by Bikeman
riklaunim wrote:
jbeale wrote:You can get almost the same effect already as a true 20 sec. exposure, by taking four 5-second exposures and adding them together.
Not always. 4 frames will only increase SNR by 2x (sqrt). Also if the signal at 5 sec exposure is below noise level it won't work.
Just for the records: in terms of SNR, if you ignore read-out noise and only look at the quantum noise , it is the same: increasing teh exposure duration by a factor of 4 will increase SNR by a factor of 2, in BOTH cases (one longer exposure vs adding 4 exposures). It's just the total photon-gathering-time that counts.

Cheers
HB

Re: long exposure not working on B+

Posted: Sat Dec 06, 2014 5:27 pm
by Burngate
riklaunim wrote:... Also if the signal at 5 sec exposure is below noise level it won't work.
Surely, as long as the noise has the right shape and is enough to take the signal above one ADC level, it will work?

Re: long exposure not working on B+

Posted: Sun Jan 24, 2016 12:48 am
by bprats99
Since I'm a noobie I'm a bit confused, I thought the maximum exposure was 6 000 000 ? When I set raspistill -ss 10000000 the annotation on the frame and preview window appears to max about 6100000 and never more. I update/upgrade often and I believe the PICam is version 1.3.8. I get sufficient exposure on the moon at F11 -ss 10000 and stars that appear small in the frame -ss 6000000 at F2 using a Pentax Super Takamar 50mm lens with 2X adapter for a total of 100mm in a scratch Pentax body.

If I have your attention, I have tried -awb off and -drc high hoping for some photo magic but I don't see a difference with the moon or stars, only that it screws up the on frame text annotation. Can anyone tell me if you see an improvement using these parameters ?? I am using PNG file output. Thanks, Bill

Re: long exposure not working on B+

Posted: Sun Jan 24, 2016 3:30 pm
by 6by9
bprats99 wrote:Since I'm a noobie I'm a bit confused, I thought the maximum exposure was 6 000 000 ? When I set raspistill -ss 10000000 the annotation on the frame and preview window appears to max about 6100000 and never more. I update/upgrade often and I believe the PICam is version 1.3.8. I get sufficient exposure on the moon at F11 -ss 10000 and stars that appear small in the frame -ss 6000000 at F2 using a Pentax Super Takamar 50mm lens with 2X adapter for a total of 100mm in a scratch Pentax body.
Approx 6 secs is the max. Don't expect more.
If I have your attention, I have tried -awb off and -drc high hoping for some photo magic but I don't see a difference with the moon or stars, only that it screws up the on frame text annotation. Can anyone tell me if you see an improvement using these parameters ?? I am using PNG file output. Thanks, Bill
PNG isn't the best choice of format - all the processing is done as yuv, so converting to rgb to save it is just another step. It's also not hardware accelerated, unlike jpeg.
Drc will override awb off - sorry, just one of those conflicting requirements things.