Framboise
Posts: 8
Joined: Thu Dec 13, 2012 10:22 pm

file size limit on raspivid ?

Tue May 28, 2013 3:54 pm

hi,
When i use:
raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

(recording 1 hour video and writing it to usbstick)
i get a error:
mmal: Failed to write buffer data - aborting

and when i see the file size it it exactly 2048 Mib

is there a size limit in the program ?
The usbstick is formatted in EXT4 but i tried FAT32 and NTFS with the same result.
greetings
pj

cccsssddd
Posts: 8
Joined: Fri Jan 06, 2012 11:40 am

Re: file size limit on raspivid ?

Tue May 28, 2013 4:25 pm

I'd be interested in this too - i've got the camera but not fired up the Pi to try anything with it yet - Will be wanting to do something on a rolling window of video i.e keep the last hour so this may well affect me

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Tue May 28, 2013 4:39 pm

Hmm. That's interesting.

I'm having a flashback to some years ago, lseek not being able to seek past 2048MB, due to the file offset pointer being 32bits. I wonder if this is similar.

I'll look in to it.

In the meantime try dropping the bit rate if you need to record for longer. Defaults to 17MBits I think. Even half of that still produces decent imagery. Or drop to 720p.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Tue May 28, 2013 4:42 pm

Ah. Here we go.

http://stackoverflow.com/questions/7307 ... write-in-c

I need to enable long file support. Somehow.

Good find.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Tue May 28, 2013 5:17 pm

OK, can see what I need to do - but the SD card I'm using is only 2GB! So can't test the fix. Doh! Will do it tomorrow.

(For those interested, -D_FILE_OFFSET_BITS=64 is required in flags supplied to gcc).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

erikcf
Posts: 19
Joined: Thu May 23, 2013 4:17 am

Re: file size limit on raspivid ?

Tue May 28, 2013 6:37 pm

If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264

Framboise
Posts: 8
Joined: Thu Dec 13, 2012 10:22 pm

Re: file size limit on raspivid ?

Tue May 28, 2013 9:37 pm

jamesh wrote:OK, can see what I need to do - but the SD card I'm using is only 2GB! So can't test the fix. Doh! Will do it tomorrow.

(For those interested, -D_FILE_OFFSET_BITS=64 is required in flags supplied to gcc).
Thanks!

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Tue May 28, 2013 10:54 pm

erikcf wrote:If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264
Don't know...but its an interesting question!

Try it and see! Bump the bitrate up to 30Mbs so you don't have to wait so long.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

gregeric
Posts: 1509
Joined: Mon Nov 28, 2011 10:08 am

Re: file size limit on raspivid ?

Wed May 29, 2013 9:08 am

jamesh wrote:OK, can see what I need to do - but the SD card I'm using is only 2GB! So can't test the fix. Doh! Will do it tomorrow.

(For those interested, -D_FILE_OFFSET_BITS=64 is required in flags supplied to gcc).
I've come across this issue before; you may also need -D_LARGEFILE_SOURCE and -D_LARGEFILE64_SOURCE, then fopen/fseeko/ftello wil be substituted with their 64 bit clones fopen64 etc. File pointers should also be of type off_t for seemless transition to 64 bit file handling.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 9:28 am

gregeric wrote:
jamesh wrote:OK, can see what I need to do - but the SD card I'm using is only 2GB! So can't test the fix. Doh! Will do it tomorrow.

(For those interested, -D_FILE_OFFSET_BITS=64 is required in flags supplied to gcc).
I've come across this issue before; you may also need -D_LARGEFILE_SOURCE and -D_LARGEFILE64_SOURCE, then fopen/fseeko/ftello wil be substituted with their 64 bit clones fopen64 etc. File pointers should also be of type off_t for seemless transition to 64 bit file handling.
Yes, I think I do have some missing flags as my test with just -D_FILE_OFFSET_BITS=64 fails as soon as I try to write data.

Cheers.

James
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 9:45 am

OK, so you need _LARGEFILE_SOURCE as well as -D_FILE_OFFSET_BITS=64

Still need to test writing > 2GB, but smaller files work OK with the new defines.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Framboise
Posts: 8
Joined: Thu Dec 13, 2012 10:22 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 9:58 am

erikcf wrote:If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264

Thanks, this works!
i am past the 2048 now and will let it run to the supposed runtime of 1 hour.
(i was too late to change bitrate to 30mb so it is still on default )
only thing is the -v does not do anything, not a big problem for me
(2,713 MiB and counting....)
greetings
pj

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 10:10 am

Framboise wrote:
erikcf wrote:If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264

Thanks, this works!
i am past the 2048 now and will let it run to the supposed runtime of 1 hour.
(i was too late to change bitrate to 30mb so it is still on default )
only thing is the -v does not do anything, not a big problem for me
(2,713 MiB and counting....)
greetings
pj
-v is turned off when outputting to stdout - it corrupts the stream if you start printing stuff in the middle of it
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
leol
Posts: 147
Joined: Fri Jan 13, 2012 4:27 pm
Location: Haute-Vienne, France

Re: file size limit on raspivid ?

Wed May 29, 2013 10:35 am

Wouldn't it be better if the output from -v went to stderr?

Leo
jamesh wrote:
Framboise wrote:
erikcf wrote:If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264

Thanks, this works!
i am past the 2048 now and will let it run to the supposed runtime of 1 hour.
(i was too late to change bitrate to 30mb so it is still on default )
only thing is the -v does not do anything, not a big problem for me
(2,713 MiB and counting....)
greetings
pj
-v is turned off when outputting to stdout - it corrupts the stream if you start printing stuff in the middle of it

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 11:23 am

leol wrote:Wouldn't it be better if the output from -v went to stderr?

Leo
Oops. You are right, it does actually do that. I could remove the -v restriction. It's a handover from when everything (including -v) went to stdout.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Framboise
Posts: 8
Joined: Thu Dec 13, 2012 10:22 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 11:40 am

jamesh wrote:
Framboise wrote:
erikcf wrote:If you send the output to stdout and redirect that to a file, does that work around the issue? For example, instead of:

raspivid -v -t 3600000 -o /media/usbdisk/vid/video.h264

Try this:

raspivid -v -t 3600000 -o - > /media/usbdisk/vid/video.h264

Thanks, this works!
i am past the 2048 now and will let it run to the supposed runtime of 1 hour.
(i was too late to change bitrate to 30mb so it is still on default )
only thing is the -v does not do anything, not a big problem for me
(2,713 MiB and counting....)
greetings
pj
-v is turned off when outputting to stdout - it corrupts the stream if you start printing stuff in the middle of it
Ok that makes sense.
Now then, a 1 hour file was recorded with a total size of a little over 6 Gig on my usb stick.
so:
raspivid -t 3600000 -o - > /media/usbdisk/vid/video.h264
works fine for bigger files
I played it and, i does play good up to about 30 min, then the contrast seems to switch to very low. This lasts for about 15 minutes and it then suddenly switches back to normal. (lighting conditions stayed the same.)

Although this does not seem to be related to the file-size issue, i thought I'd mention it.
This was the first time i recorded over 20 minutes with it.
I will try some more and see if the contrast glitch occurs again.

Thanks and greetings,
pj

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 4:27 pm

OK, testing with the flags above and I created a 5GB file so looks like a good fix. It'll be released sometime in the future!

What's annoying is the the file took up the whole SD card, but then the Pi crashed, and I thought I had protected against that happening...
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: file size limit on raspivid ?

Wed May 29, 2013 4:40 pm

Linux is very particular about having the system partition fill up to 0 bytes free. I made a separate partition for data to guard against that.

Seaton
Posts: 2
Joined: Wed May 29, 2013 1:06 pm

Re: file size limit on raspivid ?

Thu May 30, 2013 12:38 am

I tried the 'raspivid -v -t 3600000 -o - > video.h264' method, but also added -b 8000000 to get a 1 hour 8Mb/s video and I got a 1 hour video with a file-size of 2.8GB on my 32GB Class 10 SD card.

Great I thought - problem solved, as the result looked great when I viewed it briefly.

However, I have just had a more thorough check and have discovered the video to start chopping (ie become blocky ranging from short blocks to almost the whole image), but only starting at 43 minutes and 40 seconds up until the end. This just happens to work out at around the 2GB mark :o

I'm not sure if this was pure coincidence - I will try again in daylight later today.

Has anyone created a faultless video using this method?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Thu May 30, 2013 5:49 am

Bit rate is too high I think, so probably you exceeded the top speed of the system. 30->35Mbits/s is the max you really need to go to.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Thu May 30, 2013 5:49 am

towolf wrote:Linux is very particular about having the system partition fill up to 0 bytes free. I made a separate partition for data to guard against that.
Ah, that's probably it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Seaton
Posts: 2
Joined: Wed May 29, 2013 1:06 pm

Re: file size limit on raspivid ?

Thu May 30, 2013 8:45 pm

jamesh wrote:Bit rate is too high I think, so probably you exceeded the top speed of the system. 30->35Mbits/s is the max you really need to go to.
I set the bit rate to 8Mbits/s (-b 8000000), and also the frame rate to 25 fps so there shouldn't have been a problem.

Anyway, I re-ran the command again - it was exactly the same command line as I had run it from a Bash script. The file size was again 2.8GB.

I've just spent an hour watching a very boring video of a empty street (I feel sorry for the old bill who have to view hours of CCTV) - however, this time the video was faultless, so I have no idea why the previous video stream became and remained corrupted from the 2GB point. I will try a bit more experimenting with various bit rates etc and keeping my fingers crossed that it was just a one off.

Gigadan
Posts: 2
Joined: Wed Jul 17, 2013 3:46 pm

Re: file size limit on raspivid ?

Sun Jul 21, 2013 11:23 am

I've just run into the same issue. I'm a computer veteran but a Linux/Pi newb so am I right in thinking that the above are flags for a kernel rebuild? Can I do this on the Pi (don't mind if it takes overnight) or do I need a cross compiler, or have I got completely the wrong end of the stick?
Many Thanks
Dave

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24180
Joined: Sat Jul 30, 2011 7:41 pm

Re: file size limit on raspivid ?

Sun Jul 21, 2013 6:03 pm

No, I think it's just the build option for the Raspivid application. Somewhere in the cmake files I think (in userland), I don't use that particular build system so cannot help much more than that.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Gigadan
Posts: 2
Joined: Wed Jul 17, 2013 3:46 pm

Re: file size limit on raspivid ?

Thu Jul 25, 2013 7:49 pm

Just the job. Got the source code building first and then tweaked the makefile to add the compiler directives and all is well. Just generated a 3.5Gb video file with no issues.
Many thanks for the pointer.

Return to “Camera board”