jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Full FOV preview, 60 and 90fps modes released

Fri Mar 14, 2014 8:20 pm

For those who didn't see the post in the V4L thread, we now how a full FOV preview mode for still viewfinder, and up to 90fps modes at VGA.

See here for more details...

http://www.raspberrypi.org/forum/viewto ... ad#p520078

Raspistill/YUV and vid have been updated and can be downloaded and built from the Raspberry Pi github page here https://github.com/raspberrypi/userland

As an example,

raspivid -w 640 -h 480 -fps 90 -t 10000 -o ninetyfps.h264

You will need to set the resolution low in order for it to get the fps right as resolution is preferred over fps when selecting the mode.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

Chris_Reynolds
Posts: 72
Joined: Mon May 14, 2012 7:25 am

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 7:10 am

This is great news. A quick download and build last night and I tried out the 90 fps feature and it worked very well (even in the gloomy light of my office). I have plans to use this as a tool for analysing fencing (the one with swords) as being able to show a 3x slower video of action immediately to a fencer will be a great coaching aid. Allied with the 7 (or 10) inch touch screen this will be a brilliant low cost portable gadget.

Thank you to everyone who worked on getting this sorted out.

SnowLeopard
Posts: 106
Joined: Sun Aug 18, 2013 6:10 am

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 4:36 pm

Woohoo!!! Thank-you! Thank-you! Thank-you!
A $50 device that does 90fps!

User avatar
mikronauts
Posts: 2750
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 4:42 pm

Excellent news - and perfect timing!

I am about to add one of my Pi cam's to one of my bots, and I'd like to stream mpeg4 back from the bot at a lower resolution.

I am hoping for 800x480 @ 30fps over wifi.

Oops. Forgot the Pi Cam was 4:3... I'll go with 640x480 :)
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

Chris_Reynolds
Posts: 72
Joined: Mon May 14, 2012 7:25 am

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 5:03 pm

I've had more of a play today and I'm still happy but the picture quality on both 60 and 90 fps is definitely worse than 30 fps at 640x480. It looks badly anti-aliased if that makes sense. It is more than usable so don't take this as a complaint but more of a query as to whether this is expected and what other people are seeing. I'll try and post some stills later.

Thanks

Chris

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 5:26 pm

Chris_Reynolds wrote:I've had more of a play today and I'm still happy but the picture quality on both 60 and 90 fps is definitely worse than 30 fps at 640x480. It looks badly anti-aliased if that makes sense. It is more than usable so don't take this as a complaint but more of a query as to whether this is expected and what other people are seeing. I'll try and post some stills later.
I'll partly refer you to my post on the V4L2 thread http://www.raspberrypi.org/forum/viewto ... 32#p520132
For VGA output at anything up to 42fps the senor will be reading out 1296x972 and then downscaling in the ISP. Above 42fps the sensor is only reading out VGA and the ISP doesn't need to resize. When you start with fewer pixels, the output is inherently worse.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 6:22 pm

Chris_Reynolds wrote:I've had more of a play today and I'm still happy but the picture quality on both 60 and 90 fps is definitely worse than 30 fps at 640x480. It looks badly anti-aliased if that makes sense. It is more than usable so don't take this as a complaint but more of a query as to whether this is expected and what other people are seeing. I'll try and post some stills later.

Thanks

Chris
I suggest setting the preview to be a VGA sized window rather than letting it upscale to full screen - then you can see it pixel->pixel displayed. Should look OK. VGA scaled up to 1080 does look aliased.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

Chris_Reynolds
Posts: 72
Joined: Mon May 14, 2012 7:25 am

Re: Full FOV preview, 60 and 90fps modes released

Sat Mar 15, 2014 8:57 pm

Thanks for the replies. I was being a bit thick with the upscaling James. I was just playing back the h264 file with omxplayer and that scaled it to full screen (1280x1024ish) which made the aliasing seem much worse. Playing it back in a 1:1 window looks much better. You can still see the aliasing compared to a straight 640x480x30fps but I can see from 6by9's post why that is (and you can clearly see the step as you go from 42 to 43 fps). I appreciate you both taking time to reply.

Out of interest I was able to prove to my son that he does take his eye off the ball when catching as you can clearly see his eye tracking a ball until it is about 3 inches from his hand when he starts looking up and away and you would never have seen that without extra fps. Pi+picamera+7" screen has a real potential in sports coaching - excellent stuff.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 5:08 pm

Chris_Reynolds wrote:Thanks for the replies. I was being a bit thick with the upscaling James. I was just playing back the h264 file with omxplayer and that scaled it to full screen (1280x1024ish) which made the aliasing seem much worse. Playing it back in a 1:1 window looks much better. You can still see the aliasing compared to a straight 640x480x30fps but I can see from 6by9's post why that is (and you can clearly see the step as you go from 42 to 43 fps). I appreciate you both taking time to reply.

Out of interest I was able to prove to my son that he does take his eye off the ball when catching as you can clearly see his eye tracking a ball until it is about 3 inches from his hand when he starts looking up and away and you would never have seen that without extra fps. Pi+picamera+7" screen has a real potential in sports coaching - excellent stuff.
Cool - glad people are using it already. 6x9 and my boss deserves all the vast majority of credit for this one. I was stuck, and they sorted out the register sets (as it happens some of my numbers were not too bad - just bad enough). All I really did was pester until it got done. I've really been too busy with actual work to spend any real time on it.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

ejarosek
Posts: 29
Joined: Mon Aug 27, 2012 6:39 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 6:01 pm

So, I am also seeing what I think Chris is talking about although my test is a bit different.

When I run this command:

Code: Select all

./raspivid -t 30000 -fps 30 -w 1280 -h 720 -b 2000000 -o 720p-30fps-light.h264
Things look good...same as always.
https://www.dropbox.com/s/mwpgyq2b7jrvd ... light.h264

When I run this:

Code: Select all

./raspivid -t 30000 -fps 60 -w 1280 -h 720 -b 4000000 -o 720p-60fps-light.h264
The image quality looks to be about 1/4 resolution (even though it is 1280x720 at 60fps).
https://www.dropbox.com/s/ixwv9hzr32f7c ... light.h264

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 6:14 pm

ejarosek wrote:So, I am also seeing what I think Chris is talking about although my test is a bit different.

When I run this command:

Code: Select all

./raspivid -t 30000 -fps 30 -w 1280 -h 720 -b 2000000 -o 720p-30fps-light.h264
Things look good...same as always.
https://www.dropbox.com/s/mwpgyq2b7jrvd ... light.h264

When I run this:

Code: Select all

./raspivid -t 30000 -fps 60 -w 1280 -h 720 -b 4000000 -o 720p-60fps-light.h264
The image quality looks to be about 1/4 resolution (even though it is 1280x720 at 60fps).
https://www.dropbox.com/s/ixwv9hzr32f7c ... light.h264
Please reread the post about this feature.
The new mode that the sensor is producing is 640x480 at 60fps (or 90fps). The system does then allow you to upscale it to 720P, but you'll have started with 640x360 (after cropping 4:3 to 16:9) worth of pixels and it can't magic extra in quality, just interpolate.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ejarosek
Posts: 29
Joined: Mon Aug 27, 2012 6:39 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 6:21 pm

Ah, found it. Sorry, when I read the post here about VGA at 90fps I made the assumption that the 720p at 60fps was also supported. I see how in the V4L thread where you say it is limited to 49fps.

Thank you much.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 6:23 pm

Just for completeness (6x9 has posted this in the other thread).

Available modes now are:

2592x1944, 1-15fps, video or stills mode, Full sensor full FOV, default stills capture
1920x1080, 1-30fps, video mode, 1080p30 cropped
1296x972, 1-42fps, video mode, 4by3 aspect binned full FOV. Used for stills preview in raspistill.
1296x730, 1-49fps, video mode, 16by9 aspect , binned, full FOV (width), used for 720p
640x480, 42.1-60fps, video mode, upto VGAp60 binned
640x480, 60.1-90fps, video mode, upto VGAp90 binned
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

ejarosek
Posts: 29
Joined: Mon Aug 27, 2012 6:39 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 6:52 pm

Thanks for the summary Jamesh.

One note, it appears the bitrate calculation is based on 30fps. So, if I set the bitrate to 4Mbps for 1280p at 49fps it actually produces a bitrate of 6.4Mbps (4Mbps * 49/30). Not sure if that is the expected behavior, just something I observed.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 7:48 pm

ejarosek wrote:Thanks for the summary Jamesh.

One note, it appears the bitrate calculation is based on 30fps. So, if I set the bitrate to 4Mbps for 1280p at 49fps it actually produces a bitrate of 6.4Mbps (4Mbps * 49/30). Not sure if that is the expected behavior, just something I observed.
Slightly unexpected. The codec has two modes of operation - one where it expects the framerate that it is told at the start to be accurate, and the other where it looks at the timestamps.
With the RaspiVid setup (and V4L2) it should be told the correct framerate at the start, but it sounds like it is always being told 30. I'd hope that was a glitch in RaspiVid, but the first mode also has issues if you're going to change framerate whilst encoding. Perhaps the second mode might be the better one to choose, although the downside is that the initial bitrate control isn't as good whilst it i learning the framerate. There's never one perfect solution!

I'll have a discussion with JamesH to see if there is anything obvious.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

towolf
Posts: 421
Joined: Fri Jan 18, 2013 2:11 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 8:19 pm

Playing with raspivid now, because I cannot get bcm2835_v4l2 to work anymore. The new modes are really cool and look good mostly. Really happy all this is available now.

Is there information about where the VGA mode gets its pixels from? Is it reading every 4th line and column from the sensor, or what? I assume there’s no binning involved because of staircase effect (aliasing). And there seems to be a slightly smaller FoV, maybe that is because mod([2592, 1944], [640, 480]) == [32, 24]?

And is there a qualitative difference between [email protected] and [email protected], other than frame rate?

And then [email protected] is pretty cool and good looking. I’m only wondering about the odd number, is it based on gross/net data rates on the bus? Say, at 50fps it would fit PAL television. Is this a fixed value, or empirical?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Sun Mar 16, 2014 8:28 pm

towolf wrote:Playing with raspivid now, because I cannot get bcm2835_v4l2 to work anymore. The new modes are really cool and look good mostly. Really happy all this is available now.

Is there information about where the VGA mode gets its pixels from? Is it reading every 4th line and column from the sensor, or what? I assume there’s no binning involved because of staircase effect (aliasing). And there seems to be a slightly smaller FoV, maybe that is because mod([2592, 1944], [640, 480]) == [32, 24]?

And is there a qualitative difference between [email protected] and [email protected], other than frame rate?

And then [email protected] is pretty cool and good looking. I’m only wondering about the odd number, is it based on gross/net data rates on the bus? Say, at 50fps it would fit PAL television. Is this a fixed value, or empirical?
Empirical. Might be able to tweak it to 50 - 6by9 has already tried, but it may be the clock calculation are slightly out (We have a suspicion the numbers are based on a 24Mhz clock, whereas the board is 25Mhz)- we will need new numbers from Omnivision to sort that out I believe.

There are reg set differences between 60 and 90, but since these sets case from Omnivision, it's difficult to tell what. It quite possible we coudl get rid fo the 60fps mode, and just use 90, but maybe there is a quality difference that means its worth keeping.

As for how the mode is worked out for VGA- the registers have an options for 2x2 binning, which I think is turned on. How they then go from there to VGA I'm not sure. I'll have to ask Omnivision. It's not simple apparently.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

fazzy
Posts: 3
Joined: Mon Mar 03, 2014 5:30 am

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 3:43 pm

Hi,

Can this be applied using the simple rpi-update or does it require downloading and building?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 3:50 pm

fazzy wrote:Hi,

Can this be applied using the simple rpi-update or does it require downloading and building?
I believe sudo rpi-update should get everything you need.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

User avatar
DougieLawson
Posts: 37703
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 3:58 pm

fazzy wrote:Hi,

Can this be applied using the simple rpi-update or does it require downloading and building?
It works well with rpi-update & reboot.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 4:59 pm

6by9 wrote:
ejarosek wrote:Thanks for the summary Jamesh.

One note, it appears the bitrate calculation is based on 30fps. So, if I set the bitrate to 4Mbps for 1280p at 49fps it actually produces a bitrate of 6.4Mbps (4Mbps * 49/30). Not sure if that is the expected behavior, just something I observed.
Slightly unexpected. The codec has two modes of operation - one where it expects the framerate that it is told at the start to be accurate, and the other where it looks at the timestamps.
I'll have a discussion with JamesH to see if there is anything obvious.
We've just had a look - that is some of the weirdest logic I have come across for a while!
The encoder is set up with the framerate from the encoder component OUTPUT port. The settings are copied over from the input port, but only if the output framerate was set to 0. Seeing as the default set by the component is 30, I don't see that is ever likely to happen!

James will look into a fix for Raspivid (we think just setting the framerate to 0 when setting up the output port should suffice - he's tied up with other things at the moment, so if anyone else wants to give it a try, then feel free).
It looks like I lucked out when I wrote the V4L2 driver and set the framerate on the encoder output port as a matter of course. There are probably a few nasty cases where you could set the format and then change the framerate which I won't catch, but you can't have everything ;)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
redhawk
Posts: 3465
Joined: Sun Mar 04, 2012 2:13 pm
Location: ::1

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 6:38 pm

Is 25fps or 50fps possible with the Pi Camera (i.e. not 60fps internally with frames dropped here there and everywhere)??

As a PAL TV owner the current frame rate modes of 30, 60, 90 are not suitable for PAL without jerky frames. :(

Richard S.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 8115
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Full FOV preview, 60 and 90fps modes released

Wed Mar 19, 2014 8:36 pm

redhawk wrote:Is 25fps or 50fps possible with the Pi Camera (i.e. not 60fps internally with frames dropped here there and everywhere)??

As a PAL TV owner the current frame rate modes of 30, 60, 90 are not suitable for PAL without jerky frames. :(

Richard S.
If you ask for a framerate, then provided you haven't exceeded any of the hardware limitations, that is exactly what you get. Not any arbitrary framerate with frames dropped. We only ever drop frames if we run out of buffering for some reason.
Within the sensor each frame is extended in length by adding a number of dummy lines to the end, and as the pixel rate is constant that changes the framerate.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

punktea
Posts: 21
Joined: Thu Jul 18, 2013 8:55 am

Re: Full FOV preview, 60 and 90fps modes released

Thu Mar 20, 2014 5:00 am

Available modes now are:

2592x1944, 1-15fps, video or stills mode, Full sensor full FOV, default stills capture
1920x1080, 1-30fps, video mode, 1080p30 cropped
1296x972, 1-42fps, video mode, 4by3 aspect binned full FOV. Used for stills preview in raspistill.
From above specifications, I would like to record at 2592x1944, 10 fps but I got errors

# raspivid -o clip.h264 -w 1920 -h 1080 -fps 10 -b 1000k -t 10000
-> OK

# raspivid -o clip1.h264 -w 2592 -h 1944 -fps 10 -b 1000k -t 10000
mmal: mmal_vc_port_enable: failed to enable port vc.ril.video_encode:in:0(OPQV): EINVAL
mmal: mmal_port_enable: failed to enable connected port (vc.ril.video_encode:in:0(OPQV))0x1507d90 (EINVAL)
mmal: mmal_connection_enable: output port couldn't be enabled
mmal: main: Failed to connect camera video port to encoder input
mmal: Argument is invalid

Once error occured, I can no longer record in any resolution and I have to reboot.

Based on the description "2592x1944, 1-15fps, video or stills mode, Full sensor full FOV, default stills capture",
I am guessing that 2592x1944 defaults to stills capture mode and I have to switch it to video mode
before running raspivid but I could not find anything describing switching mode in the documentation

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 25443
Joined: Sat Jul 30, 2011 7:41 pm

Re: Full FOV preview, 60 and 90fps modes released

Thu Mar 20, 2014 9:09 am

Although the sensor can do 15fps at full res, the rest of the system will struggle badly to keep up (SD card as much as anything!) - and in this case you have greatly exceeded the capacity of the encoder. It maxes out at 1080p30. You are also using a stills mode for video, which might also be causing problems.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

Return to “Camera board”