redovan
Posts: 4
Joined: Tue Oct 07, 2014 2:24 am

Image stabilization

Mon Oct 13, 2014 11:20 pm

Can anyone confirm that the pi camera video stabilization (--vstab option in raspivid) actually works?

Based on what I've read it should be cropping the video by 5-10% and using the extra region to filter out frame jitter, which I would expect to be pretty noticeable. (see http://www.raspberrypi.org/forums/viewt ... 7#p561467/)

However I've taken several videos with controlled camera shake and stabilization on/off, and side by side I cannot really see a difference. If there is one it's a lot more subtle than I'd expect...

Thanks,
-redovan
Last edited by redovan on Tue Oct 14, 2014 2:07 pm, edited 1 time in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Tue Oct 14, 2014 8:38 am

I cannot remember whether its turned on or not. It is also something that needs tuning to the particular camera, and I am pretty sure we never spent any time on that.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

redovan
Posts: 4
Joined: Tue Oct 07, 2014 2:24 am

Re: Image stabilization

Tue Oct 14, 2014 2:58 pm

jamesh wrote:I cannot remember whether its turned on or not.
Not turned on in the firmware? If I remember from looking through the application code it was being set from an MMAL / OpenMAX standpoint.

Can you point me towards any resources that might help me track this down? If there's a way to turn it on short of modifying the gpu firmware blob I'd like to try it, even without any tuning for the camera.

Thanks for your quick reply.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Tue Oct 14, 2014 3:32 pm

I *think* it needs to be turned on in the blob, but its quite possible its already turned on. If the mmal parameter does not appear to do anything, I'm not sure what to suggest. The effect can be small - you really need to have two devices on a shaker rig and compare the two outputs very closely. Although the smaller FOV should be a giveaway.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Tue Oct 14, 2014 7:17 pm

I mentioned this one to one of other the main camera developers who used to be at Broadcom. It rang bells that a bug had been introduced a fair while back which meant that stabilisation never did any compensation. I can't remember the details, but I have checked and the main stabilisation module is built into the firmware.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

redovan
Posts: 4
Joined: Tue Oct 07, 2014 2:24 am

Re: Image stabilization

Thu Oct 23, 2014 12:48 am

6by9 wrote:I mentioned this one to one of other the main camera developers who used to be at Broadcom. It rang bells that a bug had been introduced a fair while back which meant that stabilisation never did any compensation.
Any word on this? I hooked up a separate pi+camera board with a clean raspbian image, confirmed that the -vs / --vstab option has no noticeable effect on captured video, including no reduction in field of view.

Raspivid does appear to be setting MMAL_PARAMETER_VIDEO_STABILISATION to true, and I tried enabling the parameter via the uv4l pi camera driver as well just to use a different path. Nothing I've tried has resulted in any noticeable stabilization occurring.

I really appreciate the help.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Fri Oct 24, 2014 8:16 am

redovan wrote:Any word on this? I hooked up a separate pi+camera board with a clean raspbian image, confirmed that the -vs / --vstab option has no noticeable effect on captured video, including no reduction in field of view.
Seeing as yesterday I happened to be sitting next to one of my ex-colleagues who is still currently working for Broadcom, I got a sneaky look at the later development tree looking for the fix. Annoyingly it appears to already be present on the Pi branch so it's going to take a bit more digging, and I'm afraid with the new job time is rather in short supply. It is on the list for when I do get some free time though.
I have done a quick check with the V4L2 driver and video stabilisation, and I agree that it doesn't appear to do anything.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Piper984
Posts: 13
Joined: Tue Oct 07, 2014 7:37 pm

Re: Image stabilization

Fri Jan 16, 2015 10:35 pm

Does anyone have any new information on this topic? I also see no reduction in field of view in my stream coming from raspivid -vs. From the source in githut, I see the parameter is taken and passed to MMAL...

Code: Select all

int raspicamcontrol_set_video_stabilisation(MMAL_COMPONENT_T *camera, int vstabilisation);
But in mmalomx_util_params_camera.c I think the passed parameter isn't set?

Code: Select all

{ MMAL_PARAMETER_ALGORITHM_CONTROL_ALGORITHMS_VIDEO_STABILISATION,      OMX_CameraDisableAlgorithmVideoStabilisation},
... so hopefully someone knows if there is an issue w/ the current release binaries, current source, or if there a bug with Openmax? Information/guidance on this would be much appreciated. I'm eager to try this feature if available.

Thanks,

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Sat Jan 17, 2015 7:58 am

Piper984 wrote:Does anyone have any new information on this topic?
Sorry not had a chance to look at it. It is on the list, but I get very little time to look into Pi issues.
mmalomx_util_params_camera.c isn't actually used on the GPU side, so that's a red herring.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Piper984
Posts: 13
Joined: Tue Oct 07, 2014 7:37 pm

Re: Image stabilization

Tue Jan 20, 2015 4:02 am

Thanks for letting me know re: GPU and source code. Anyone have any advice on how we could follow up more on this topic? Would be great to know if 1) it's a bug that can be fixed, 2) it's a feature in development, or 3) it's a deprecated option.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Tue Jan 20, 2015 2:12 pm

Piper984 wrote:Thanks for letting me know re: GPU and source code. Anyone have any advice on how we could follow up more on this topic? Would be great to know if 1) it's a bug that can be fixed, 2) it's a feature in development, or 3) it's a deprecated option.
Please note my sig!
Broadcom made the whole team that developed the Pi chip redundant back in August, along with those developing mobile phone chips at other sites around the world. There is a team at Pi Towers providing support, but none of them were involved in imaging. Imaging support generally falls to me or jamesh (limited due to a conflict of interest with his new job), and we're both doing it voluntarily.

Video stabilsation did work on previous shipping products (Nokia N8 and Pureview 808 spring to mind). It's therefore a bug, but whether it can be fixed depends on finding some time to look into it.
I'm afraid people nagging gets my back up a little and makes me less likely to want to look into their specific issue. That may be unfair, but I only get so much time to look into stuff, and being extra grumpy doesn't really motivate me.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Piper984
Posts: 13
Joined: Tue Oct 07, 2014 7:37 pm

Re: Image stabilization

Tue Jan 20, 2015 10:41 pm

Hello 6x9, thanks for the information. Sorry if you thought I was nagging. I appreciate the time you volunteer to continue to make the Pi so awesome. Please advise if you think there is a way I could help with this specific issue? It's a feature I have a need for, so I'd be willing to do some work to bring it up to a working feature if I can.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Wed Jan 21, 2015 6:48 am

Piper984 wrote:Hello 6x9, thanks for the information. Sorry if you thought I was nagging. I appreciate the time you volunteer to continue to make the Pi so awesome. Please advise if you think there is a way I could help with this specific issue? It's a feature I have a need for, so I'd be willing to do some work to bring it up to a working feature if I can.
It's in the GPU firmware, hence the very limited number f people who can work on it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Wed Feb 25, 2015 9:41 pm

I've finally found a few minutes to look at this. A block of config was missing on the sensor tuning.
I've done a blind copy/paste from another tuning and will ask Dom to release it. I don't recall this being used in anger for a fair while, so it may be hideously broken.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Thu Feb 26, 2015 9:37 am

6by9 wrote:I've finally found a few minutes to look at this. A block of config was missing on the sensor tuning.
I've done a blind copy/paste from another tuning and will ask Dom to release it. I don't recall this being used in anger for a fair while, so it may be hideously broken.
The tuning from the Galileo should be OK - we spent a lot of time on that. Ask Cressida for more details!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Thu Feb 26, 2015 10:01 am

jamesh wrote:
6by9 wrote:I've finally found a few minutes to look at this. A block of config was missing on the sensor tuning.
I've done a blind copy/paste from another tuning and will ask Dom to release it. I don't recall this being used in anger for a fair while, so it may be hideously broken.
The tuning from the Galileo should be OK - we spent a lot of time on that. Ask Cressida for more details!
But that will only be on that one specific branch that we no longer have access to (ie the Perforce server still even turned on?), and may also have had impacts on the tuner code or other algo lumps. (Given a bit of luck I could actually still extract the tuning off a real device DCC file if it were just the tuning!)
I've just copied it from the standard DK camera as that is on the Pi branch.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Thu Feb 26, 2015 10:03 am

6by9 wrote:
jamesh wrote:
6by9 wrote:I've finally found a few minutes to look at this. A block of config was missing on the sensor tuning.
I've done a blind copy/paste from another tuning and will ask Dom to release it. I don't recall this being used in anger for a fair while, so it may be hideously broken.
The tuning from the Galileo should be OK - we spent a lot of time on that. Ask Cressida for more details!
But that will only be on that one specific branch that we no longer have access to (ie the Perforce server still even turned on?), and may also have had impacts on the tuner code or other algo lumps. (Given a bit of luck I could actually still extract the tuning off a real device DCC file if it were just the tuning!)
I've just copied it from the standard DK camera as that is on the Pi branch.
OK, any problems talk to Cressida. She's the expert on this.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Thu Feb 26, 2015 5:30 pm

Dom has done his magic, so rpi-update should get all the new goodness.
I have seen that we do now get the slight zoom in when stabilisation is on, but not checked out the results when the camera is shaken.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
DougieLawson
Posts: 41456
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Image stabilization

Thu Feb 26, 2015 6:26 pm

6by9 wrote:Dom has done his magic, so rpi-update should get all the new goodness.
I do wish there was a better way to get this stuff updated without running rpi-update (and without building from a clone of the userland git). There's a growing band of folks who really want rpi-update to be removed to a hidden special corner of the internet.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Thu Feb 26, 2015 7:03 pm

This sort of change requires a change to the GPU firmware only (start.elf), so I wonder if there needs to be a parameters to rpi-update to only update that (not that that would be enough in some circumstances).
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Thu Feb 26, 2015 7:38 pm

DougieLawson wrote:I do wish there was a better way to get this stuff updated without running rpi-update (and without building from a clone of the userland git). There's a growing band of folks who really want rpi-update to be removed to a hidden special corner of the internet.
Would sudo apt-get dist-upgrade really be much better?! And either you have to build the kernel yourself, or you wait on a 6-8 week release cycle. Perhaps it should just be called "next" alongside those 6-8 week release branches - if something is broken in between those releases then tough.

Not my call anyway, I just try to make people happy by getting features integrated.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
DougieLawson
Posts: 41456
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Image stabilization

Thu Feb 26, 2015 7:42 pm

I, really, don't know the answer. The fundamental problem is the overlapping updates in the GPU blob (that the regular user can't debug). I got bitten by a rpi-update that left my 2B unbootable this week and it looks like it's start.elf that breaks it.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29037
Joined: Sat Jul 30, 2011 7:41 pm

Re: Image stabilization

Fri Feb 27, 2015 9:35 am

Best option is to ensure that rpi-update, by default, can never break the system. ie It's tested fully.

Perhaps another option to rpi-update would then get bleeding edge firmware.

sudo rpi-update bleedingedge

or perhaps

sudo rpi-update mightbreakstuff
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Application's Team.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11487
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Image stabilization

Fri Feb 27, 2015 10:10 am

jamesh wrote:Best option is to ensure that rpi-update, by default, can never break the system. ie It's tested fully.
Lack of manpower. How many were there in the infrastructure team at Broadcom for all the per-checkin tests? I don't see that being viable for Pi, and there were the odd breakages even with all the testing in place.
jamesh wrote:Perhaps another option to rpi-update would then get bleeding edge firmware.

sudo rpi-update bleedingedge

or perhaps

sudo rpi-update mightbreakstuff
As I said, adopt the kernel model - a reasonably well tested release every 6-8 weeks, and if you pick up the intermediate dev tree then it's on your own head. I'm not sure they're that far off that anyway with Raspbian official releases.
I quite like the idea of having a "for-dom" branch :D
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
DougieLawson
Posts: 41456
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Image stabilization

Fri Feb 27, 2015 12:08 pm

Perhaps the answer is to only ever publish the "bleeding egde" untested stuff on BRANCH=next and only have camera code and fully tested kernel/bootcode/"firmware" stuff on the master branch. That way I have to do something special to attempt to render my RPi as a non-bootable zombie.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Return to “Camera board”