gr3go
Posts: 13
Joined: Tue Oct 11, 2016 4:59 pm

raspivid motion vectors

Thu Jul 13, 2017 1:42 pm

Hi guys,

I'm playing around with raspivid with motion vectors output enabled.
sudo /opt/vc/bin/raspivid -o /dev/adam/out%04d.h264 -x /dev/adam/mv%04d.bin -w 1280 -h 760 -fps 30 -t 0 --ev 7 -hf -sg 100 -wr 100 -g 10

I've covered the camera sensor with my finger, and I've been expecting to see only zeroes in the mv files, but unfortunately the values are not zeroes.
https://www.screencast.com/t/lxWfPFLM1j

Is there a recommended threshold for considering an vector intensity as movement?


Thanks!

cpunk
Posts: 85
Joined: Thu Jun 29, 2017 12:39 pm

Re: raspivid motion vectors

Thu Jul 13, 2017 9:41 pm

Interesting... I have not tried with total darkness, although I am also building an application which consumes motion vectors.

I can test the situation you describe tomorrow. I don't know how the encoder is supposed to behave (or if we can influence how it does), but I can at least comment on how it behaves here, and what my post-filtering code thinks of the vectors. :)

gr3go
Posts: 13
Joined: Tue Oct 11, 2016 4:59 pm

Re: raspivid motion vectors

Thu Jul 13, 2017 10:13 pm

Ok, thanks. Waiting for your input :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9232
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: raspivid motion vectors

Fri Jul 14, 2017 8:01 am

Please remember that these motion vectors are for optimising the video encoding, and as such it is just looking for the area that will give the best compression efficiency. If the whole scene is near identical (ie black), then you will get weird results. This is not object tracking.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

cpunk
Posts: 85
Joined: Thu Jun 29, 2017 12:39 pm

Re: raspivid motion vectors

Sat Jul 15, 2017 1:45 pm

Sorry for double-posting, but I'll also mention this in the thread that Hove started, to ask if he's already using this (since drone-flying people would likely be especially interested in getting undesired noise out of motion vectors)...

...but here's what I found: automatic exposure produces noisy motion vectors. To get quality and regularity, use the command line switch "-ex off". If the image looks poor (dark, uncontrastful), apply a fixed amount of brightness (e.g. "-br 70") or a fixed amount of contrast, but don't switch on auto-exposure. :)

As for total darkness, I now have quality darkness with no motion detected. :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 9232
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: raspivid motion vectors

Mon Jul 17, 2017 9:25 am

cpunk wrote:Sorry for double-posting, but I'll also mention this in the thread that Hove started, to ask if he's already using this (since drone-flying people would likely be especially interested in getting undesired noise out of motion vectors)...

...but here's what I found: automatic exposure produces noisy motion vectors. To get quality and regularity, use the command line switch "-ex off". If the image looks poor (dark, uncontrastful), apply a fixed amount of brightness (e.g. "-br 70") or a fixed amount of contrast, but don't switch on auto-exposure. :)

As for total darkness, I now have quality darkness with no motion detected. :)
I'll say it again, please remember that these motion vectors are for optimising the video encoding, and as such it is just looking for the area that will give the best compression efficiency.
Any change in exposure or gain will cause the whole scene to change. The motion vector search looks for the region that closest matches the new macroblock.

Contrast and brightness are processing steps done in the YUV domain. Mess around with them too much and you'll get a noisy image which will again screw up your motion vectors. You're much better off altering the shutter speed (-ss option).
And disabling auto exposure will only be viable if you have a fairly tightly controlled scene, otherwise areas will become either saturated or black very easily.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

cpunk
Posts: 85
Joined: Thu Jun 29, 2017 12:39 pm

Re: raspivid motion vectors

Mon Jul 17, 2017 8:17 pm

Yes, saturation problems I've already seen. :) What is curious to me, is that despite them, I get higher quality motion estimation, even if the image quality is nowhere near cheerful. :) For example, I'm currently looking at a grayed-out frame with contrast and brightness totally out there... and it's tracking things nicely. :)

And of course, I've moved over into time-domain filtering with my work around yesterday. There is only so much one can expect to get from a H264 coarse estimation process... and I think I've got more than I wanted out of it. :) Now I just feed it as input into my own motion estimation layer, and out of that filtering, I should get steadily identified landmark data. :)

I always expected that my software would need to work with noisy inputs and I'm happy to note that the Pi camera + GPU combo are providing me with quite juicy and edible data. :)

But thanks for the tip, I'll try playing with shutter speed too. :)

cpunk
Posts: 85
Joined: Thu Jun 29, 2017 12:39 pm

Re: raspivid motion vectors

Mon Jul 17, 2017 8:49 pm

Just tested... given my conditions, messing with brightness and contrast is vastly preferable to setting a shutter speed. Shutter speed causes lamps to flicker despite the anti-flicker option being set, presumably because it overrides that... brightness and contrast occur as post-processing and don't interfere with anti-flicker... so they are better for my use case. :)

Return to “Camera board”