ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

OpenMax Discarding Frames

Wed Jun 28, 2017 8:19 am

Hello All,
I am encoding video from camera into three different bit rates using openmax. After encoding i saved it to files and it works fine. Then i convert this saved file into mpegts which works fine but when i tried to take live video and convert it into mpegts and after conversion into mpegts i want to save my output to file it does not work. I checked my message log which is showing me discarded frames.
Ilyani

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 9:48 am

Your message log where? In your app, from GPU logs (vcdbg), or somewhere else?
What "doesn't work"? No file, or not playable? What are doing with regard header bytes?

How have you determined that frames have been discarded by IL? That is actually valid for bit rate control should the bitrate really overshoot (this is single pass encoding, so you react to what the last frame produced and set the quantisation parameters assuming the next frame is of a similar complexity).
How are you splitting your images into the multiple encoders? Video_splitter will discard buffers if the output ports don't consume the data fast enough. When tunneled some components only hold the last presented frame for processing, so again will discard data.

You haven't given enough information to be able to advise further. Can you either provide more detail, or post your code somewhere where we can do a quick review of it?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 10:04 am

First of All thnx for your reply. Yes I checked my discarded frames in vcdbg. Also i am using video_splitter for splitting my video for different bit rates encoders. Output video is playable but of short duration. How to force splitter not to discard frames. My code is on Pastebin at https://pastebin.com/L7GGpRn0

Ilyani

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 10:53 am

ilyani wrote:First of All thnx for your reply. Yes I checked my discarded frames in vcdbg.
So exactly what message are you seeing logged there? Video splitter doesn't appear to log anything when it discards frames, so I'm assuming you're not seeing that.
ilyani wrote:Also i am using video_splitter for splitting my video for different bit rates encoders. Output video is playable but of short duration. How to force splitter not to discard frames.
You don't - you make things consume the data in a timely manner. The camera isn't going to stop - it will produce N frames per second. Unless you have infinite buffering in the system then something has to give if you're not keeping up.
ilyani wrote:My code is on Pastebin at https://pastebin.com/L7GGpRn0
What resolution are you capturing at?
The encoder is designed for 1080P30, or around 62megapixels/second. Splitting it between multiple streams has a small overhead, but it means that 1024x768@30fps is over the limit (70.7MPix/s). 800x600@30 should be achievable (43MPix/s).
The pipeline you have set up will be hitting memory pretty hard, but there's not much you can do about that.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 11:18 am

I checked with sudo vcdbg log msg and i got this log which is attached. I am using 640x480 resolution @30fps.
Attachments
2017-06-28-125651_1920x1080_scrot (1).png
2017-06-28-125651_1920x1080_scrot (1).png (46.9 KiB) Viewed 1701 times

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 12:33 pm

So it's not OpenMax dropping frames.
venc_rc = Video ENCoder Rate Control
You've set parameters such that rate control can't achieve your requested bitrate, so it drops to the option of last resort which is skipping the frame (in theory it could encode all "skip" macroblocks, but that has no benefit over producing a buffer with 0 bytes in length).

You've got calls in to setQPLimits to set OMX_IndexParamBrcmVideoEncodeMinQuant and OMX_IndexParamBrcmVideoEncodeMaxQuant. Why?
You can see from that log that QP is pegged at 48 and not being allowed to change.
Set the bitrate mode to OMX_Video_ControlRateVariable, and the bitrate to that you desire, and then leave other things alone. Likewise setLowLatency and setEED.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Wed Jun 28, 2017 12:52 pm

Now it's not dropping any frame.thnx :)

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 4:03 pm

Now the problem i am getting is my pi is giving me device busy status when i run my program and if i check my log with vcdbg it is dropping frames.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 4:24 pm

Details please!
What device is busy?
How are you determining that you are dropping frames? If it rate control again, then that is legitimate if that is the only way the target bitrate can be achieved.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 4:28 pm

The device is a Logitech C210 camera.and i am checking my log with vcdbg . It is showing some frame drops with different Qp. Some times device work and sometimes show busy but the output file is of very short duration

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 4:38 pm

ilyani wrote:The device is a Logitech C210 camera.
V4L2 will only allow one client at a time requesting buffers from the camera. Are you sure you're not trying to run two instances of your app simultaneously? That appear to be anything to do with IL.
ilyani wrote:and i am checking my log with vcdbg . It is showing some frame drops with different Qp. Some times device work and sometimes show busy but the output file is of very short duration
As I've said, the "venc_rc: frame N aborted" message is valid if you are running at low bitrates, or the bitrate control overshoots the target bitrate by too large a margin.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 4:44 pm

I tried previously it is working fine. Today i tried with the same code without changing but it is giving this error. and how to check that my usb cam is working with only one code at a time.I am running only one command line and executing only one pogram. I restarted my pi but the problem remain same.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Mon Jul 03, 2017 6:59 pm

The problem is sometimes webcam works and sometimes it generates error resource busy. Due to this video duration is short and also ffprobe is unable to calculate video duration as i am assigning pts with the actual time

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Tue Jul 04, 2017 9:49 am

If i try with ffmpeg it shows device or resource busy error for 2 to 3 times and then it starts working.After quitting if i try again then it generates the same error for 2 to 3 times and then starts working.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4449
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: OpenMax Discarding Frames

Tue Jul 04, 2017 1:41 pm

If ffmpeg is having issues accessing your USB webcam, then you need to debug that.

Try running "dmesg" to see whether the kernel is throwing errors.
You can also enable more debug from the V4L2 core generally by approaches such as

Code: Select all

echo -n 0xFFFF | sudo tee /sys/module/videobuf2_core/parameters/debug
or other similar debug point in sysfs for uvcvideo. Again the output will be logged to dmesg.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

ilyani
Posts: 11
Joined: Thu Jan 12, 2017 9:28 pm

Re: OpenMax Discarding Frames

Tue Jul 04, 2017 2:00 pm

dmesg is generating some information with some errors but other debug command is giving me nothing. Also i tried with vlc which if i try 2 to 3 times works for one time and then generate error. Basically this problem starts after recent update.

Return to “OpenMAX”

Who is online

Users browsing this forum: No registered users and 1 guest