kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

VPU - Playing Video - Formats

Mon Jul 24, 2017 9:02 am

Hi,

I've been playing around with the hello_videocube and other VPU examples that play the 'Big Buck Bunny' sample video.

Problem is I have yet to figure out how to convert (say) an existing video to the same format that will play.

Anyone have any ideas, I've used Handbrake et al to convert but don't know what format/container/etc I need.

I understood that any x264 encode video would play but I'm not having much luck hitting on the magic formula.

Anyone give me any pointers as I'm starting to get quite frustrated lol. Maybe a conversion tool and necessary parameters?

Kev

Note mediainfo on the working video file -

Code: Select all

Complete name                            : test.h264
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 29.6 MiB

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings, CABAC                   : No
Format settings, ReFrames                : 2 frames
Format settings, GOP                     : M=2, N=24
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
EDIT:
As an aside, I get the same issues with the il_render_video example in the 'Raspberry Pi GPU Audio Video Programming' book, which uses effectively the same VPU setup.
Last edited by kevcal on Mon Jul 24, 2017 9:38 am, edited 2 times in total.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20773
Joined: Sat Jul 30, 2011 7:41 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:04 am

H264 should just work, as long as you keep it to 4.1 level.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
Imperf3kt
Posts: 1383
Joined: Tue Jun 20, 2017 12:16 am
Location: Australia

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:13 am

Make sure the output is 8bit and not 10bit.
If you're using handbrake, this is probably default for you. According to your mediainfo readout, I can't see any obvious reason the file won't work.

Try xmedia recode, it has better profiles and is more beginner friendly as well as being quite powerful for more advanced users.
In my experience, its more efficient than handbrake.
Stop plugging your fan directly into the GPIO 5v
https://www.electronics-tutorials.ws/power/transient-suppression.html

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:37 am

Ah, that was the mediainfo for the video that does work, sorry if it wasn't clear.
Thanks for the speedy responses, I'll try xmedia and report back :)

Yeah Level 4.1 is what I am trying to create with Handbrake, no luck yet.

User avatar
RaTTuS
Posts: 10110
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:47 am

hopw about posting the meta info of the one that does not work :roll:
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:48 am

Wow..!
I've been banging my head against a wall on this, 'xmedia recode' appears to be the answer.
I selected 'h264 raw video' and it's looking hopeful; don't knwo what I should choose, maybe there are better options?
Bit jumpy atm, but that's probably all down to the debug in my program (I hope!).
Thanks very much - no idea what I was doing before but this method appears idiot-proof, which is what I needed lol :)

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 10:22 am

>> how about posting the meta info of the one that does not work

True, lol - mediainfo for all the files that didn't work would be every file I have apart from that one (BBB) that does work..

But here's an example of a few that didn't work.. be interesting still to know why they don't.

Code: Select all

General
Unique ID                                : 327861515401527430373765722317893654291 (0xF6A7D4BD3A7F608BD53F33698F17D713)
Complete name                            : hbout.mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 31.8 MiB
Duration                                 : 30s 125ms
Overall bit rate                         : 8 842 Kbps
Encoded date                             : UTC 
Writing application                      : HandBrake 0.10.2 2015060900
Writing library                          : Lavf55.12.0 / Lavf55.12.0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 30s 125ms
Bit rate                                 : 8 666 Kbps
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 24.000 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.174
Stream size                              : 31.1 MiB (98%)
Writing library                          : x264 core 142 r2479 dd79a61
Encoding settings                        : cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=6 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=240 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=10 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=50000 / vbv_bufsize=62500 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=1:1.00
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Code: Select all

General
Unique ID                                : 146208872809913618651181288651015648346 (0x6DFECDFC19480F49B8C55CB5C024F45A)
Complete name                            : hbtest.mkv
Format                                   : Matroska
Format version                           : Version 2
File size                                : 16.5 MiB
Duration                                 : 30s 30ms
Overall bit rate                         : 4 614 Kbps
Encoded date                             : UTC 
Writing application                      : HandBrake 0.10.2 2015060900
Writing library                          : Lavf55.12.0 / Lavf55.12.0

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings, CABAC                   : No
Format settings, ReFrames                : 3 frames
Format settings, GOP                     : M=1, N=32
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 30s 30ms
Bit rate                                 : 4 522 Kbps
Width                                    : 720 pixels
Height                                   : 406 pixels
Display aspect ratio                     : 16:9
Original display aspect ratio            : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.516
Stream size                              : 16.2 MiB (98%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
It's probably all due to me not understanding the code/container meanings. I've tried reading wikis, etc on this but I'm not getting it. I'm just too thick I think.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5164
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 11:44 am


kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 12:04 pm

I'll give this a try, thanks.
The ffmpeg command line that works for me is:
ffmpeg -i big_buck_bunny_1080p_h264.mov -vcodec copy -bsf h264_mp4toannexb -an bbb.h264

What it does is:
-an: stripping audio
-vcodec copy: copying the encoded video data without reencoding
-bsf h264_mp4toannexb: change Quicktime style NAL headers (with length field) to standard NAL headers with 0x00000001 sync code.

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 12:32 pm

Still having issues - the following one created using ffmpeg does not work -

Code: Select all

mediainfo out.h264 
General
Complete name                            : out.h264
Format                                   : AVC
Format/Info                              : Advanced Video Codec
File size                                : 2.71 MiB
Writing library                          : x264 core 152 r2851 ba24899
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Video
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : [email protected]
Format settings, CABAC                   : Yes
Format settings, ReFrames                : 4 frames
Width                                    : 640 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 4:3
Frame rate mode                          : Variable
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Writing library                          : x264 core 152 r2851 ba24899
Encoding settings                        : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=6 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Command was

Code: Select all

ffmpeg -i SVmp4_720x480_10mb.mp4 -c:v libx264 -profile:v main -level:v 4.1 out.h264
Anything obvious wrong with this file?

Kev

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 4:14 pm

Mmm..
Maybe it is playing.. it's what I do with the EGL image after that may be the issue.
Using the above encoding line but adding scale=1920:1080 fixed things..
More investigation required methinks!

Thanks for all the input, it's been a great help.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 20773
Joined: Sat Jul 30, 2011 7:41 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 8:47 pm

So you are doing something with the resulting images? Not using omxplayer or similar? That wasn't at all clear. Usually the GPU outputs directly to the display, which is all done in HW. That should work fine. Once you start outputting to the 3D system things will slow down a lot.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Mon Jul 24, 2017 9:36 pm

No, I'm setting up the VPU explicitly then using EGL image and then displaying it using the Allegro 5 graphics library...

Had to hack the library to use the memory bitmap returned from eglCreateImageKHR()..
Seems to work well, had some issues as forgot the image sizes opengles uses and stuff..
I'm getting 28fps or so (on rpi model b) with what I have at the moment.
Have currently set max fps to 30, will up that and see how many fps I lose rendering video and resizing and blitting it to screen...

Been an interesting mess around project ;)

This part was a problem as everything I'd done was using the BBB demo video to test and then it all fell apart when I tried using other videos...

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5164
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: VPU - Playing Video - Formats

Tue Jul 25, 2017 3:28 pm

kevcal wrote:
Mon Jul 24, 2017 12:32 pm
Command was

Code: Select all

ffmpeg -i SVmp4_720x480_10mb.mp4 -c:v libx264 -profile:v main -level:v 4.1 out.h264
Anything obvious wrong with this file?
Any reason why you are not using the "-bsf h264_mp4toannexb" option suggested?

kevcal
Posts: 15
Joined: Sun Nov 18, 2012 12:56 pm

Re: VPU - Playing Video - Formats

Wed Jul 26, 2017 7:12 am

It didn't appear to work when I tried it, but have now figured out my outputting of the rendered image to screen was an issue (to do with sizing of bitmap I'm rendering to not matching video resolution) so will be revisiting the conversion today..

I admit I'm more than a little confused about all these differing codecs and mkv/mp4/x264/etc etc ;)

Currently trying to get the code to complete the rendering phase after it's finished the file reading; something wrong with my code to do with 'end of stream'.. Think I'll sort it tomorrow now.

Return to “C/C++”