linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Fri Aug 03, 2012 11:02 pm

[quote="jacksonliam"The Hubble clip, it has no sound so does it need demuxing? I thought your latest upload was test2?[/quote]

Sorry, you're right, test2 is the last one I uploaded. And yes, it does need demuxing (use extract_mpeg from libmpeg2 or just use hst_2.m2v from the zip file I uploaded).

User avatar
JonathanGraham
Posts: 39
Joined: Thu Jul 05, 2012 5:55 am

Re: MPEG-2 Decoding

Sat Aug 04, 2012 9:05 pm

bbb wrote: * Difference between -ao null and -ao alsa is only 0.2%. Not worth investigating further solutions like a openmaxil 'ao' driver.
* ALSA driver on the PI does not accept floating point input.
Yes, for at least the AC3 decode about 1% of overall CPU is spent in ffmpeg/libavcodec/fmtconvert.c in a function called int32_to_float_fmul_scalar_c which is just:

Code: Select all

static void int32_to_float_fmul_scalar_c(float *dst, const int *src, float mul, int len){
    int i;
    for(i=0; i<len; i++)
        dst[i] = src[i] * mul;
}
Which based on some output from gcc works out to about 19 instructions per move. The following:

Code: Select all

static void int32_to_float_fmul_scalar_c(register float *dst, register  int *src, register float mul, int len){
register int n = len>>3;
do {
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        *dst++ = *src++*mul;
        } while ( --n > 0);
}
Drops that to about 8 instructions per move (the header file says that len is only going to be multiples of 8). Give it a shot and see what you think.

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Sun Aug 05, 2012 10:07 am

Does anyone know if openmax itself supports floats (presumably converting to ints on the GPU)? If so, then it would seem to make it worthwhile (or perhaps adding float support to ALSA if feasible).

I've made my code available on githiub now, as part of my planned project to write client software for the Pi for DVB-over-IP streaming:

http://github.com/linuxstb/pidvbip

My "sample1.c" program is now called "mpeg2test.c"

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Sun Aug 05, 2012 10:14 am

linuxstb wrote:Does anyone know if openmax itself supports floats (presumably converting to ints on the GPU)? If so, then it would seem to make it worthwhile (or perhaps adding float support to ALSA if feasible).
It does not seems to be supported:
https://github.com/raspberrypi/firmware ... ender.html

(Btw if your target is dvb, do you know vdr and vomp)

I try today to download, it seems that they are gone. I would like to test them with my transcoder, can you please make them availiable to me.

Marten

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Sun Aug 05, 2012 1:42 pm

Sorry, the link to my test files was always broken. It should now be working:

http://linuxstb.cream.org/mpeg2_samples_m2v.zip (52MB)

Yes, I'm familiar with VDR, and used it with a "full-featured" DVB card back in its early days, although I chose not to use it because I wanted something to record the original transport streams, and VDR remultiplexed them into its own strange format (the format the full-featured DVB cards used). I recall seeing something about VDR changing to use TS format - did that ever happen?

I've also played a little with the MediaMVP, but never looked at vomp. I recently started writing a streaming MPEG-2 TS player for the MVP, but stopped when I heard about the Pi.

To be honest, I'm still unsure of the approach to take to meet my DVB streaming needs - whether to do it all myself, or to use/adapt existing software. Nothing out there seems to meet my needs perfectly, but I'll look at VDR/vomp.

But I guess we're getting off-topic for this thread...

umaar
Posts: 1
Joined: Sun Aug 05, 2012 4:38 pm

Re: MPEG-2 Decoding

Sun Aug 05, 2012 4:45 pm

Hallo linuxstb,

if you want use vdr or any kind of dvb you could try it with http://www.minidvblinux.de/

You could find there http://www.minidvblinux.de/download.php ... .14_46.tgz for client modus

if you want to use http://www.minidvblinux.de/download.php ... .14_46.tgz you could use a DVB-T USB.

The client version already works with HD Stream but SD Stream doesn't work because of the missing mpeg2 support.

Perhaps you could use this image.

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Sun Aug 05, 2012 5:32 pm

For those interested in discussing DVB on the Pi, I've started a new thread here:

http://www.raspberrypi.org/phpBB3/viewt ... 35&t=13627

Can we keep this one on-topic to the technicalities of MPEG-2 decoding?

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Sun Aug 05, 2012 5:49 pm

linuxstb wrote:Sorry, the link to my test files was always broken. It should now be working:

http://linuxstb.cream.org/mpeg2_samples_m2v.zip (52MB)

...
Thanks got it. I am wondering, why the sequence headers were removed in some of the elementary streams.
I had to reinsert them with ProjectX.
I will post benchmarks later.

Marten

Btw. vdr uses TS since 1.7. branch, (it is the unstable branch, but undeed very stable)

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Sun Aug 05, 2012 8:32 pm

My benchmark results with the my transcoding code:
bbc dvb-t 50.9
bbc dvb-s 39.1
centaur 51.6
dave 78.98
hst 50.53
ARD dvb-c 30.1
( this is my sample german public television,now I know why my fps values were so bad)

Marten

User avatar
JonathanGraham
Posts: 39
Joined: Thu Jul 05, 2012 5:55 am

Re: MPEG-2 Decoding

Mon Aug 06, 2012 1:32 pm

MartenR wrote:My benchmark results with the my transcoding code:
Are those benchmarks just for the transcoding or also the display? i.e. is your code converting to mp4 and then pushing the resultant stream through OpenMAX to decode and then display? Are you doing audio as well? I'm just trying to figure out how much of a real advantage we're seeing here.

My progress in other areas:

Most of my tests are with DVDs. Right now I'm pretty much at the point where I can play a number of DVD's over the network with audio and < 1% framedrop. I'm also realizing that not all DVD video streams are created equal for example a high-quality rips like Disney's Bolt and the Star Wars Movies play back almost perfectly whereas my copy of Ong-Bak (The definitive Mui-Thai movie) drops close to 4% on network play.

I might hook up a DVD drive over USB to eliminate the network element from the tests but considering how slow our raw reads are on SD I find it pretty much equivalent to network play of SMB and slower than NFS (of course).

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Mon Aug 06, 2012 2:10 pm

So far only transcoding, everything in memory.
(But the code is still unoptimized, gettings bugs out had higher priority)

If you see the hello_video sample and switch it to mpeg4 playback, CPU load is only 5% including disk and I can write directly to the omx buffer.
So I do not expect much additional load.

Marten

User avatar
JonathanGraham
Posts: 39
Joined: Thu Jul 05, 2012 5:55 am

Re: MPEG-2 Decoding

Wed Aug 08, 2012 3:42 am

JonathanGraham wrote: Actually in mplayer I believe most codecs use the libvo function display_osd to accomplish soft/DVD subtitles. It looks trivial to implement - I just haven't had much time.
I just implemented this in a quick and dirty way. Using the built-in draw_alpha functions (similar to the way the xv driver does) and it works - doesn't seem too much of a drain on resources either.

I tried some DVD's as well as some soft-subbed ogm files and the only real problems seem to be some ghosting probably due to the crazy buffer scheme used in mplayer's direct rendering system. I can probably fix this and reduce resource use by simply drawing to a new element.

Once I get that working I'll see if I can fish out some low-bitrate files with styled subs which would be the next logical thing to test.

bbb
Posts: 55
Joined: Sat Jun 02, 2012 9:52 am

Re: MPEG-2 Decoding

Wed Aug 08, 2012 10:38 am

linuxstb wrote:Does anyone know if openmax itself supports floats (presumably converting to ints on the GPU)? If so, then it would seem to make it worthwhile (or perhaps adding float support to ALSA if feasible).

I've made my code available on githiub now, as part of my planned project to write client software for the Pi for DVB-over-IP streaming:

http://github.com/linuxstb/pidvbip

My "sample1.c" program is now called "mpeg2test.c"
I have had a look at this code. I added -O2 to the CLFAGS and disabled asserts by commenting out #include <assert.h> and adding #define assert(exp) (void) exp which got a little bit more speed.
With a slight overclock, the PI playing centaur_2.m2v and hst_2.m2v just about at real time, but still not quite fast enough to allow enough spare CPU to also do the audio decoding

So I had a play with gprof and the idct_add stage is taking quite a fair wack of the total time and its using the C version, which could do with a bit of optimization :) If I can figure out the format of the 2 data blocks passed in I might be able to get a bit more out of it.

I had a go at tweaking mpeg2test.c but it is as good as it is going to get from what I can tell :)

I might have a go at getting some decoding code working that uses libavcodec with your dismanx code, it be interesting to profile it and see what idct implementation it is using.

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: MPEG-2 Decoding

Wed Aug 08, 2012 8:42 pm

I've got libavcodec outputting to dispmanx, see my zip above, each frame gets memcpy to a buffer the right format for dispmanx, I think that causes a ~5fps hit, would be great if you got the libavcodec buffers to output in the right format!

Without sound UK DVB samples play at faster than real time! I have some source code somewhere which will output sound and video from libavcodec but it uses SDL and the whole thing is just a mess of SDL code, stripping it out in place of dispmanx seems slightly outside of my ability.

Hexagon
Posts: 10
Joined: Tue Jul 31, 2012 2:14 pm

Re: MPEG-2 Decoding

Wed Aug 08, 2012 8:58 pm

JonathanGraham: I would really like to try out your mplayer build, any repo or binaries available? :)

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Wed Aug 08, 2012 10:38 pm

bbb wrote:I have had a look at this code. I added -O2 to the CLFAGS and disabled asserts by commenting out #include <assert.h> and adding #define assert(exp) (void) exp which got a little bit more speed.
With a slight overclock, the PI playing centaur_2.m2v and hst_2.m2v just about at real time, but still not quite fast enough to allow enough spare CPU to also do the audio decoding

So I had a play with gprof and the idct_add stage is taking quite a fair wack of the total time and its using the C version, which could do with a bit of optimization :) If I can figure out the format of the 2 data blocks passed in I might be able to get a bit more out of it.

I had a go at tweaking mpeg2test.c but it is as good as it is going to get from what I can tell :)

I might have a go at getting some decoding code working that uses libavcodec with your dismanx code, it be interesting to profile it and see what idct implementation it is using.
Thanks for the feedback. Regarding the asserts, the "correct" way to disable them is to add the -DNDEBUG flag (or #define NDEBUG in the code, before assert.h is included).

I'm finding it hard to benchmark this code accurately - no two runs ever produce the same FPS...

Following from your comment about idct_add, I tried again to incorporate the ARM version from the Rockbox project. There are two versions there - one for ARMv4/5 (I think), and the other for ARMv6.

My previous attempts with the ARMv6 version have just resulted in segmentation faults. Today I tried the other version, and this works fine, giving about a 2fps speedup on a test file that decodes at around 26fps.

I've now committed that change to my git repo, and have also included the armv6 version if anyone wants to try and find out why it doesn't work...

But I'm guessing there still room to improve that code if it can be optimised for our ARM.

BTW, do you (or anyone else) have any tips for benchmarking this code? I find I can get up to 2 or 3fps differences with the same code, depending on when I run it.

User avatar
JonathanGraham
Posts: 39
Joined: Thu Jul 05, 2012 5:55 am

Re: MPEG-2 Decoding

Thu Aug 09, 2012 2:33 pm

Hexagon wrote:JonathanGraham: I would really like to try out your mplayer build, any repo or binaries available? :)
Hey thanks, I'm flattered. Let me see what I can come up with in the next day or two...

I have two optimizations in place one is I have my clock at 875Mhz (no change to ram) and I have compiled my kernel without framebuffer support.

My build with those two things in place I can...

Play a number of dvd rips (9000Kbps) with 2 channel AC3 sound with < 1% frame drop
Play almost any low bitrate video (4000Kbps) on a supported codec (DivX, XVID, OGG)
Soft subtitles and OSD appear to work with some noise (but I'm going to try to fix that today)

Caveats...

No native acceleration so H264 MP4 etc play like a slide show...if they play at all
There's a bug in my stride code somewhere so some DVDs or other video at certain sizes crash the player.
Last edited by JonathanGraham on Thu Aug 09, 2012 2:40 pm, edited 1 time in total.

User avatar
JonathanGraham
Posts: 39
Joined: Thu Jul 05, 2012 5:55 am

Re: MPEG-2 Decoding

Thu Aug 09, 2012 2:38 pm

linuxstb wrote: I'm finding it hard to benchmark this code accurately - no two runs ever produce the same FPS...
My advice:

i) Try to remove sources of variance: i.e. test with USB off or without FB support interrupts and any unnecessary services. Interrupts kill performance metrics.

ii) Do a large sample run of at least 15 attempts. I use a simple shell for loop:

Code: Select all

for x in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15:
do
./test_case1
./test_case2
done
ii) Take the mean, median and standard deviation of the mean of the values. If the mean and median of whatever you're measuring for case1 (ie fps) is better than case 2 and the standard deviation of the mean of case 1 is equal to or smaller than case2. Then it is likely case1 is better performing (in the measured metric) than case2.

Hexagon
Posts: 10
Joined: Tue Jul 31, 2012 2:14 pm

Re: MPEG-2 Decoding

Fri Aug 10, 2012 5:55 am

JonathanGraham wrote: Hey thanks, I'm flattered. Let me see what I can come up with in the next day or two...
Awesome, thanks! I've followed this thread from day one :)
JonathanGraham wrote: My build with those two things in place I can...

Play a number of dvd rips (9000Kbps) with 2 channel AC3 sound with < 1% frame drop
Play almost any low bitrate video (4000Kbps) on a supported codec (DivX, XVID, OGG)
Soft subtitles and OSD appear to work with some noise (but I'm going to try to fix that today)

Caveats...

No native acceleration so H264 MP4 etc play like a slide show...if they play at all
There's a bug in my stride code somewhere so some DVDs or other video at certain sizes crash the player.
That's perfectly fine, it's possible to select which player to use for which file format in XBMC. So omxplayer will still be used for mpeg4.

Further on, basic support for mpeg2 playback would be a huuge improvement for XBian. -> http://forum.xbian.org/viewtopic.php?f=8&t=3

Keep up the great work!

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Fri Aug 10, 2012 8:16 am

All,

I've been working on a DVB streaming client for the Pi for use with Tvheadend (which uses its own htsp protocol for streaming). This protocol is very easy to deal with as it provides packets containing complete frames, so no parsing overhead or copying of the data is needed before passing it into libmpeg2 - I just read it from a network socket and then pass a pointer to that data to libmpeg2.

(I think libmpeg2 is doing a little bit of data copying internally though - I need to see if I can persuade it not to do that).

This code is in my git repo - https://github.com/linuxstb/pidvbip - as htsptest.c I've also modularised all my other code, creating a threaded mpeg-2 decoder and a separate "vo_pi.c" file for screen output.

I have a couple of issues which I am hoping someone reading this thread can help with though:

1) Timing of frame output. In my earlier tests, I was able to use the "sync" variant of the update function to get steady 25fps output (on streams that decoded faster than that). However, this doesn't appear to work any more. I'm guessing it's due to the optimisations suggested earlier in this thread by JonathanGraham. I need to go back to an older version of this code and check it again. Currently my code just uses the system clock to attempt to display frames at the correct time, usleep()ing if the frame has been decoded too early.

2) Interlaced output. No-one has mentioned this so far, but if I understand correctly, MPEG-2 can store interlaced frames in one of two ways - each field being encoded independently, or a frame consisting of two interlaced field being compressed, and then the display program should display each field in turn at the appropriate point in time. IIUC, this second method is how DVB is normally broadcast.

I was wondering if ffmpeg/mplayer did this "properly" or if they simply display a 50Hz interlaced stream as a 25fps progressive stream (which is also what I'm currently doing). I think there would be quite a performance hit if we do this, unless there is a way for the GPU to assist.


As for current performance, htsptest manages *on average* to decode all my DVB-T streams in realtime, but it doesn't always manage to decode every frame within the 40ms period between display. This is on an unoptimised Pi (700MHz clock, standard kernel configuration). CPU usage ranges between about 50% and 90% according to "top".

User avatar
jacksonliam
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm

Re: MPEG-2 Decoding

Fri Aug 10, 2012 9:10 am

I'm thinking we won't get deinterlacing (perhaps with MartinR 's method?) , I'm pretty sure I read in the omx thread that deinterlacing is done on the cpu atm anyway. It might be worth watching if that becomes gpu code it may be useful to us.

Linuxstb, does your code do sound? I thought media players used the sound to keep video in sync? If you changed the update_sync call to just update, then changing it back will limit your decoding loop to vsync

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Fri Aug 10, 2012 9:33 am

I'm thinking we won't get deinterlacing (perhaps with MartinR 's method?) , I'm pretty sure I read in the omx thread that deinterlacing is done on the cpu atm anyway
Well, with the transcoder you will have this. Since there is an OMX component which does deinterlacing after mpeg4 decoding.

Marten

linuxstb
Posts: 77
Joined: Sat Jul 07, 2012 11:07 pm

Re: MPEG-2 Decoding

Fri Aug 10, 2012 10:05 am

MartenR wrote:Well, with the transcoder you will have this. Since there is an OMX component which does deinterlacing after mpeg4 decoding.

Marten
How dependent is your transcoder on libav/ffmpeg? Would it be possible to extract the code into a standalone tool/library?

MartenR
Posts: 46
Joined: Sat Mar 03, 2012 9:15 am

Re: MPEG-2 Decoding

Fri Aug 10, 2012 11:07 am

How dependent is your transcoder on libav/ffmpeg? Would it be possible to extract the code into a standalone tool/library?
It uses a lot of stuff from libav. In principle it is possible to extract it. But I think it is ok to use libav. I will use it as a static library only with the codecs compiled in which I use for my project.

Marten

shalo
Posts: 74
Joined: Tue May 08, 2012 7:25 pm

Re: MPEG-2 Decoding

Fri Aug 10, 2012 11:43 am

I guess it may only be slightly related but I think could be of interest to people involved in this thread. Check out the notes for the latest pi firmware, "Preparation for additional codecs".

see: https://github.com/raspberrypi/firmware

Return to “Advanced users”