Domarius
Posts: 37
Joined: Sat Oct 14, 2017 1:44 am

SDL + libvlc choppy full screen on Pi 3, but actual VLC playback is fine

Sun Apr 07, 2019 2:50 am

Under Raspbarian, using the included VLC app, playing videos full screen on my 800x480 display works great, at full speed, no frames dropped. But in my app compiled with SDL 1.2 and libvlc, the full screen playback is choppy!

More detail: I'm making an app with video playback in it, and using the SDL1.2 example from the libvlc docs, found here https://wiki.videolan.org/LibVLC_SampleCode_SDL/ NOTE: there are 2 examples given on that page, the first one for SDL 1.2, and the second one for SDL2. Both examples play the video in a small 320x240 picture within side the main window. I'm using the SDL 1.2 one, because the SDL2 one was choppy on the Pi 3. But I've since tweaked the SDL 1.2 one to be fullscreen, and now it is choppy also. I gathered from some prior error messages that it's scaling in software.

My question: What is the actual VLC player doing differently that it runs great at fullscreen, while my libvlc implementation is choppy? I found some older posts related to enabling experimental Pi OpenGL for better performance, but unless this is what the actual VLC app is doing, I'd rather avoid something experimental!

Here's my code for reference... You must include the libraries SDL and libvlc.

Code: Select all

/* libSDL and libVLC sample code
 * Copyright © 2008 Sam Hocevar <[email protected]>
 * license: [http://en.wikipedia.org/wiki/WTFPL WTFPL] */

#include <stdio.h>
#include <stdint.h>
#include <math.h>
#include <stdlib.h>
#include <assert.h>

#include <SDL/SDL.h>
#include <SDL/SDL_mutex.h>

#include <vlc/vlc.h>

int window_width = 640;
int window_height = 480;
int fullscreen_width = 0;
int fullscreen_height = 0;
int current_width = 0;
int current_height = 0;
bool fullscreen = false;
int screen_options = SDL_ANYFORMAT | SDL_HWSURFACE | SDL_DOUBLEBUF;
SDL_Surface *screen;

int video_width = 0;
int video_height = 0;

struct ctx
{
    SDL_Surface *surf;
    SDL_mutex *mutex;
};

static void *lock(void *data, void **p_pixels)
{
    struct ctx *ctx = data;

    SDL_LockMutex(ctx->mutex);
    SDL_LockSurface(ctx->surf);
    *p_pixels = ctx->surf->pixels;
    return NULL; /* picture identifier, not needed here */
}

static void unlock(void *data, void *id, void *const *p_pixels)
{
    struct ctx *ctx = data;

    /* VLC just rendered the video, but we can also render stuff */
    uint16_t *pixels = *p_pixels;
    int x, y;

    for(y = 10; y < 40; y++)
        for(x = 10; x < 40; x++)
            if(x < 13 || y < 13 || x > 36 || y > 36)
                pixels[y * video_width + x] = 0xffff;
            else
                pixels[y * video_width + x] = 0x0;

    SDL_UnlockSurface(ctx->surf);
    SDL_UnlockMutex(ctx->mutex);

    assert(id == NULL); /* picture identifier, not needed here */
}

static void display(void *data, void *id)
{
    /* VLC wants to display the video */
    (void) data;
    assert(id == NULL);
}

static bool set_fullscreen(bool b)
{
    fullscreen = b;
    int options;
    if(fullscreen)
    {
        options = screen_options | SDL_FULLSCREEN;
        current_width = fullscreen_width;
        current_height = fullscreen_height;
    }
    else
    {
        options = screen_options;
        current_width = window_width;
        current_height = window_height;
    }
    printf("Setting resolution %i,%i\n",current_width, current_height);
    screen = SDL_SetVideoMode(current_width, current_height, 0, options);
    if(!screen)
    {
        printf("cannot set video mode\n");
        return false;
    }
    return true;
}

int main(int argc, char *argv[])
{
    libvlc_instance_t *libvlc;
    libvlc_media_t *m;
    libvlc_media_player_t *mp;
    char const *vlc_argv[] =
    {
        //"--no-audio", /* skip any audio track */
        "--no-xlib", /* tell VLC to not use Xlib */
    };
    int vlc_argc = sizeof(vlc_argv) / sizeof(*vlc_argv);

    SDL_Surface *empty;
    SDL_Event event;
    SDL_Rect rect;
    int done = 0, action = 0, pause = 0, n = 0;

    struct ctx ctx;

    if(argc < 2)
    {
        printf("Usage: %s <filename>\n", argv[0]);
        return EXIT_FAILURE;
    }

    /*
     *  Initialise libSDL
     */
    if(SDL_Init(SDL_INIT_VIDEO | SDL_INIT_EVENTTHREAD) == -1)
    {
        printf("cannot initialize SDL\n");
        return EXIT_FAILURE;
    }


    //Create window that fills the desktop resolution
    const SDL_VideoInfo* vid_info = SDL_GetVideoInfo();
    fullscreen_width = vid_info->current_w;
    fullscreen_height = vid_info->current_h;
    if(argc > 2)//TODO: Remove all of this when have a settings file.
    {
        fullscreen_width = atoi(argv[2]);
        fullscreen_height = atoi(argv[3]);
    }
    window_width = fullscreen_width / 2;
    window_height = fullscreen_height / 2;

    video_width = (int)((4.0 / 3.0) * (float)fullscreen_height); //Force 4:3
    video_height = fullscreen_height;
    current_width = fullscreen_width;
    current_height = fullscreen_height;

    set_fullscreen(true);

    printf("Desktop res %i, %i\n", fullscreen_width, fullscreen_height);

    //Renderable surface for video,not using videowidth, videoheight right now because I want it to be full screen.
    //VIDEOWID
    empty = SDL_CreateRGBSurface(SDL_SWSURFACE, video_width, video_height,
                                 32, 0, 0, 0, 0);
    ctx.surf = SDL_CreateRGBSurface(SDL_SWSURFACE, video_width, video_height,
                                    16, 0x001f, 0x07e0, 0xf800, 0);
    ctx.mutex = SDL_CreateMutex();

    /*
     *  Initialise libVLC
     */
    libvlc = libvlc_new(vlc_argc, vlc_argv);
    m = libvlc_media_new_path(libvlc, argv[1]);
    mp = libvlc_media_player_new_from_media(m);
    libvlc_media_release(m);

    libvlc_video_set_callbacks(mp, lock, unlock, display, &ctx);
    libvlc_video_set_format(mp, "RV16", video_width, video_height, video_width*2);
    libvlc_media_player_play(mp);

    /*
     *  Main loop
     */
    rect.w = 0;
    rect.h = 0;
    rect.x = (current_width - video_width) / 2;
    rect.y = (current_height - video_height) / 2;

    while(!done)
    {
        action = 0;

        /* Keys: enter (fullscreen), space (pause), escape (quit) */
        while( SDL_PollEvent( &event ) )
        {
            switch(event.type)
            {
            case SDL_QUIT:
                done = 1;
                break;
            case SDL_KEYDOWN:
                action = event.key.keysym.sym;
                break;
            }
        }

        switch(action)
        {
        case SDLK_ESCAPE:
            done = 1;
            break;
        case SDLK_RETURN:
            set_fullscreen(!fullscreen);
            break;
        case SDLK_SPACE:
            pause = !pause;
            break;
        }

        //rect.x = (int)((1. + .5 * sin(0.03 * n)) * (current_width - video_width) / 2);
        //rect.y = (int)((1. + .5 * cos(0.03 * n)) * (current_height - video_height) / 2);
        //if(!pause)
            //n++;

        /* Blitting the surface does not prevent it from being locked and
         * written to by another thread, so we use this additional mutex. */
        SDL_LockMutex(ctx.mutex);
        SDL_BlitSurface(ctx.surf, NULL, screen, &rect);
        SDL_UnlockMutex(ctx.mutex);

        SDL_Flip(screen);
        SDL_Delay(10);

        SDL_BlitSurface(empty, NULL, screen, &rect);
    }

    /*
     * Stop stream and clean up libVLC
     */
    libvlc_media_player_stop(mp);
    libvlc_media_player_release(mp);
    libvlc_release(libvlc);

    /*
     * Close window and clean up libSDL
     */
    SDL_DestroyMutex(ctx.mutex);
    SDL_FreeSurface(ctx.surf);
    SDL_FreeSurface(empty);

    SDL_Quit();

    return 0;
}

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 6640
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: SDL + libvlc choppy full screen on Pi 3, but actual VLC playback is fine

Sun Apr 07, 2019 7:45 am

The Raspbian version of vlc has various patches that improve performance.

Firstly it uses hardware to resize and format convert the frames to be exactly the same size as the output window. This avoids any processing on the arm other than a simple blit (which is actually still pretty expensive in terms of memory bandwidth). It also composes any overlays (eg subtitles or on screen displays) into the video frame.

When in full screen mode it takes a different path again and uses MMAL video render components to render the video. This makes best use of the hardware, and is effectively the same as omxplayer.

When using libvlc I expect you get the raw video frames out at their native size. Getting that onto the screen efficiently is a trickier path. I don't know whether you can use any of the accelerated functions in that use case.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Domarius
Posts: 37
Joined: Sat Oct 14, 2017 1:44 am

Re: SDL + libvlc choppy full screen on Pi 3, but actual VLC playback is fine

Sun Apr 07, 2019 11:20 pm

Wow thank you. That certainly is doing a lot more then.

Well, the python implementation seems to run fast, perhaps I’ll persevere with that...
I’ll try asking at the VLC forums, see what they say. Thanks again.

Return to “Graphics programming”