Page 1 of 1

How to use mmal to decode jpeg image

Posted: Sat Nov 24, 2018 5:33 am
by ksharindam
Is it possible to decode a jpeg image using mmal?
If yes , then tell me how. It wiil be helpful if you give code example.

I know it is possible using openmax but i want to use simpler mmal api.

Re: How to use mmal to decode jpeg image

Posted: Mon Nov 26, 2018 3:19 pm
by ksharindam
Finally found the example at ... peg/jpeg.c

Re: How to use mmal to decode jpeg image

Posted: Tue Dec 04, 2018 4:11 am
by Stephbaker
Thanks for the information! I really wasn't looking for this question but I had it in my mind, too! Thaks again

Re: How to use mmal to decode jpeg image

Posted: Mon Oct 28, 2019 11:32 am
by stefano.setti
Thanks for the code, but in my enviroment doesn't work.
There is a problem on line:

Code: Select all

while ((buffer = mmal_queue_get(context.queue)) != NULL)
It never enter the loop.
I am using Raspberry Pi 3B and this Raspbian version:

Code: Select all

Linux raspberrypi 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux
I am tring to decode this jpg file:

Code: Select all

JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=0], progressive, precision 8, 1920x1082, frames 3
What I am doing wrong?

After I solve this problem I need to display the image on screen, someone have an example to do that?


Re: How to use mmal to decode jpeg image

Posted: Mon Oct 28, 2019 2:06 pm
by dom
stefano.setti wrote:
Mon Oct 28, 2019 11:32 am

Code: Select all

JPEG image data, Exif standard: [TIFF image data, little-endian, direntries=0], progressive, precision 8, 1920x1082, frames 3
What I am doing wrong?
Progressive jpegs are not handled by gpu. You'll need an arm side decoder for that.
From wikipedia
There is also an interlaced progressive JPEG format, in which data is compressed in multiple passes of progressively higher detail. This is ideal for large images that will be displayed while downloading over a slow connection, allowing a reasonable preview after receiving only a portion of the data. However, support for progressive JPEGs is not universal.

Re: How to use mmal to decode jpeg image

Posted: Mon Oct 28, 2019 3:53 pm
by stefano.setti
Thank you, with a non progressive jpeg it works.
Now I missing last step.
I am tring to mix this 2 examples: ... peg/jpeg.c
But it shows a grey square with black lines.
Can you help me to make my script works?

Code: Select all

Copyright (c) 2018 Raspberry Pi (Trading) Ltd

Decodes a JPEG image into a memory buffer.

#include <errno.h>
#include <fcntl.h>
#include <math.h>
#include <poll.h>
#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>

//#include <interface/vcsm/user-vcsm.h>
#include "bcm_host.h"
#include "interface/mmal/mmal.h"
#include "interface/mmal/util/mmal_default_components.h"
#include "interface/mmal/util/mmal_util_params.h"
#include "interface/mmal/util/mmal_util.h"
#include "interface/mmal/util/mmal_connection.h"
#include "interface/mmal/mmal_queue.h"
#include "interface/vcos/vcos.h"

#define MAX_BUFFERS 2

static inline int warn(const char *file, int line, const char *fmt, ...)
   int errsv = errno;
   va_list va;
   va_start(va, fmt);
   fprintf(stderr, "WARN(%s:%d): ", file, line);
   vfprintf(stderr, fmt, va);
   errno = errsv;
   return 1;

#define CHECK_CONDITION(cond, ...) \
do { \
   if (cond) { \
      int errsv = errno; \
      fprintf(stderr, "ERROR(%s:%d) : ", \
         __FILE__, __LINE__); \
      errno = errsv; \
      fprintf(stderr,  __VA_ARGS__); \
      abort(); \
   } \
} while(0)
#define WARN_ON(cond, ...) \
   ((cond) ? warn(__FILE__, __LINE__, __VA_ARGS__) : 0)
#define ERRSTR strerror(errno)
#define CHECK_STATUS(status, ...) WARN_ON(status != MMAL_SUCCESS, __VA_ARGS__); \
   if (status != MMAL_SUCCESS) goto error;

static FILE *source_file;

/* Macros abstracting the I/O, just to make the example code clearer */
#define SOURCE_OPEN(uri) \
   source_file = fopen(uri, "rb"); if (!source_file) goto error;
#define SOURCE_READ_CODEC_CONFIG_DATA(bytes, size) \
   size = fread(bytes, 1, size, source_file); rewind(source_file)
   a->length = fread(a->data, 1, a->alloc_size - 128, source_file); \
   a->offset = 0
#define SOURCE_CLOSE() \
   if (source_file) fclose(source_file)

/** Context for our application */
static struct CONTEXT_T {
   VCOS_SEMAPHORE_T semaphore;
   MMAL_QUEUE_T *queue;
   MMAL_STATUS_T status;
} context;

static void log_format(MMAL_ES_FORMAT_T *format, MMAL_PORT_T *port)
   const char *name_type;

      fprintf(stderr, "%s:%s:%i", port->component->name,
               port->type == MMAL_PORT_TYPE_CONTROL ? "ctr" :
                  port->type == MMAL_PORT_TYPE_INPUT ? "in" :
                  port->type == MMAL_PORT_TYPE_OUTPUT ? "out" : "invalid",

   case MMAL_ES_TYPE_AUDIO: name_type = "audio"; break;
   case MMAL_ES_TYPE_VIDEO: name_type = "video"; break;
   case MMAL_ES_TYPE_SUBPICTURE: name_type = "subpicture"; break;
   default: name_type = "unknown"; break;

   fprintf(stderr, "type: %s, fourcc: %4.4s\n", name_type, (char *)&format->encoding);
   fprintf(stderr, " bitrate: %i, framed: %i\n", format->bitrate,
            !!(format->flags & MMAL_ES_FORMAT_FLAG_FRAMED));
   fprintf(stderr, " extra data: %i, %p\n", format->extradata_size, format->extradata);
      fprintf(stderr, " samplerate: %i, channels: %i, bps: %i, block align: %i\n",
               format->es->audio.sample_rate, format->es->audio.channels,
               format->es->audio.bits_per_sample, format->es->audio.block_align);

      fprintf(stderr, " width: %i, height: %i, (%i,%i,%i,%i)\n",
               format->es->video.width, format->es->video.height,
               format->es->video.crop.x, format->es->video.crop.y,
               format->es->video.crop.width, format->es->video.crop.height);
      fprintf(stderr, " pixel aspect ratio: %i/%i, frame rate: %i/%i\n",
               format->es->video.par.num, format->es->video.par.den,
               format->es->video.frame_rate.num, format->es->video.frame_rate.den);


   default: break;


   fprintf(stderr, " buffers num: %i(opt %i, min %i), size: %i(opt %i, min: %i), align: %i\n",
            port->buffer_num, port->buffer_num_recommended, port->buffer_num_min,
            port->buffer_size, port->buffer_size_recommended, port->buffer_size_min,

/** Callback from the control port.
 * Component is sending us an event. */
static void control_callback(MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer)
   struct CONTEXT_T *ctx = (struct CONTEXT_T *)port->userdata;

   switch (buffer->cmd)
      /* Only sink component generate EOS events */
      /* Something went wrong. Signal this to the application */
      ctx->status = *(MMAL_STATUS_T *)buffer->data;

   /* Done with the event, recycle it */

   /* Kick the processing thread */

/** Callback from the input port.
 * Buffer has been consumed and is available to be used again. */
static void input_callback(MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer)
   struct CONTEXT_T *ctx = (struct CONTEXT_T *)port->userdata;

   /* The decoder is done with the data, just recycle the buffer header into its pool */

   /* Kick the processing thread */

/** Callback from the output port.
 * Buffer has been produced by the port and is available for processing. */
static void output_callback(MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer)
   struct CONTEXT_T *ctx = (struct CONTEXT_T *)port->userdata;

   /* Queue the decoded video frame */
   mmal_queue_put(ctx->queue, buffer);

   /* Kick the processing thread */

#define WIDTH  128
#define HEIGHT 128

typedef struct {

int print_supported_formats(MMAL_PORT_T *port)

   MMAL_SUPPORTED_ENCODINGS_T sup_encodings = {{MMAL_PARAMETER_SUPPORTED_ENCODINGS, sizeof(sup_encodings)}, {0}};
   ret = mmal_port_parameter_get(port, &sup_encodings.header);
   if (ret == MMAL_SUCCESS || ret == MMAL_ENOSPC)
      //Allow ENOSPC error and hope that the desired formats are in the first
      //MAX_ENCODINGS_NUM entries.
      int i;
      int num_encodings = (sup_encodings.header.size - sizeof(sup_encodings.header)) /
      if(num_encodings > MAX_ENCODINGS_NUM)
         num_encodings = MAX_ENCODINGS_NUM;
      for (i=0; i<num_encodings; i++)
         printf("%u: %4.4s\n", i, (char*)&sup_encodings.encodings[i]);
   return 0;

static void callback_vr_input(MMAL_PORT_T *port, MMAL_BUFFER_HEADER_T *buffer)

int main(int argc, char **argv)
    MMAL_COMPONENT_T *decoder = NULL;
    MMAL_POOL_T *pool_in = NULL, *pool_out = NULL;
    MMAL_BOOL_T eos_sent = MMAL_FALSE, eos_received = MMAL_FALSE;
    unsigned int in_count = 0, out_count = 0;

    MMAL_COMPONENT_T *render = NULL;
    MMAL_PORT_T *input;
    MMAL_POOL_T *pool;
    int i;

    mmal_component_create("", &render);
    input = render->input[0];


    input->format->encoding = ENCODING;
    input->format->es->video.width  = VCOS_ALIGN_UP(WIDTH,  32);
    input->format->es->video.height = VCOS_ALIGN_UP(HEIGHT, 16);
    input->format->es->video.crop.x = 0;
    input->format->es->video.crop.y = 0;
    input->format->es->video.crop.width  = WIDTH;
    input->format->es->video.crop.height = HEIGHT;


    mmal_port_parameter_set_boolean(input, MMAL_PARAMETER_ZERO_COPY, MMAL_TRUE);

    input->buffer_size = input->buffer_size_recommended;
    input->buffer_num = input->buffer_num_recommended;
    if (input->buffer_num < 2)
        input->buffer_num = 2;
    pool = mmal_port_pool_create(input, input->buffer_num, input->buffer_size);

    if (!pool) {
        printf("Oops, ,pool alloc failed\n");
        return -1;

        param.hdr.size = sizeof(MMAL_DISPLAYREGION_T);

        param.set = MMAL_DISPLAY_SET_LAYER;
        param.layer = 128;    //On top of most things

        param.set |= MMAL_DISPLAY_SET_ALPHA;
        param.alpha = 255;    //0 = transparent, 255 = opaque

        param.fullscreen = 0;
        param.dest_rect.x = 100;
        param.dest_rect.y = 200;
        param.dest_rect.width = WIDTH;
        param.dest_rect.height = HEIGHT;
        mmal_port_parameter_set(input, &param.hdr);

    mmal_port_enable(input, callback_vr_input);
    for (i=0; i<10; i++) {
        buffer = mmal_queue_wait(pool->queue);

        // Write something into the buffer.
        memset(buffer->data, (i<<4)&0xff, buffer->alloc_size);

        buffer->length = buffer->alloc_size;
        mmal_port_send_buffer(input, buffer);



    return 0;


    if (argc < 2)
        fprintf(stderr, "invalid arguments\n");
        return -1;



    vcos_semaphore_create(&context.semaphore, "example", 1);


    /* Create the decoder component.
        * This specific component exposes 2 ports (1 input and 1 output). Like most components
        * its expects the format of its input port to be set by the client in order for it to
        * know what kind of data it will be fed. */
    status = mmal_component_create(MMAL_COMPONENT_DEFAULT_IMAGE_DECODER, &decoder);
    CHECK_STATUS(status, "failed to create decoder");

    /* Enable control port so we can receive events from the component */
    decoder->control->userdata = (void *)&context;
    status = mmal_port_enable(decoder->control, control_callback);
    CHECK_STATUS(status, "failed to enable control port");

    /* Set the zero-copy parameter on the input port */
    //   status = mmal_port_parameter_set_boolean(decoder->input[0], MMAL_PARAMETER_ZERO_COPY, MMAL_TRUE);
    //   CHECK_STATUS(status, "failed to set zero copy - %s", decoder->input[0]->name);

    /* Set the zero-copy parameter on the output port */
    status = mmal_port_parameter_set_boolean(decoder->output[0], MMAL_PARAMETER_ZERO_COPY, MMAL_TRUE);
    CHECK_STATUS(status, "failed to set zero copy - %s", decoder->output[0]->name);

    /* Set format of video decoder input port */
    MMAL_ES_FORMAT_T *format_in = decoder->input[0]->format;
    format_in->type = MMAL_ES_TYPE_VIDEO;
    format_in->encoding = MMAL_ENCODING_JPEG;
    format_in->es->video.width = 0;
    format_in->es->video.height = 0;
    format_in->es->video.frame_rate.num = 0;
    format_in->es->video.frame_rate.den = 1;
    format_in->es->video.par.num = 1;
    format_in->es->video.par.den = 1;

    status = mmal_port_format_commit(decoder->input[0]);
    CHECK_STATUS(status, "failed to commit format");

    MMAL_ES_FORMAT_T *format_out = decoder->output[0]->format;
    format_out->encoding = MMAL_ENCODING_I420;

    status = mmal_port_format_commit(decoder->output[0]);
    CHECK_STATUS(status, "failed to commit format");

    /* Display the output port format */
    fprintf(stderr, "%s\n", decoder->output[0]->name);
    fprintf(stderr, " type: %i, fourcc: %4.4s\n", format_out->type, (char *)&format_out->encoding);
    fprintf(stderr, " bitrate: %i, framed: %i\n", format_out->bitrate,
            !!(format_out->flags & MMAL_ES_FORMAT_FLAG_FRAMED));
    fprintf(stderr, " extra data: %i, %p\n", format_out->extradata_size, format_out->extradata);
    fprintf(stderr, " width: %i, height: %i, (%i,%i,%i,%i)\n",
            format_out->es->video.width, format_out->es->video.height,
            format_out->es->video.crop.x, format_out->es->video.crop.y,
            format_out->es->video.crop.width, format_out->es->video.crop.height);

    /* The format of both ports is now set so we can get their buffer requirements and create
        * our buffer headers. We use the buffer pool API to create these. */
    decoder->input[0]->buffer_num = decoder->input[0]->buffer_num_recommended;
    decoder->input[0]->buffer_size = decoder->input[0]->buffer_size_recommended;
    decoder->output[0]->buffer_num = decoder->output[0]->buffer_num_recommended;
    decoder->output[0]->buffer_size = decoder->output[0]->buffer_size_recommended;
    pool_in = mmal_port_pool_create(decoder->input[0],

    /* Create a queue to store our decoded frame(s). The callback we will get when
        * a frame has been decoded will put the frame into this queue. */
    context.queue = mmal_queue_create();

    /* Store a reference to our context in each port (will be used during callbacks) */
    decoder->input[0]->userdata = (void *)&context;
    decoder->output[0]->userdata = (void *)&context;

    /* Enable all the input port and the output port.
        * The callback specified here is the function which will be called when the buffer header
        * we sent to the component has been processed. */
    status = mmal_port_enable(decoder->input[0], input_callback);
    CHECK_STATUS(status, "failed to enable input port");

    status = mmal_port_enable(decoder->output[0], output_callback);
    CHECK_STATUS(status, "failed to enable output port");

    pool_out = mmal_port_pool_create(decoder->output[0],

    while ((buffer = mmal_queue_get(pool_out->queue)) != NULL)
        printf("Sending buf %p\n", buffer);
        status = mmal_port_send_buffer(decoder->output[0], buffer);
        CHECK_STATUS(status, "failed to send output buffer to decoder");

    /* Component won't start processing data until it is enabled. */
    status = mmal_component_enable(decoder);
    CHECK_STATUS(status, "failed to enable decoder component");

    /* Start decoding */
    fprintf(stderr, "start decoding\n");

    /* This is the main processing loop */
    while(!eos_received && out_count < 10000)
        VCOS_STATUS_T vcos_status;
        /* Wait for buffer headers to be available on either of the decoder ports */
        vcos_status = vcos_semaphore_wait_timeout(&context.semaphore, 2000);
        if (vcos_status != VCOS_SUCCESS)
            fprintf(stderr, "vcos_semaphore_wait_timeout failed - status %d\n", vcos_status);

        /* Check for errors */
        if (context.status != MMAL_SUCCESS)

        /* Send data to decode to the input port of the video decoder */
        if (!eos_sent && (buffer = mmal_queue_get(pool_in->queue)) != NULL)
            if(!buffer->length) eos_sent = MMAL_TRUE;

            buffer->flags = buffer->length ? 0 : MMAL_BUFFER_HEADER_FLAG_EOS;
            buffer->pts = buffer->dts = MMAL_TIME_UNKNOWN;
            fprintf(stderr, "sending %i bytes\n", (int)buffer->length);
            status = mmal_port_send_buffer(decoder->input[0], buffer);
            CHECK_STATUS(status, "failed to send buffer");
            fprintf(stderr, "Input buffer %p to port %s. in_count %u\n", buffer, decoder->input[0]->name, in_count);

        /* Get our output frames */
        while ((buffer = mmal_queue_get(context.queue)) != NULL)
            fprintf(stderr, "asd!!!!!!\n");
            /* We have a frame, do something with it (why not display it for instance?).
            * Once we're done with it, we release it. It will automatically go back
            * to its original pool so it can be reused for a new video frame.
            eos_received = buffer->flags & MMAL_BUFFER_HEADER_FLAG_EOS;

            if (buffer->cmd)
                fprintf(stderr, "received event length %d, %4.4s\n", buffer->length, (char *)&buffer->cmd);
                if (buffer->cmd == MMAL_EVENT_FORMAT_CHANGED)
                MMAL_EVENT_FORMAT_CHANGED_T *event = mmal_event_format_changed_get(buffer);
                if (event)
                    fprintf(stderr, "----------Port format changed----------\n");
                    log_format(decoder->output[0]->format, decoder->output[0]);
                    fprintf(stderr, "-----------------to---------------------\n");
                    log_format(event->format, 0);
                    fprintf(stderr, " buffers num (opt %i, min %i), size (opt %i, min: %i)\n",
                            event->buffer_num_recommended, event->buffer_num_min,
                            event->buffer_size_recommended, event->buffer_size_min);
                    fprintf(stderr, "----------------------------------------\n");

                //Clear out the queue and release the buffers.
                while(mmal_queue_length(pool_out->queue) < pool_out->headers_num)
                    buffer = mmal_queue_wait(context.queue);
                    fprintf(stderr, "Retrieved buffer %p\n", buffer);

                //Assume we can't reuse the output buffers, so have to disable, destroy
                //pool, create new pool, enable port, feed in buffers.
                mmal_port_pool_destroy(decoder->output[0], pool_out);

                status = mmal_format_full_copy(decoder->output[0]->format, event->format);
                decoder->output[0]->format->encoding = MMAL_ENCODING_I420;
                decoder->output[0]->buffer_num = MAX_BUFFERS;
                decoder->output[0]->buffer_size = decoder->output[0]->buffer_size_recommended;

                if (status == MMAL_SUCCESS)
                    status = mmal_port_format_commit(decoder->output[0]);
                if (status != MMAL_SUCCESS)
                    fprintf(stderr, "commit failed on output - %d\n", status);

                mmal_port_enable(decoder->output[0], output_callback);
                pool_out = mmal_port_pool_create(decoder->output[0], decoder->output[0]->buffer_num, decoder->output[0]->buffer_size);
                fprintf(stderr, "decoded frame (flags %x, size %d) count %d\n", buffer->flags, buffer->length, out_count);

                // Do something here with the content of the buffer
                mmal_port_send_buffer(input, buffer);




        /* Send empty buffers to the output port of the decoder */
        while ((buffer = mmal_queue_get(pool_out->queue)) != NULL)
            printf("Sending buf %p\n", buffer);
            status = mmal_port_send_buffer(decoder->output[0], buffer);
            CHECK_STATUS(status, "failed to send output buffer to decoder");

    /* Stop decoding */
    fprintf(stderr, "stop decoding\n");

    /* Stop everything. Not strictly necessary since mmal_component_destroy()
        * will do that anyway */

    /* Cleanup everything */
    if (pool_in)
        mmal_port_pool_destroy(decoder->input[0], pool_in);
    if (pool_out)
        mmal_port_pool_destroy(decoder->output[0], pool_out);
    if (decoder)
    if (context.queue)

    return status == MMAL_SUCCESS ? 0 : -1;
I didn't find an example that do what I need: decode and image and render on screen without x11.

Re: How to use mmal to decode jpeg image

Posted: Wed Oct 30, 2019 8:56 pm
by dividuum
stefano.setti wrote:
Mon Oct 28, 2019 3:53 pm
I didn't find an example that do what I need: decode and image and render on screen without x11.
If that's your goal, what's wrong with the advice I gave you in another thread? ... 7&t=254969

Re: How to use mmal to decode jpeg image

Posted: Wed Nov 06, 2019 7:24 am
by stefano.setti
dividuum your code show the image, but I cannot manage to modify it to resize the image and position.
For example how can I show a FHD image in bottom right corner of the screen with 500x500px resolution and keep show the all other screen portion?