dpq
Posts: 9
Joined: Mon Nov 24, 2014 4:03 pm

OpenMAX and progressive JPEG

Mon Nov 24, 2014 4:09 pm

Is it possible to get progressive JPEG hardware-accelerated encoding to work on Raspberry Pi as of 2014? I.e.:
  • Is it possible to use existing hardware for this purpose by fixing the drivers?
  • Can anyone recommend another board [relatively cheap] onto which this task can be offloaded?
Thank you.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5398
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 5:04 pm

No support for progressive jpeg encode or decode on Pi.
I would be surprised if you can find another boards that supports hardware accelerated progressive jpeg encode/decode.

Why would you want to encode progressive jpegs?
They were useful in the days of dial up internet where people waited tens of seconds for a jpeg to download, but I'm struggling to see the value now.

dpq
Posts: 9
Joined: Mon Nov 24, 2014 4:03 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 5:34 pm

We're developing a mobile robotic system that does a lot of processing in the data center (i.e. the device is pretty stupid, and most of the CV/navigation/AI stuff is done "in the cloud", if you excuse me).
For that purpose, we need to produce a fixed frame rate video stream from the forward camera even in fluctuating network conditions to maintain a predictable response timing. Fixed frame rate can be achieved by varying the load, so in poor network conditions we only send 50kB frames, and in better network conditions we may send 80kB frames.

Using a baseline JPEG is a poor option, since a partial image may not be useful for the computer vision software at all. So whatever the image format, we need to use some kind of interlacing to drop the quality but still allow the CV to mostly do its job. On a side note, the video stream must not use interframe predictions to make the system more resistant to frame corruption and losses.

As it is, we are down to MJPEG stream with progressively encoded frames, a JPEG2000 stream (I believe it does some wavelet magic and it should theoretically be possible to similarly cut off the latter part of the image to achieve a result similar to ProgressiveJPEG) and WebP (which afaik also supports progressive encoding). Since we aim for relatively high video feed frame rates (10-20fps), non-accelerated encoding can hardly be used: our software currently spends about 400-500msec on encoding each frame.

Do you have any suggestions? Thank you.

plugwash
Forum Moderator
Forum Moderator
Posts: 3522
Joined: Wed Dec 28, 2011 11:45 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 6:23 pm

I wonder how CPU intensive converting from baseline jpeg to progressive jpeg is and in particular whether encoding as baseline jpeg in the GPU and then translating to progressive in software would be more or less CPU intensive than encoding to progressive in software.

dpq
Posts: 9
Joined: Mon Nov 24, 2014 4:03 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 7:09 pm

If I'm not mistaken, such a conversion is implemented as decoding baseline and re-encoding the data progressively. I'm not qualified enough to understand if it is viable to utilitize the GPU somehow, like crafting an image from every 16-th pixel and baseline-encoding it, then crafting an image from every 8-th pixel and baseline-encoding it and appending to the previous image, etc.

plugwash
Forum Moderator
Forum Moderator
Posts: 3522
Joined: Wed Dec 28, 2011 11:45 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 8:58 pm

It's possible to convert from baseline to progressive or vice-versa without complete decoding, jpegtran can do it. However the main reason people do it is to preseve quality, so I don't know how it compares performance wise.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24979
Joined: Sat Jul 30, 2011 7:41 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 9:41 pm

Would H264 be better? Although not sure you can vary the bitrate as you go along.

Also, with an overcloked Pi, can you do the encode in SW fast enough? Depends in the required resolution I expect.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I own the world’s worst thesaurus. Not only is it awful, it’s awful."

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5398
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 9:51 pm

dpq wrote: Do you have any suggestions? Thank you.
You can pass a quality factor into the jpeg encoder. That will vary the encoded jpeg size.
Have a feedback loop where slow network reduces next jpeg quality.

dpq
Posts: 9
Joined: Mon Nov 24, 2014 4:03 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 9:53 pm

We've tried jpegtran before and were unsatisfied with its performance. Meanwhile, it appears that we can work around this issue by simply downscaling the image in question with a bilinear filter using data on connection quality collected from sending the previous frame. If OpenMAX supports bilinear filtering/scaling [does it?], that is.

dpq
Posts: 9
Joined: Mon Nov 24, 2014 4:03 pm

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 9:55 pm

jamesh wrote:Would H264 be better? Although not sure you can vary the bitrate as you go along.
Nope, H264 wouldn't be better. We can't rely on protocols that use interframe predictions, since each and every frame must be independent for network resiliency purposes. And making bitrate variable would be problematic for us as well.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5398
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: OpenMAX and progressive JPEG

Mon Nov 24, 2014 10:29 pm

dpq wrote:We've tried jpegtran before and were unsatisfied with its performance. Meanwhile, it appears that we can work around this issue by simply downscaling the image in question with a bilinear filter using data on connection quality collected from sending the previous frame. If OpenMAX supports bilinear filtering/scaling [does it?], that is.
Yes, the resize component will do this. (Although changing the jpeg quality level seems more appropriate).

Return to “OpenMAX”