jkw
Posts: 12
Joined: Thu Oct 02, 2014 5:20 pm

GPU efficienty

Wed Nov 04, 2015 8:53 pm

Hi, I've create a nice little android + rpi + amazon EC2 project, in which multiple PIs are
connected to my server. Each PI can stream pictures from its camera to the server and eventually my phone.

Everything is running pretty ridged but I'm not too happy with the performance (ultimately I'm unhappy with the
incoming framerate). All my cameras are connected via WiFi and I live in a very crowded neighbourhood.

I've measured a few times the uplink from a typical camera and ended up with the numbers below:
Wireless station + dsl uplink + EC2 server = 0.8Mbps
Wired station + dsl uplink + EC2 server = 1.2Mbps which is ok, att provides "up to" 1.5Mbps
Wired station + local server = ~10Mbps, @100% CPU on the client, which is fine, so it's not a question of the server speed / architecture

So given that I can't change the wireless speed ( in fact I can, I've switched a unit to 5Ghz and had an enormous speed increase - but at that point I was pretty soon limited by AT&T's high speed upload of 1.5Mbps which is a joke but the fastest option they provide in tech-city Housten.... too bad I'm not living in Europe or at least in a Google Fiber town .. but ok .. )

So again, I have a link of ~0.8Mbps, lets keep that in mind.

My first generation of camera where USB webcams. I've written a small client that used the USB Cam + pygame + PIL for the time-stamp. The time-stamp is a MUST HAVE, otherwise the photo look really boring ..

I've measured the timing here:
~100ms to grab a frame from the USB camera.. ok so we are at 10fps now, pretty good
+50ms to a add the time-stamp .. well ~8fps .. all right
+250ms to encode it as JPG .. damn .. 2.5fps
For reference (we'll need that later) the photos are 1280x720,75% jpeg quality, ~71kB
compared to a 24bit BMP PIL reaches a compression ratio of 1280x720x3/71kB = 39/1 @75%


Ok, so I've read about the GPU and the build-in jpeg compression. I've modified the source code that I've found online, using the MMAL API of the VideoCore IV, connected a PiCam, started grabbing frames at 5Mpx, 30fps, jpeg compressed in "no time" ... WOW Really amazing, had to write to RAM to keep up with the amount of incoming data.

BUT, those files are really really big... .. ok lets scale them down a little bit, we don't need 5Mpx anyway.

Lets go for 1280x720, 75% jpeg quality.
Hmm filesize was around 440kB per picture, that is only 0.22fps ... not acceptable.
I've played around a little bit and right now I'm running at 512*288, 25% jpeg quality, ~44kB
800kbps / (44kB * 8) = [fps] -> gives me decent combination of 2fps@0.7Mbps
Looking at the compression ratio: 512*288*3/44kB = 10/1 @25%

That is all OK, but not too good.
The current GPU version is neat because the camera uses <5% and the remaining power can be spend on the Network handling, ect, but the usb software approach had way nicer pictures "highres" ;) but I've ended up with something around 1fps

Is anyone aware of a method to improve the jpeg compression of the gpu?
Given that PIL can compress an image more then 6x efficient (size-wise) I would love to see something like
1280x720,50% @ 50kB for higher quality or 512*288,25% @ 10kB for higher frame rates.
Any ideas?

Thanks

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 10566
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GPU efficienty

Wed Nov 04, 2015 11:40 pm

JPEG Q-factor is NOT a percentage, and there is no defined correct compression ratio for a given Q-factor.

This has already been discussed to death on the camera forum - viewtopic.php?f=43&t=73174 is the main thread. Make sure you've disabled EXIF and thumbnails too, as those bloat up the image quite considerably (often 60kB per image).

I had raised the mapping of Q-factor to quantisation settings with the JPEG codec engineer when back at Broadcom as we always tended to have to use smaller numbers. I never got him to actually look into it and compare Q-factors to libJPEG or similar. Not going to happen now.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jkw
Posts: 12
Joined: Thu Oct 02, 2014 5:20 pm

Re: GPU efficienty

Thu Nov 05, 2015 1:06 pm

WOW!
Thank you very very much, I've set the Q-Factor to 10 and the image shirked down to ~5kB.
After increasing the resolution to 1280x720 the size came up to 60-80kB, so pretty much the
same as PIL. Now I'm streaming at least 2-3 fps at 720p .. thats pretty much what I was looking for.

Amazing! So apparently I had the quality at/close to 100% all the time (Q-Factor 45/25).
The image looked kind of compressed, but I guess that was just due to the scaling from 512x288 to my full screen resolution.

"Problem" solved. Thanks

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 10566
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: GPU efficienty

Thu Nov 05, 2015 1:32 pm

You can use ImageMagik's "identify" function to extract the Quality factor that it considers the image to be at.

Code: Select all

sudo apt-get install imagemagick
identify -verbose <filename>
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “Graphics, sound and multimedia”