wibble82
Posts: 66
Joined: Sun Jan 27, 2013 5:06 pm
Contact: Website

Re: A c++ camera api

Fri Nov 01, 2013 3:55 pm

peepo wrote:Chris,

thanks for the keyboard input details,
I usually connect to RPi over ethernet
and hence save output as .h264 to file or stream using rtsp,
is this possible currently?
my raspivid based attempts failed. eg -o is not accepted
iirc this is simply sending data to stdout,
however as I have no idea about GPU....

Of course, I appreciate the intention of providing an API
but the explanation can be even more useful,
in the sense that one can develop a true understanding.

in my cursory run ~7fps
cpu was not running anything else, beyond system
seems slow, why would this be?

very much looking forward to your further blog reports

thanks again
Hi

Sorry - just to clarify, I wasn't sure if you were looking at the gpu camera demo (http://robotblogging.blogspot.co.uk/201 ... ng-on.html) or the original api demo (http://robotblogging.blogspot.co.uk/201 ... i-for.html). The controls I mentioned were for the gpu demo - the simple api one has no controls and just shows most minimal use of the api.

If you want the details of how it works internally, it is documented over the 6 'pi eye' posts beforehand. However it's a complex system so there's no simple explanation. Basically access to the camera is via the multimedia adaption layer (mmal), which is a fairly low level interface to the OMX layer, which is the core of the raspberry pi's multi media system. It is fairly well explained across those posts though, and I made an effort to comment all the mmal code in camera.cpp which should help. Sadly no simple explanation for a complex system though.

It doesn't stream to h264 - the purpose of it was to output to memory so it could be used in-app. I'm afraid you'll need a tv to see the results! I too run over ethernet, but I also have the pi plugged into the tv so I can work on my pc but see the results on screen. The difficulty with h264 is that you'd have to pump the output back into mmal, and through an h264 ecoder which would be very troublesome indeed! There is example code in there somewhere to save textures to a png file, but it's very slow so wouldn't work in real time.

I could believe the gpu demo would see about 7fps, as its doing all 16 images at once! If you hit 'd' to toggle the actual image processing demo you should see a much more appreciable frame rate. similarly, if you use it for your own project and aren't doing 16 filters per frame it should run at a good rate. The api demo gives me 15fps at 720p (as it ships), and hits 30fps at half res. Next version is coming along soon though which will run much faster as it is entirely gpu based.

wibble82
Posts: 66
Joined: Sun Jan 27, 2013 5:06 pm
Contact: Website

Re: A c++ camera api

Fri Nov 01, 2013 4:03 pm

btw - I've been running the app without the full xwindows gui running - I don't know what the consequences of having the whole gui system in the background would be. Could be a cause of slow down though.

User avatar
jbeale
Posts: 3581
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: A c++ camera api

Fri Nov 01, 2013 7:35 pm

My 'picam' reported fps framerates, without X running (with X running, it is very nearly the same.) This is after waiting for the value to stabilize (seems to have a long-time constant filter on it)
7 at startup
12 after 'd' (fullscreen demo mode)

framerates drift +/- 1 fps even with nothing changing (that I know of)

('d' toggles feature-detect mode on/off, 'a' steps through each filter as fullscreen display, then 's' to see 16-up display)

Above numbers with no overclock in /boot/config.txt but if I set arm_freq=800 in the /boot/config.txt file then I get: 8 at startup, 14.8 after 'd'
Those values remain the samve with arm_freq=900
Last edited by jbeale on Fri Nov 01, 2013 8:01 pm, edited 1 time in total.

wibble82
Posts: 66
Joined: Sun Jan 27, 2013 5:06 pm
Contact: Website

Re: A c++ camera api

Fri Nov 01, 2013 7:49 pm

Interesting - I'm not over clocked my side. Oh well - I'll finish off the full gpu version next week and we'll see where that gets us.

rmsalinas
Posts: 12
Joined: Tue Sep 03, 2013 3:44 pm

Re: A c++ camera api

Fri Nov 01, 2013 8:23 pm

Hi everyone,

I have developed a library for fully controlling the camera in C++. You might be interested. Take a look at :

http://www.uco.es/investiga/grupos/ava/node/40

Cheers

wibble82
Posts: 66
Joined: Sun Jan 27, 2013 5:06 pm
Contact: Website

Re: A c++ camera api

Sat Nov 02, 2013 12:19 am

rmsalinas wrote:Hi everyone,

I have developed a library for fully controlling the camera in C++. You might be interested. Take a look at :

http://www.uco.es/investiga/grupos/ava/node/40

Cheers
Yeah I saw that one - i liked especially how it picks up the existence of opencv and creates the extra lib for it - I'm no master of make files so wasn't sure of the best approach to do that.

I would check out the image resize component in mmal or omx to do your image space conversion gpu side!

User avatar
peepo
Posts: 306
Joined: Sun Oct 21, 2012 9:36 am

Re: A c++ camera api

Sat Nov 02, 2013 3:58 pm

Chris,

okay, worked through pi_eyed 1-5 and feeling like a break.
a lot to take in...

1 & 4 dont build, I think worth having...
also the simple egl rect stuff would be good, though maybe one can source this stuff elsewhere easily enough?

It would be really nice to have an idea where you are taking this api, ie roadmap or mission statement?
eg things I like include:
output to file or stream, even at lower quality,
luminosity is sufficient for many concept-projects, rgb a luxury
of course there are many others,

point being to enable easy comprehension so others can fiddle.
graphics.cpp holds the openGL guff, could it be annotated?
iiuc this is the GPU code so worth understanding better?

thanks so much once again.

~:"

wibble82
Posts: 66
Joined: Sun Jan 27, 2013 5:06 pm
Contact: Website

Re: A c++ camera api

Sat Nov 02, 2013 6:14 pm

Hey

I'm going to release a second attempt next week. I've simplified the api slightly, and internally rewritten it to go direct to the omx layer. I'll make sure the graphics stuff I ship with it is better commented.

It will supply rgba by default and provide various facilities for converting to grey scale form (rgba, 32 bit float and interlaced), down sampling and processing.

My key reason for doing all this is that I'm building a robot with stereo vision, but I wanted to make my work useful to others. I guess my key objective is to get data out of the camera and run through various image processing systems (gpu and opencv) in real time, but I'd like anyone else doing something similar to benefit.

In terms of re-encoding, it's not on my list right now. My objective is to get the data out of the camera in its raw form. Perhaps I'll look into it at some point , but my focus is getting the input and analysing it.

Hope you like what it is us far - as I said, should have something much better next week! :)

User avatar
jbeale
Posts: 3581
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: A c++ camera api

Tue Nov 05, 2013 9:58 pm

wibble82 wrote:I'm going to release a second attempt next week. I've simplified the api slightly, and internally rewritten it to go direct to the omx layer. I'll make sure the graphics stuff I ship with it is better commented.
Looking forward to it!

arcanon
Posts: 36
Joined: Wed Nov 14, 2012 9:18 am

Re: A c++ camera api

Sun Nov 10, 2013 10:32 am

So I played around with this this weekend
stereo.jpg
stereo camera
stereo.jpg (63.94 KiB) Viewed 4416 times
Here is the youtube result in anaglyph stereo http://www.youtube.com/watch?v=j3TuHyuE4ew

Details here http://subgroup-ash.blogspot.de/2013/11 ... -so-i.html

In short I streamed the 2 cameras to a PC where I decoded using the GPU and composed the anaglyph stereo with a simple CUDA kernel. Its a bit over kill for just this simple application, but I can then use other features of OpenCV, as the above aproach will be. The nice thing is then you can use it as a live stereo camera :). The stereo angle needs to be better done, but my eyes were quite forgiving.

bazis
Posts: 11
Joined: Wed Dec 12, 2012 9:12 pm

Re: A c++ camera api

Tue Sep 01, 2015 8:40 pm

Hi,
In case anyone is interested, we've published a tutorial showing how to use raspicam with a cross-compiler: http://visualgdb.com/tutorials/raspberry/camera/

Return to “Camera board”