jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Sat Dec 01, 2012 2:56 pm

Snesors have two interfaces - CSI2 or CCP2 out for image data, and I2C in for control. Both interfaces are connected direcly to the GPU. I don't think you can get at the i2c from the Arm, and even if you could it would conflict with the i2c access that is required from the GPU. All the I2C gives you anyway if control of the sensor itself, it has nothing to do with all the post processing of the image data that is done by the GPU.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

edzieba
Posts: 25
Joined: Fri Jul 29, 2011 6:59 pm

Re: Camera module! (And a picture of JamesH)

Sat Dec 01, 2012 3:20 pm

I was thinking more long the lines of disconnecting the camera module entirely from the MIPI interface, and just hooking up the I2C header (or directly to the pins with some fine wire, if the camera board doesn't have am appropriate header) directly to the Pi's on-board I2C.
post processing of the image data that is done by the GPU
Ah, I'd forgotten about that.

The ability, even with a bit of fiddling, to get raw output would be valuable both for swapping lenses and for things like removing the IR filter (skipping the de-Bayer filter to reduce blurring, treaking each sub-pixel as a monochrome pixel), mainly for on-board processing of the image.

Paul Jurczak
Posts: 30
Joined: Wed Feb 22, 2012 11:46 pm

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 12:05 am

edzieba wrote:Point. Would it be easier (or even possible) to have anyone who wants more control fiddle with lens compensation and the like directly (by hooking the camera module up via I2C and flipping bits in the register) without any modifications to the driver? Can you even do that via I2C?
I2C is already available on S5 camera connector (pins 13, 14), see: http://elinux.org/File:Raspi-Model-AB-M ... 9x1024.png. So if you can write to SDA0/SCL0 I2C bus, you should be able to write to camera registers and turn ISP functions off. Google will find you OV5647 datasheet with control registers description.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 9:07 am

I think that bus is only accessible from the GPU, but could be wrong - but if you can write it the problem is you now have the GPU and the Arm writing sensor registers which will either confuse the sensor or confuse the GPU, as the sensor is now not doing what it expects. Note the GPU needs to communicate with the camera all the time to set it up for the right mode, streaming etc.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

pygmy_giant
Posts: 1562
Joined: Sun Mar 04, 2012 12:49 am

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 10:09 am

Will the camera module work with the model A?

Paul Jurczak
Posts: 30
Joined: Wed Feb 22, 2012 11:46 pm

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 10:27 am

jamesh wrote:I think that bus is only accessible from the GPU, but could be wrong - but if you can write it the problem is you now have the GPU and the Arm writing sensor registers which will either confuse the sensor or confuse the GPU, as the sensor is now not doing what it expects. Note the GPU needs to communicate with the camera all the time to set it up for the right mode, streaming etc.
Once you start streaming video, in most cases, you don't need to write to image sensor. This could be the time for user application to inject it's own I2C traffic. Of course, garbage-in-garbage-out rule applies.

What was asked several times in this thread is to get an API call to disable ISP functions on image sensor. This would be preferable over writing directly to ISP control registers, but acceptable, if former is not available. Function of registers 0x5000 through 0x5005 is orthogonal to image control settings (width, height, frame rate, region of interest, etc.), so they could be used without catastrophic interference with camera control code running on GPU.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 2:31 pm

Paul Jurczak wrote:
jamesh wrote:I think that bus is only accessible from the GPU, but could be wrong - but if you can write it the problem is you now have the GPU and the Arm writing sensor registers which will either confuse the sensor or confuse the GPU, as the sensor is now not doing what it expects. Note the GPU needs to communicate with the camera all the time to set it up for the right mode, streaming etc.
Once you start streaming video, in most cases, you don't need to write to image sensor. This could be the time for user application to inject it's own I2C traffic. Of course, garbage-in-garbage-out rule applies.

What was asked several times in this thread is to get an API call to disable ISP functions on image sensor. This would be preferable over writing directly to ISP control registers, but acceptable, if former is not available. Function of registers 0x5000 through 0x5005 is orthogonal to image control settings (width, height, frame rate, region of interest, etc.), so they could be used without catastrophic interference with camera control code running on GPU.
Once streaming you should be OK, but the GPU needs to change mode, stream on/off etc when capturing images (vs viewfinder) and therefore you need to be very careful (if indeed the Arm and GPU can both access the I2C at the same time).

I aware of what people have been asking for, and I have stated that some of the requests would require quite a bit or work (I work with the camera team at Broadcom. and worked on the Nokia 808 so do have some experience here!), and time is in very short supply I am afraid. Note that there is disabling stuff on the sensor, but also disabling various stages of the GPU ISP, which are different. There is also the question of what Broadcom will allow to be exposed - for example, the tuner files for the sensor are all text files, so could be passed to the GPU to give extremely fine control over the ISP, but 1) That might give away some secrets 2) Is extremely difficult to use without lots of experience.

I'm sure once the basic camera is working and released, we will have time to add extra features, but don't expect everything all at once, that's simply not going to happen. As a minimum we will be releasing the camera with software to take captures and record video at different resolutions, bit rates, fps etc. There will be some ability to set various parameters (e.g. ISO, white balance, AE etc. Similar set of features to the 808 for example), but being able to change the ISP parameters is unlikely at first.

Of course, I have assumed that this sensor uses the GPU ISP, and doesn't have its own internal ISP and therefore bypasses the GPU stages - I haven't checked, but I think that its the former!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5242
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 5:02 pm

pygmy_giant wrote:Will the camera module work with the model A?
Yes.

Paul Jurczak
Posts: 30
Joined: Wed Feb 22, 2012 11:46 pm

Re: Camera module! (And a picture of JamesH)

Sun Dec 02, 2012 9:20 pm

jamesh wrote:......
Of course, I have assumed that this sensor uses the GPU ISP, and doesn't have its own internal ISP and therefore bypasses the GPU stages - I haven't checked, but I think that its the former!
If OV5647 is the sensor you are using, then it does have its own ISP, which can perform auto exposure, auto gain, auto white balance, lens correction, defective pixel cancellation, post binning processing, etc.

I understand time and other constraints you have to work with. I'm just playing a role of "squeaky wheel" here, with hope of getting some of these things done in the future :-)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 10:09 am

Paul Jurczak wrote:
jamesh wrote:......
Of course, I have assumed that this sensor uses the GPU ISP, and doesn't have its own internal ISP and therefore bypasses the GPU stages - I haven't checked, but I think that its the former!
If OV5647 is the sensor you are using, then it does have its own ISP, which can perform auto exposure, auto gain, auto white balance, lens correction, defective pixel cancellation, post binning processing, etc.

I understand time and other constraints you have to work with. I'm just playing a role of "squeaky wheel" here, with hope of getting some of these things done in the future :-)
If that's the case then it should be fairly easy to plumb in any missing settings in from Arm space via OpenMAX. I've recently been doing something similar with another sensor and Android on the same GPU. I'll check the OV5647 driver and see what it implements. Note that external ISP (ie in the sensor) is fairly new code in the GPU source tree, so may be missing some features.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 10:57 am

Basically I would be happy to get a working camera even without these fine tunings. In my opinion those can be done in later upgrades.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 1:51 pm

shuckle wrote:Basically I would be happy to get a working camera even without these fine tunings. In my opinion those can be done in later upgrades.
That's what gonna happen! Simply isn't the time to put this more esoteric stuff in that most people won't use.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 6:07 pm

Since things are quite far already, could you publish the example code for reading the camera? It's something we all want to see anyway. :P

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5242
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 7:37 pm

Some video from the camera board, and a demo of streaming video over network:
http://www.raspberrypi.org/archives/2629

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 8:06 pm

cheery wrote:Since things are quite far already, could you publish the example code for reading the camera? It's something we all want to see anyway. :P
Dom is the guy who has been working on the Arm side and getting all that working - hopefully he may be able to share!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5242
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 8:57 pm

cheery wrote:Since things are quite far already, could you publish the example code for reading the camera? It's something we all want to see anyway. :P
We will release source code for everything that runs on the arm. (I need to check if that's covered by existing SLA, or if we need to rubber stamp some more files).
We are using mmal for this (an alternative to openmax, that's a little easier to use).
Have a look here for the mmal API:
https://github.com/raspberrypi/userland ... rface/mmal

arm2
Posts: 253
Joined: Thu Dec 15, 2011 3:46 pm

Re: Camera module! (And a picture of JamesH)

Mon Dec 03, 2012 9:40 pm

Dom: In the Video on the front page you say 'a couple of seconds delay' How come so long?
Where is all the data for that two seconds?
I realise that at certain stages there is one possibly two frame delay
I think Path is something like: CCD grab: Encode: Stream: Decode : Display
Say 50 FPS How can each stage take more than one 1/50 second, as it has to start the next frame after 1/50 second. Bank swapping at the encode and or decoder stage could add another frame delay each so I can get to a seven frame delay, but nearly 100! How?

Paul Jurczak
Posts: 30
Joined: Wed Feb 22, 2012 11:46 pm

Re: Camera module! (And a picture of JamesH)

Tue Dec 04, 2012 1:13 am

dom wrote:.......
We are using mmal for this (an alternative to openmax, that's a little easier to use).
Have a look here for the mmal API:
https://github.com/raspberrypi/userland ... rface/mmal
Is Multi-Media Abstraction Layer (MMAL) just Broadcom's API or is it a vendor neutral standard in making?

User avatar
cheery
Posts: 219
Joined: Wed Jan 25, 2012 9:39 pm

Re: Camera module! (And a picture of JamesH)

Tue Dec 04, 2012 1:39 am

Apparent from the example in the comments of that header you've got way to get full frames at least and maybe encode/decode them. That's good.

If you can bind it to a texture, that'd be nice bonus. Sure it's not much copying to do it yourself or big thing. But if these things happen in the GPU and firmware, why download and upload every frame between two different address spaces? Anyway it's something I wondered with the videos as well once I saw the OMX video decoding example that sits in the /opt directory.

One minor thing. Please use 'continue' only if it makes sense to do so. See this part of the example?

Code: Select all

while ((buffer = mmal_queue_get(queue_decoded_frames)) != NULL)
{
    if (buffer->cmd)
    {
        // event
        mmal_buffer_header_release(buffer);
        continue;
    }
    // frame
    mmal_buffer_header_release(buffer);
}
It could've been written:

Code: Select all

while ((buffer = mmal_queue_get(queue_decoded_frames)) != NULL)
{
    if (buffer->cmd)
    {
        // event
    } else {
        // frame
    }
    mmal_buffer_header_release(buffer);
}
Bit easier to read and recognize what is going on.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5242
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Camera module! (And a picture of JamesH)

Tue Dec 04, 2012 5:31 pm

arm2 wrote:Dom: In the Video on the front page you say 'a couple of seconds delay' How come so long?
Where is all the data for that two seconds?
I realise that at certain stages there is one possibly two frame delay
I think Path is something like: CCD grab: Encode: Stream: Decode : Display
Say 50 FPS How can each stage take more than one 1/50 second, as it has to start the next frame after 1/50 second. Bank swapping at the encode and or decoder stage could add another frame delay each so I can get to a seven frame delay, but nearly 100! How?
In mplayer. I don't know it's buffering logic, but when playing from a network stream, it waits until it has a couple of seconds in its cache.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5242
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Camera module! (And a picture of JamesH)

Tue Dec 04, 2012 5:34 pm

Paul Jurczak wrote:Is Multi-Media Abstraction Layer (MMAL) just Broadcom's API or is it a vendor neutral standard in making?
It's a Broadcom API. OpenMAX will continue to be supported, but there's less complexity with MMAL.

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Camera module! (And a picture of JamesH)

Wed Dec 05, 2012 7:14 am

What is actually the viewing angle of this camera module?
Chaning the lens seems to be next to impossible, so we are stuck to what we get, but what do we get?
70 degrees? 130 degress?

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 22044
Joined: Sat Jul 30, 2011 7:41 pm

Re: Camera module! (And a picture of JamesH)

Wed Dec 05, 2012 9:41 am

shuckle wrote:What is actually the viewing angle of this camera module?
Chaning the lens seems to be next to impossible, so we are stuck to what we get, but what do we get?
70 degrees? 130 degress?
Check here. http://www.raspberrypi.org/phpBB3/viewt ... 25#p178813
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

shuckle
Posts: 565
Joined: Sun Aug 26, 2012 11:49 am
Location: Finland

Re: Camera module! (And a picture of JamesH)

Wed Dec 05, 2012 10:46 am

Hmm. I probably do not understand this correctly, but the only angle I see is from the Omnivision pdf file: lens chief ray angle: 24°
I hope I got that wrong as this is pretty narrow in my opinion.
I would prefer something like 120 degrees: http://www.alibaba.com/product-gs/51627 ... 1080P.html

Well, I will see it when I can buy it :)

Ravenous
Posts: 1956
Joined: Fri Feb 24, 2012 1:01 pm
Location: UK

Re: Camera module! (And a picture of JamesH)

Wed Dec 05, 2012 10:49 am

That's probably the half angle - centre ray to edge. So the full diagonal angle would be 48 degrees. (Maybe.)

Return to “Camera board”