Camera Interface Specs

[Hardware_man]

I just looked at 2 camera modules yesterday that output SDI. If I'm correct, that is a simple cable adaptor to convert SDI to HDMI.

[Hardware_man]

Oops, I meant DVI, not SDI. I looked at 2 different camera modules that output DVI yesterday.

[jamesh]

I'd be interested to know how they program the camera up for different modes, AGC, AWB etc, since DVI is effectively a one way protocol. I expect you plug in and go and get whatever the designer wants, which means you are stuck with a relatively simple camera and probably fairly bad picture quality.

[Hardware_man]

Jamesh,

Most of these camera modules have a separate serial data bidirectional port. They usually support both RS-232 Voltage levels and also logic levels. They tend to use the Sony Visca data format for controlling the camera and getting data and acknowledgements back from the camera. Sony Visca is “almost standard" as each camera module has different features and not all camera modules support all Sony Visca commands.

Hardware_man

[jamesh]

The camera modules supported on the Raspi are either CSI-2 or CCP-2 with i2c communication. All camera phone modules use some combination of this.

[Hardware_man]

Getting back to the "universal" video input concept, the Pi would not have to control the camera in this situation.

[jamesh]

Getting back to the "universal" video input concept, the Pi would not have to control the camera in this situation.

Which means....the camera would be pretty rubbish - no way of setting any of the settings you would normally find on a camera. Completely automated, including the lens. CAF or nothing for example.

[Hardware_man]

Jamesh,

We have tested the Sony FCB-ix11. This is only a SD camera (480i, composite video out) and got excellent results using the factory default settings for everything. We only used the VISCA to initiate a self calibration after power up and to control the zoom lens.

We are moving up to a HD (720p) camera and need something small to record HD video. A lot of these camera modules also have a connector that you can connect a button matrix to (12 or 16 buttons and diodes) if you don’t want to use the VISCA interface.

We only need the Pi for the H.264 encoder and SD card. We don’t need the Pi to control the camera. The camera works fine without any need for additional control. If somebody wants the Pi to control one of these types of camera module, they can write simple code and use a serial port on the Pi, but this is not mandatory. The VISCA is well documented.

Hardware_man

[jamesh]

Much easier and cheaper to wait for the Foundation camera module ($25, 2 months), unless you need a decent lens, in which case you may need to look elsewhere, because the Foundation camera module is the only way you are going to get camera in for some time. Even if the work started now (it's not going to), I cannot see being able to get HDMI in (or any other protocol) working effectively within 6 months.

FYI, the camera module is working up to 1080p30, plus 5MP capture. Picture quality improved a lot today once we got a good ISP engineer working on it and we have determined places where work need to be done on the AWB. Talking to an Omnivision engineer visiting tomorrow. Still a ways to go though, 60 and 90fps modes not yet working. Currently writing a new Linux capture program so people have some example code to play with. Part time though, so things are slow.

[bernd71]

We only need the Pi for the H.264 encoder and SD card. We don’t need the Pi to control the camera. The camera works fine without any need for additional control. If somebody wants the Pi to control one of these types of camera module, they can write simple code and use a serial port on the Pi, but this is not mandatory. The VISCA is well documented.

Hardware_man

Isn't it easier to use an external h264 encoder connected via USB or a network camera with h264 support instead of looking for someone spending month in developing something like a HDMI interface?

[B1234]

Isn't it easier to use an external h264 encoder connected via USB or a network camera with h264 support instead of looking for someone spending month in developing something like a HDMI interface?

You'd think so, wouldn't you.

Just some people have an idea and won't abandon it as long as theres a tiny sliver of a chance they might succeed.

[Hardware_man]

For our application, the camera has to be very small, very light weight and have a remotely controllable analog / mechanical zoom lens. We also need both an SD composite video output and some type of HD video output. So far, the only camera modules that we have found that meet these requirements do not have an on board H.264 encoder. If we found one that met these requirements but didn’t also have a SD composite video output, I could probably do a small board to create SD composite from the HD output. But the SD composite has to be very low delay so converting after the encoder might be a problem. If it had no SD composite output but both non encoded HD and H.264 encoded HD outputs , then I could generate SD composite from the non encoded HD.

If you know of a camera module that meets these requirements and has an on board H.264 encoder, please let me know. Definitely make my life easier!

Hardware_man

[Hardware_man]

I’m shocked! I thought that men who have such vast intelligence, knowledge and wisdom as to be able to diagnose me as obsessive / compulsive for keeping this thread going would certainly be able to provide an “off the shelf” solution for my technical requirements. Yet no one has replied. Shocked, totally shocked!

Hardware_man

[jamesh]

I’m shocked! I thought that men who have such vast intelligence, knowledge and wisdom as to be able to diagnose me as obsessive / compulsive for keeping this thread going would certainly be able to provide an “off the shelf” solution for my technical requirements. Yet no one has replied. Shocked, totally shocked!

Hardware_man

There isn't an off the shelf solution. That might explain the lack of response. You can be the most intelligent person in the world, but there still won't be a solution.

[B1234]

I thought that men who have such vast intelligence, knowledge and wisdom as to be able to diagnose me as obsessive / compulsive for keeping this thread going would certainly be able to provide an “off the shelf” solution for my technical requirements.

We're smart enough to know you, being a hardware engineer, should have given up long ago and put your energy into making what you need from scratch, instead of trying to reuse something that isnt designed for the job you want.

Find a something to do the encoding with, find a few other components, throw them at a PCB, whatever sticks is what you need.

[Hardware_man]

Unfortunately, we are a very small start up company with a very limited development budget. After an extensive search, I’m finding most IC manufactures of H.264 encoders are as paranoid as Broadcom. If you don’t want to buy a gazillion chips, you can’t even get a data sheet. TI is the one exception with their TMS320DM365 / 368. But that is a $1,600 Eval board. If it weren’t for the foundation, none of us could even buy the Broadcom chip. That’s why I was hoping there would have been more interest for a video in on the Pi.

Hardware_man

[jamesh]

There is interest. Just not enough to pay the rather large dev costs of what you propose in a short timescale. I'm not even saying it will never be done. Just not right now. I bet there are quite a few thousands of people who would buy a board like this.

Actually, there is an alternative for your zoomable lens. You could use a very high resolution sensor and use digital zoom, as is done on the Nokia 808, the sensor from which is already supported by the GPU. But I doubt you could buy the sensors.

But as a complete aside, you are a small startup, presumably already in business, and yet you now find that you cannot get the bits you need? And $1600 is a pretty decent price for an Eval board - they do cost a lot to make. I'd try haggling to get the price down a bit. You might get some traction if they like your product and the sales prospects.

[Hardware_man]

It would have to be an extreemly high resolution CCD.

[Hardware_man]

Sorry, I posted before I had completed my thought. Right now, we are getting the performance we need with a 720 x 1280 CCD (approximately 1 Mega Pixel) and a 15:1 analog zoom lens. If I understand this correctly, I would need a 15 Mega Pixel CCD to get the same performance doing strictly digital zoom. This is doable, but would certainly be a major project as I would be getting into resolutions beyond the commercially available specifications.

Hardware_man

[jamesh]

Sorry, I posted before I had completed my thought. Right now, we are getting the performance we need with a 720 x 1280 CCD (approximately 1 Mega Pixel) and a 15:1 analog zoom lens. If I understand this correctly, I would need a 15 Mega Pixel CCD to get the same performance doing strictly digital zoom. This is doable, but would certainly be a major project as I would be getting into resolutions beyond the commercially available specifications.

Hardware_man

The Nokia 808 uses a 41MP sensor, made by Toshiba, supported by the GPU. Since the 808 is the last of the Symbian phones it isn't selling in huge numbers - maybe Toshiba have some spare. Would require some legal work I reckon to get it on the PI, plus a PCB.

Although I think your maths is wrong if you are meaning a 15x zoom. 41 would not be enough. I think we get 4x lossless zoom on the 808, although that is for an 8MP result.

[B1234]

Perhaps there is another way to achieve your goal?

Does it have to be real time video processing via a controller?

Would almost real time work, such that you have a camera that streams to something, and the Pi transcodes it into H.264 or just outputs it via HDMI?

[Hardware_man]

WOW! That must have had some “serious” data throughput.

Assuming 16 bits per pixel at 30 fps, that’s 41E6 x 16 x 30 = 19.7 giga bits per second! Can CSI-2 handle that? Did you need all 4 LVDS data lanes (you only bring 2 LVDS data lanes out to the CSI-2 connector on the Pi)?

Hardware_man

[Hardware_man]

B1234,

So your suggesting I use the Ethernet input and record to the Pi SD card that way? But I would need a camera that already does some compression to get the data rates down to the Ethernet speed.

Hardware_man

[jamesh]

WOW! That must have had some “serious” data throughput.

Assuming 16 bits per pixel at 30 fps, that’s 41E6 x 16 x 30 = 19.7 giga bits per second! Can CSI-2 handle that? Did you need all 4 LVDS data lanes (you only bring 2 LVDS data lanes out to the CSI-2 connector on the Pi)?

Hardware_man

Yes, it was quite something to get that going. Although its 10bpp for the Bayer capture I think, and of course when zoomed in you are not using the entire sensor - just the zoomed in bit (which is bigger than 1080 of course - you have to use the next up camera mode). 2 data lanes. The Videocore GPU is the only mobile phone SoC in the world capable of handling a sensor that large.

Still a lot of data whizzing about. We had to turn of things like SD ram calibration (RAM needs calibration for temperature) during video record as it could reduce the available memory bandwidth enough to cause transfer problems. At these rates 90ns is quite important.

[Hardware_man]

JamesH,

I thought about what B1234 said again. If I use 720P mode in my camera (720 x 1280) at 30 FPS, I just make it at USB2.0 speed

720 x 1280 x 16 x 30 = 442.4 M bps. And USB2.0 is specified for 480 M bps maximum. Maybe just enough overhead left for Vsyns, H sync, Frame, sync, etc. As the Pi is set up now, with simple high level code commands, maybe in OpenMAX, can I “route” the USB to the input of the H.264 encoder?

Hardware_man