CSI-2 and DSI Connectors?


21 posts
by brianm734 » Sat Jul 28, 2012 12:43 am
When will the camera be available for the Pi? I would like to buy 2.
What is the pinout of the CSI-2 and DSI connectors?
Where can I get drivers for these connectors?
What LCD's work with the DSI connection?
Does the DSI connector support touch-screen LCD's?
Thank you.
Posts: 3
Joined: Fri Jul 27, 2012 10:30 pm
Location: Ann Arbor, MI USA
by jbeale » Sat Jul 28, 2012 1:20 am
The pinout is shown in the schematic, but it probably won't help you. As of now, there is nothing that functions when connected to the CSI or DSI connectors. A camera is coming, but date not yet announced. A LCD display has not yet been shown or announced. You cannot program it yourself because it is controlled by the GPU, to which there is no direct user access.
See also:
http://www.raspberrypi.org/wp-content/u ... s-R1.0.pdf
http://elinux.org/RPi_Low-level_peripherals#MIPI_CSI-2
http://elinux.org/RPi_Low-level_peripherals#DSI
User avatar
Posts: 2120
Joined: Tue Nov 22, 2011 11:51 pm
by brianm734 » Sat Jul 28, 2012 5:30 pm
I was looking for a response from someone involved in the design of the device.

Yes, I am aware there is a camera in the works, I have seen it.
The pins are all functional on both connectors.
I don't need to program the GPU. I can change its behaviour through the CPU from software or through the JTAG connector. There is no need to "program" anything - the pins are already assigned, all I need are drivers for the Linux distro I use.
Posts: 3
Joined: Fri Jul 27, 2012 10:30 pm
Location: Ann Arbor, MI USA
by jbeale » Sat Jul 28, 2012 9:21 pm
brianm734 wrote:The pins are all functional on both connectors.
I don't need to program the GPU. I can change its behaviour through the CPU from software or through the JTAG connector. There is no need to "program" anything - the pins are already assigned, all I need are drivers for the Linux distro I use.

If that is true, I look forward to official confirmation of this, as it is contrary to my understanding of what has been posted here by Broadcom folks, for example jamesh viewtopic.php?f=64&t=11385#p124506
and Gert viewtopic.php?f=7&t=5493#p73386
User avatar
Posts: 2120
Joined: Tue Nov 22, 2011 11:51 pm
by brianm734 » Sat Jul 28, 2012 11:23 pm
I don't see anything in those posts that contradicts my statements. The "GPU binary blob" is all software, controlled by the kernal drivers. The GPU is programmed and controlled through the CPU through the drivers you install.

No hardware modification is required to the connectors.
Posts: 3
Joined: Fri Jul 27, 2012 10:30 pm
Location: Ann Arbor, MI USA
by mahjongg » Sat Jul 28, 2012 11:54 pm
The camera when it arrives will come with GPU drivers specifically for it, and only it. It is very unlikely the CSI interface used will be able to support two camera's, maybe with a "CSI switch" if such a thing can be made, but not for two camera's at the same time.

There won't be a LCD panel released, but a DSI to LVDS converter, which can be used to connect suitable LCD panels, when it comes it will come with GPU drivers specifically for it, and the LCD panels it supports.
User avatar
Forum Moderator
Forum Moderator
Posts: 6010
Joined: Sun Mar 11, 2012 12:19 am
by JeremyF » Sat Jul 28, 2012 11:59 pm
mahjongg wrote:The camera when it arrives will come with GPU drivers specifically for it, and only it. It is very unlikely the CSI interface used will be able to support two camera's, maybe with a "CSI switch" if such a thing can be made, but not for two camera's at the same time.


I asked Gert about this at some point, he gave an answer, let me find it...

EDIT: found it viewtopic.php?f=61&t=6433&p=83701&hilit=videocore+iv+bandwith+cameras#p83011
{sig} Setup: Original version Raspberry Pi (B, rev1, 256MB), Dell 2001FP monitor (1600x1200), 8GB Class 4 SD Card with Raspbian and XBMC, DD-WRT wireless bridge
Posts: 516
Joined: Wed Jan 25, 2012 9:06 pm
by mahjongg » Sun Jul 29, 2012 12:05 am
Thanks for this clear-up jeremy
User avatar
Forum Moderator
Forum Moderator
Posts: 6010
Joined: Sun Mar 11, 2012 12:19 am
by AndrewS » Sun Jul 29, 2012 8:40 am
brianm734 wrote:I don't see anything in those posts that contradicts my statements. The "GPU binary blob" is all software, controlled by the kernal drivers. The GPU is programmed and controlled through the CPU through the drivers you install.

See http://elinux.org/RPi_BCM2835_GPIOs and http://elinux.org/RPi_BCM2835_Pinout in conjunction with the other links already posted earlier. AIUI the CPU core doesn't have direct access to the SoC pins, it has to access them via the GPU core - and the GPU core determines which 'mode' the SoC pins are set up in. So in theory if all the CSI and DSI pins were exposed as GPIO lines, you could 'bit-bang' the CSI and DSI protocols over GPIO, but I believe this would be *far* too slow to work (so would need a GPU-side driver). But not all the relevant pins are exposed as GPIO anyway, so even at the very lowest level there's no way (until support gets added to the GPU firmware blob start.elf) to access these pins from the CPU side.
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by AndrewS » Mon Jul 30, 2012 3:23 am
By way of comparison, this article briefly mentions the difference between using bit-banging and using the 'proper' GPU peripheral driver http://hackaday.com/2012/07/27/gaining- ... pberry-pi/
User avatar
Posts: 3626
Joined: Sun Apr 22, 2012 4:50 pm
Location: Cambridge, UK
by adhdengineer » Mon Jul 30, 2012 2:15 pm
Will it be possible to produce your own camera module and use the CSI port to stream raw video data?

If not why not.
Posts: 5
Joined: Mon Jul 30, 2012 2:13 pm
by mahjongg » Tue Jul 31, 2012 12:43 am
adhdengineer wrote:Will it be possible to produce your own camera module and use the CSI port to stream raw video data?

If not why not.

No,

because it will need very close cooperation with the GPU software, and unless your camera mudule is 110% the same as the foundations, the the GPU software will hiccup and die on your module.
User avatar
Forum Moderator
Forum Moderator
Posts: 6010
Joined: Sun Mar 11, 2012 12:19 am
by adhdengineer » Tue Jul 31, 2012 7:52 am
mahjongg wrote:No,

because it will need very close cooperation with the GPU software, and unless your camera mudule is 110% the same as the foundations, the the GPU software will hiccup and die on your module.


surely it doesnt need that much cooperation with the GPU. the CSI interface is a fairly simple parallel bus with a few control lines. It shouldnt be that hard for hard to set the gpu up to fire off a callback every time an incoming frame is completed surely.
Posts: 5
Joined: Mon Jul 30, 2012 2:13 pm
by adhdengineer » Tue Jul 31, 2012 10:22 am
adhdengineer wrote:surely it doesnt need that much cooperation with the GPU. the CSI interface is a fairly simple parallel bus with a few control lines. It shouldnt be that hard for hard to set the gpu up to fire off a callback every time an incoming frame is completed surely.


egh. meant serial. said parallel.
I suppose if i can't hook up to the onboard CSI i could roll my own using the GPIO pins but it seems a bit silly having too.
Reading the publicly available datasheet for the broadcom chip contains no useful info either.
Posts: 5
Joined: Mon Jul 30, 2012 2:13 pm
by Gert van Loo » Tue Jul 31, 2012 12:45 pm
adhdengineer wrote:.....

surely it doesnt need that much cooperation with the GPU. the CSI interface is a fairly simple parallel bus with a few control lines. It shouldnt be that hard for hard to set the gpu up to fire off a callback every time an incoming frame is completed surely.


It's like saying: " My motor connects to my wheels. Any motor has to connect to the wheels. So it must be trivial to take an arbitrary motor and drop it into my car"
User avatar
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2110
Joined: Tue Aug 02, 2011 7:27 am
by mahjongg » Tue Jul 31, 2012 12:49 pm
LOL well put Gert.
User avatar
Forum Moderator
Forum Moderator
Posts: 6010
Joined: Sun Mar 11, 2012 12:19 am
by adhdengineer » Tue Jul 31, 2012 8:14 pm
no, it's really not. but whatever.
Posts: 5
Joined: Mon Jul 30, 2012 2:13 pm
by gsh » Tue Jul 31, 2012 8:36 pm
Actually yes it is...

The CSI interface is not as simple as you suggest, for a start the source of the clock for the interface is in the camera module not the SoC. Therefore you need to communicate with the device to set up pll's in the camera to actually start it sending us the data.

Once we start to receive the data then you need to start doing some real work.... First of all camera sensors don't give you useful RGB data but Bayer data and therefore need significant processing to just get a picture, then you need to worry about lens shading, motion compensation, auto focus, auto exposure, dead pixel elimination, denoise algorithms (of which there are many).

Once you've got that then you get a nice picture from the camera...

If you just want to talk to the CSI interface directly, there is no ARM side driver to achieve that.

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 842
Joined: Sat Sep 10, 2011 11:43 am
by adhdengineer » Wed Aug 01, 2012 7:32 am
gsh wrote:
The CSI interface is not as simple as you suggest, for a start the source of the clock for the interface is in the camera module not the SoC. Therefore you need to communicate with the device to set up pll's in the camera to actually start it sending us the data.

so implement that as one API call to configure it?

gsh wrote:Once we start to receive the data then you need to start doing some real work.... First of all camera sensors don't give you useful RGB data but Bayer data and therefore need significant processing to just get a picture,

I'm sorry. De-Bayering needs "significant processing"? in what world?

gsh wrote: then you need to worry about lens shading, motion compensation, auto focus, auto exposure, dead pixel elimination, denoise algorithms (of which there are many).

you dont need lens if you plan to make it use interchangeable lenses or even no lenses (as i do).
Don't need auto focus.
Don't need auto exposure.
Dead pixel elimination will per unique to each individual chip and can be done in post.
De-noise is again something that can be done in post.

gsh wrote:If you just want to talk to the CSI interface directly, there is no ARM side driver to achieve that.


which is my point, there should be. Then everything else should be built off that. You should not be implementing the entire camera interface as a monolithic block but as a selection of individual parts that can be exposed externally and used individually.

I'll explain my use case so you guys can get a better understanding of where i'm coming from.
I want to build a modular camera system for use in astronomy. basically there are three main uses for the system.
1) long exposure still camera. this would be taking exposures of anything up to about 10 minutes or maybe even more if the telescope is very well set up.
2) Standard movie camera capturing RAW 640x480. Doesn't need to be any greater since it's for planetary imaging and more pixels don't really affect the end result and RAW is better than compressed for this purposes.
3) Auto guider. Basically taking 2-5 second exposures and then tracking movement of a fixed point in the image and moving the telescope mount in the opposite direction to compensate. This allows for longer exposures (and therefore much better images) in case 1.
Posts: 5
Joined: Mon Jul 30, 2012 2:13 pm
by dimonic » Mon Aug 13, 2012 5:00 pm
Actually - I agree (now) with the thrust of adhdengineer's suggestions. The gritty low level stuff such as setup and image construction should be done through a driver/lib with a callable API, so as to allow for other (camera related) uses of the CSI pinout.

There should be some way of getting the clock-out of the camera to sync with the driver that is configurable at run time.

I guess his point is that the "difficulties" you describe are in fact just details - what he wants is an open architecture that he can piggy back onto and extend - which is after all the basic philosophy of the whole Pi thing, is it not?
Posts: 20
Joined: Fri Jun 08, 2012 9:08 pm
by toxibunny » Mon Aug 13, 2012 7:15 pm
Sounds perfectly reasonable to me, and not so very much more difficult to achieve. I suggest putting together a concise, polite email and firing it off to a foundation member...
note: I may or may not know what I'm talking about...
Posts: 1238
Joined: Thu Aug 18, 2011 9:21 pm