mbf
Posts: 1
Joined: Sun Jan 27, 2019 12:10 am
Location: Sunnyvale, CA
Contact: Twitter

Looking for community interest in making new sensor board

Sun Jan 27, 2019 12:36 am

I'm interested in making a DIY DSLR project out of a Raspberry Pi (likely the Zero Wireless model), and I am wondering how much support I might be able to get to connect a different Sony Exmor sensor (IMX???) with the Raspberry Pi. Currently, I'm looking at the newer models ( https://www.sony-semicon.co.jp/products ... index.html ) -- The IMX299 (4/3rds size sensor) seems like the easiest to connect to a Pi given that it also has a 4-lane MIPI interface like the Camera Board V2 (which is based on the IMX219). I understand that I'd need to get schematics and route a PCB and use a contract manufacturer and sample the parts from a distributor to create such a board.

The biggest unknowns for me are if the MIPI interface on the BCM2835 with its firmware might support the IMX299 vs the IMX219. I also do not have pricing on the sample or quantity rates of the IMX299 (or ideally in the future, the IMX410 - though that uses a different interface which would likely require additional hardware).

Would anyone else be interested in working on this with me, assuming it is possible?

Thanks!
-Mike Fried

klricks
Posts: 6594
Joined: Sat Jan 12, 2013 3:01 am
Location: Grants Pass, OR, USA
Contact: Website

Re: Looking for community interest in making new sensor board

Sun Jan 27, 2019 5:12 am

The camera module has a crypto chip that prevents 3rd party camera boards from working with a RPi.
Per this thread you could use the Compute board: https://www.raspberrypi.org/forums/view ... p?t=178038

Schematic: https://www.raspberrypi.org/documentati ... a2_2p1.pdf
Unless specified otherwise my response is based on the latest and fully updated Raspbian Buster w/ Desktop OS.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23710
Joined: Sat Jul 30, 2011 7:41 pm

Re: Looking for community interest in making new sensor board

Sun Jan 27, 2019 7:20 am

The drivers in the GPU only support the Raspbery Pi supplied sensors. You can use other mechanisms to interface with any sensors, but then you don't get to use the ISP, so no real time stuff, e.g. video or preview or AGC etc as it all needs to be done on the ARM's
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7329
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Looking for community interest in making new sensor board

Sun Jan 27, 2019 10:02 am

Standard Pi models only have 2 MIPI data lanes exposed. The Compute Modules expose all 4 lanes on CAM1 (the CAM0 peripheral only supports 2 lanes).

The firmware is closed and will only ever support the camera modules made by RPT.

However there is a standard Linux V4L2 kernel driver for the CSI2 peripheral - https://github.com/raspberrypi/linux/bl ... 5-unicam.c. It has been proven to work with three CSI2 source devices so far - OV5647 (v1 camera module), TC358743 (HDMI to CSI2 bridge chip), and ADV7282M (analogue video to CSI2 bridge). Look in the DT overlays folder and you'll find overlays to configure them. Drivers for those devices are in https://github.com/raspberrypi/linux/tr ... /media/i2c. You will need to write a driver for your new device.
Any V4L2 app will give you the basics there, but https://github.com/6by9/yavta/ plumbs V4L2 and MMAL together.

This gives you the raw Bayer frames, and manual control over exposure and gain.
Bayer needs significant extra processing, hence the ISP. Contrary to what jamesh says, so you CAN get to the ISP hardware block via MMAL. You can NOT get to the tuner algorithms for AE/AGC/AWB, therefore you have to provide those control loops and control the full pipe, and the implementation is up to you (there is a trivial "grey world" AWB algorithm in https://github.com/6by9/raspiraw).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23710
Joined: Sat Jul 30, 2011 7:41 pm

Re: Looking for community interest in making new sensor board

Sun Jan 27, 2019 11:55 am

6by9 wrote:
Sun Jan 27, 2019 10:02 am
Standard Pi models only have 2 MIPI data lanes exposed. The Compute Modules expose all 4 lanes on CAM1 (the CAM0 peripheral only supports 2 lanes).

The firmware is closed and will only ever support the camera modules made by RPT.

However there is a standard Linux V4L2 kernel driver for the CSI2 peripheral - https://github.com/raspberrypi/linux/bl ... 5-unicam.c. It has been proven to work with three CSI2 source devices so far - OV5647 (v1 camera module), TC358743 (HDMI to CSI2 bridge chip), and ADV7282M (analogue video to CSI2 bridge). Look in the DT overlays folder and you'll find overlays to configure them. Drivers for those devices are in https://github.com/raspberrypi/linux/tr ... /media/i2c. You will need to write a driver for your new device.
Any V4L2 app will give you the basics there, but https://github.com/6by9/yavta/ plumbs V4L2 and MMAL together.

This gives you the raw Bayer frames, and manual control over exposure and gain.
Bayer needs significant extra processing, hence the ISP. Contrary to what jamesh says, so you CAN get to the ISP hardware block via MMAL. You can NOT get to the tuner algorithms for AE/AGC/AWB, therefore you have to provide those control loops and control the full pipe, and the implementation is up to you (there is a trivial "grey world" AWB algorithm in https://github.com/6by9/raspiraw).
Ah, I'd forgotten about the ISP access via MMAL. Sorry for the misinformation.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Doub
Posts: 24
Joined: Fri Apr 07, 2017 9:48 pm

Re: Looking for community interest in making new sensor board

Wed Jan 30, 2019 5:06 am

I for one would be very interested in more camera options. I'd be happy to provide some software development time if I can use it on my own project. I always assumed the MIPI CSI input had to go through closed source drivers and it was impossible to make third party camera modules not based on the two official camera sensors, but I'm glad to read it might not be so closed.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23710
Joined: Sat Jul 30, 2011 7:41 pm

Re: Looking for community interest in making new sensor board

Wed Jan 30, 2019 10:35 am

Doub wrote:
Wed Jan 30, 2019 5:06 am
I for one would be very interested in more camera options. I'd be happy to provide some software development time if I can use it on my own project. I always assumed the MIPI CSI input had to go through closed source drivers and it was impossible to make third party camera modules not based on the two official camera sensors, but I'm glad to read it might not be so closed.
What you don't get is the AWB and AGC closed loop control, so you will need to provide that, and because this needs to be done on the ARM there will be a performance degradation.

And camera tuning is a horribly long and convoluted job. It's not software as such, it's setting a bucket load of parameters that are passed to the ISP stages, and having a good knowledge of the stages involved is quite important.

Not trying to put anyone off, just trying to give people an idea of the work load involved. Which is a lot.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
"My grief counseller just died, luckily, he was so good, I didn't care."

Return to “Camera board”