Sony CMOS Sensor: SonyIMX179

[sunilksingh13]

Dear Friends ,

I am looking for Linux driver for Sony 8Mp CMOS Image Sensor , IMX179.
I will be glad to receive help from any of you. We are also welcome to join our team as freelancer and benefit financially too.

Sincerely
Sunil

[jdb]

This sounds awfully like spam, but I'm going to bite anyway.

What were you hoping to achieve by interfacing an IMX179 to the Pi?

[sunilksingh13]

This sounds awfully like spam, but I'm going to bite anyway.

What were you hoping to achieve by interfacing an IMX179 to the Pi?

Sunilksingh13 : We are designing 8MP wearable camera and plan to use BCM2835 and Sony IMX179 as image sensor. Currently we developed the Prototype with Omnivision but want to move to Sony IMX 179 for several reasons.
Will appreciate your help!

[jamesh]

This sounds awfully like spam, but I'm going to bite anyway.

What were you hoping to achieve by interfacing an IMX179 to the Pi?

Sunilksingh13 : We are designing 8MP wearable camera and plan to use BCM2835 and Sony IMX179 as image sensor. Currently we developed the Prototype with Omnivision but want to move to Sony IMX 179 for several reasons.
Will appreciate your help!

Have you spoken to Broadcom about this? Because the IMX179 is not a supported camera (although the IMX175 is, and I suspect that is similar.). Broadcom are the people who would be able to a) Sell you the 2835, and b) Ensure the camera works.

Won't be cheap though, unless you use an already supported camera like the 175.

[sunilksingh13]

Hello james,

Yes we have approach them but they have not yet came back with there answer. You mention imx175 is supported, is there any code/example which you can refer too?

Broadcom is ready to sell through distributor but not ready to support directly . If we can get access to register set of MIPI CSI we can do it ourself.

Regards
Sunil

[jamesh]

Hello james,

Yes we have approach them but they have not yet came back with there answer. You mention imx175 is supported, is there any code/example which you can refer too?

Broadcom is ready to sell through distributor but not ready to support directly . If we can get access to register set of MIPI CSI we can do it ourself.

Regards
Sunil

No, you cannot do it yourself because you don't have access to the CSI-2 on the GPU, which is closed source. Which is the same reason I cannot supply any source code.

[albug]

Hello James. Thanks for your input.

Don't worry. This is not SPAM! I am an engineer working with Sunil. We have partnered recently and combined our two separate consulting/R&D companies (Bitwiz Inc and UnivLabs Inc) into a new R&D Services company called SpeedX Inc (http://www.SpeedXInc.com, site under construction now).

We are trying to cludge a driver together, from existing Raspberry Pi and other sources (ie. Roku 1 (BCM2708), Roku 2 (BCM2835), and Google Nexus 5 (Qualcom Chipset), etc..) for a Sony IMX179 8 MP imager. We have been hard at work on this project for the last 5 months and have a team of about 7 software engineers (for only embedded side) working on the embedded software for it.

We are really greatful for all of the contributors in this community. We could not have gotten as far as we have, in our project, were it not for their contribution (: We want to give back to the community also.

When starting this project, we created a Cadence Capture CIS version of the latest (Rev B) RPi schematic. We are going to feed this back to the community. Currently, only PDF schematics are available. We will provide the source design files (which many people have requested) for the RPi, which will make it easier to develop new hardware. Our prototype hardware (for a unique and leading edge wearable camera called the "CatchCAM") will be in our hands shortly. We are pushing to complete the camera driver for it now.

I have written a long sales pitch to Broadcom already, and have been pressuring them for months, pleading with them to release VideoCore ISP API's and MIPI/CSI-2 interface documentation so that we can complete a good quality driver (not cludge). I have told them that, if they provide us the documenation we need, we would create a generic more open camera interface/API (using MMAL/VideoCore and/or OpenMax), and we are willing to feed this driver/API back into the community to benefit others. This will allow the Raspberry Pi community to a) Have a second choice for a camera (other than the existing Omnivision 5 MP imager) and b) Be able to create their own image drivers for any camera and c) be able to tune required ISP functions (ie. AF, AE, AWB, EIS, BPC, BLC) for any imager (to improve quality of image).

If the community rallies around this request, maybe we can pressure Broadcom into giving us access to the API's we need. A few weeks ago, they released the 3D Graphics engine docs/API/Source. Although this did not include camera/MIPI/ISP related drivers or code, it did provide some header files which are helping us. Anyway, it appears they are open to helping this community. But, I am wondering if they realize that we are still seriously lacking in docs and API's we need to use the RPi platform as a basis for commercial products?

The biggest issue that we are facing is that it appears the RPi platform is more of a toy for university students and hobbyists to play with, than a platform from which real world products can be developed! In other words, not having access to key VideoCore API's limits the usefullness of the BCM2835 to hobbyists! For instance, another thing is that we ported an AAC and MP3 audio codec to the plaform. We got it working. But, the real way to do this is to use the VideoCore for the compression algorithm and off-load the ARM core from this work. But, since we don't have proper VideoCore access (according to another programmer we have working on this), we are running the MP3 encoder on the ARM core (which we would rather not do).

We are having similar issues with using the second SDIO port: lack of documentation. We need this to make sure we can get our WiFi module working.

For our image driver, we have found a reference I2C driver for the Omnivision imager (for control), and some others may also be provided in the source, and are modifying it to work with our Sony IMX179 imager. But, we are lacking in MIPI CSI-2 data interface info (ie. we have some register definitions but no detailed info about bits in those registers and no sample/reference code). Because we happened to have chosen a MIPI camera module which also uses 2 MIPI data lanes (like the Omnivision 5 MP imager), then we may be able to cludge something together. But, we may not be able to get access to our full 8 MP pixel array (may be stuck with 5 MP array).

Also, we have not tested yet the ISP hooks that we found (AWB, AE) and are not sure if they are even used or implemented for the RPi 5 MP Omnivision imager??? Would you happen to know??? We are going to try and implement AF and EIS on the ARM core but it would be better to be able to use the VideoCore for this purpose, if we had the neccessary access it. And, when all is said and done, we are not sure if we can get full image quality from our Sony IMX179 imager due to lack of ISP tunning hooks for the various ISP algorithms.

I understand the GPU ISP and MIPI CSI-2 code is closed source. But, I am wondering:
a) Do you have it? Can we pay you to develop the portions of the driver, for us, that are closed source?
b) Can you provide us with contact info for an engineer that actually developed any BCM2835 (or in same family) image driver (for any image sensor). Maybe they will have access to the info we need.

[jamesh]

Interesting post!

Actually, I am an engineer working on the Videocore, specifically cameras (was working on the IMX214 only today). However, I am not permitted to do outside work of this type. neither are any other Brcm employees as far as I know. In addition, any work I do would, according to mine (and all other Brcm employee contracts) be copyright Brcm. And, you would only get it in binary form -which since you wouldn't have compiler or facilities to build your own firmware, is a moot point anyway.

What I can say is that were a driver to be developed by Broadcom (and usually that costs many 10 of thousands of $) it would work in much the same way as the Omnivision currently available - all AE, AGC, AWB would be done inside the Videocore, so that is something you wouldn't need to work on. This is already done for the current Raspi camera module - try out Raspistill for good examples of what it can do. Writing your own code to control the camera is many man years of work to make it good, and for decent performance needs to be done on the GPU, the ARM simply isn't fast enough.

With regard to the Pi being a toy for university students and hobbyists.Well, yes, I guess you can say that. That is approximately its original purpose - teaching and education. Anything more is a bonus.

But, and being really honest here, unless you can persuade Brcm of the benefits of your project, with some approximate sales figures (will need to be in the high 100k's at least), it's going to be difficult to get anything out of them. That's simply the Brcm business model, and that of most advanced SoC manufacturers. The economies of scale mean that small scale stuff simply isn't cost effective.

On the other hand, the Foundation does have source code access. What it doesn't have is any camera driver experts. Bit of a catch-22.

[GarySunGang]

Hi all:
I'm new to RaspberryPI and Linux.
But I have many years experience with Camera and Cmos sensors. Also I'm familiar with SonyIMX179.

According to the post that jamesh posted,
There're no opensource of the sensor driver and the ISP configuration part now and in the future....

I just bought RaspberryPI+OV sensor one month ago, and I'm new to Linux also. I found the image quality is really bad. I wanted to improve the sensor driver and the ISP configuration of BCM2835(also the default AE algorithm is very poor), But I could not find the source code until today I read this topic and find out this is impossible.....

The raspberryPI has a working camera with 3A and ISP configuration, but the image quality is really poor. There're many obvious problems... such as even the pedestal is not subtracted correctly, AE has blinking problem, color matching is not good etc.

I guess the sensor driver and ISP configuration parts are not under Linux, the SoC runs another RTOS at same time(with Linux together).
Does anyone know about this?

[riklaunim]

I don't know what are your goals but wouldn't be easier and cheaper to get an OEM board from company like Point Grey, Ximea, IDS Imaging or Basler and provide additional hardware to make the final product?. Many sensors to choose from, good SDK, support, and some also have ARM support by default.

I use/used their "machine vision" cameras for planetary photography with good results (those brands are quire well known among astrophotographers).

[jamesh]

Hi all:
I'm new to RaspberryPI and Linux.
But I have many years experience with Camera and Cmos sensors. Also I'm familiar with SonyIMX179.

According to the post that jamesh posted,
There're no opensource of the sensor driver and the ISP configuration part now and in the future....

I just bought RaspberryPI+OV sensor one month ago, and I'm new to Linux also. I found the image quality is really bad. I wanted to improve the sensor driver and the ISP configuration of BCM2835(also the default AE algorithm is very poor), But I could not find the source code until today I read this topic and find out this is impossible.....

The raspberryPI has a working camera with 3A and ISP configuration, but the image quality is really poor. There're many obvious problems... such as even the pedestal is not subtracted correctly, AE has blinking problem, color matching is not good etc.

I guess the sensor driver and ISP configuration parts are not under Linux, the SoC runs another RTOS at same time(with Linux together).
Does anyone know about this?

You are right, the image tuning parameters are in the GPU which is closed source, so you cannot change them yourself. We are aware of a number of problems with the tuning, but simply have not had time to fix them, although I was unaware the black level was incorrect. The AE flicker we have spent a little time on, but we are quite busy at the moment, and there is no more time to spend on it.

[Toysrfun]

I am following up on this old post to see if IMX179 drivers are yet available for pi 3 B+. Interest is in using high resolution in 3D scanning. There is a large community who would benefit from the use of this camera if drivers were or are made available. Thank you.

[6by9]

I am following up on this old post to see if IMX179 drivers are yet available for pi 3 B+. Interest is in using high resolution in 3D scanning. There is a large community who would benefit from the use of this camera if drivers were or are made available. Thank you.

A lot has happened in 4 years.

All the infrastructure is in place to write your own driver if you can source the module yourself. See https://www.raspberrypi.org/forums/view ... 4#p1375802 for details.
Getting a camera to market is quite an involved process, mainly due to the compliance testing and image tuning when run through the firmware (your responsibility in the above solution). Producing lots and lots of versions of camera modules makes little commercial sense as each typically then sells in smaller quantities.
IMX179 is an 8MPix sensor the same as IMX219, so what do you think it adds over the IMX219?