danielm5
Posts: 2
Joined: Wed Jul 29, 2015 8:13 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Jul 29, 2015 10:00 pm

6by9 wrote:
danielm5 wrote:On the side, I'm fairly new to MMAL and I've noticed that I receive two callbacks for each image, one with and the other without MMAL_BUFFER_HEADER_FLAG_CODECSIDEINFO flag. Could anyone tell me what extra information is in the second call?
Er, try reading my first post under the heading "What is this not doing?"
Ooops, my fault, I've to assimilate all the information I found in this forum and all the other documentation referenced in a short time.
Glad it's of use.
Sure it is!

dand
Posts: 2
Joined: Wed Aug 05, 2015 4:16 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Aug 05, 2015 5:20 pm

jbeale wrote:Starting from a fresh Raspbian install, I did the below steps and was able to get a set of rawXXXX.raw files, and view them. So far, so good.
...

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo rpi-update
sudo reboot

git clone git://git.drogon.net/wiringPi
cd wiringPi/
git pull origin
./build

cd
wget https://raw.githubusercontent.com/6by9/userland/rawcam/camera_i2c
chmod +x camera_i2c

sudo mv /boot/start_x.elf /boot/start_x_OLD.elf
wget https://raw.githubusercontent.com/6by9/RPiTest/blob/master/rawcam/start_x.elf
sudo mv start_x.elf /boot/start_x.elf

insert these lines at end of /boot/config.txt
  start_file=start_x.elf
  dtparam=i2c_vc=on
  dtparam=i2c_arm=on

insert these lines at end of /etc/rc.local
  # enable i2c interface for raw camera access
  sudo modprobe i2c-dev

sudo reboot
  
sudo apt-get install cmake
git clone https://github.com/6by9/userland.git
cd userland; git fetch origin; git checkout -b rawcam origin/rawcam  
sudo ./buildme
sudo ln -s /opt/vc/bin/raspiraw /usr/bin/raspiraw

~/camera_i2c     # setup GPIO pins for camera operation
raspiraw         # run camera and record raw frames

Thank you 6by9 for making this possible. I really need to access that Bayer data as close to real-time as possible and this is a great starting point!
Also thank you jbeale for the detailed instructions on how to get this up and running.

Unfortunately, even though I followed jbeale's instructions to the T (that means even starting from a fresh Raspbian install!!) , I'm afraid I could not get raspiraw to run successfully. :?
Everything compiled and installed just fine. I did get to enable /dev/i2c-0 and wiringPi is successfully changing the pin modes as specified in the camera_i2c script (i.e. 0,1 to IN and 28,29 to ALT0).
However when I run raspiraw, it fails to write the registers over I2C ( I get multiple "Failed to write register index %d", d in [0,97] -- which seem to be coming from 'start_camera_streaming' in raspiraw.c).

Seems I am running into the same problem as tchiwam in viewtopic.php?p=763755#p763755 except I have the latest version of wiringPi (2.26)
I've run 'i2cdetect -y 0' before and after running 'camera_i2c' and not a single address shows up in either case (it's all dashes! :o )

I should also point out that raspistill runs just fine (even if I 'gpio unload i2c'!), which means the camera is working and well-connected. I've run out of Google query ideas and am desperate as I really want those un-debayered images!)
Any ideas on what could be happening?

Here's my setup:
  • Raspberry Pi Model B, Rev 2
  • Raspberry Pi Camera NoIR Rev 1.3
  • Linux raspberrypi 4.0.9+
  • gpio version: 2.26
Thank you in advance for any help! :)
Last edited by dand on Wed Aug 05, 2015 10:09 pm, edited 1 time in total.

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Aug 05, 2015 6:27 pm

There were a lot of steps and I thought I wrote them all down, but it is certainly possible I forgot something! Unfortunately I don't know where the problem lies, apart from the I2C camera link not working.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Aug 05, 2015 7:42 pm

You need to solve the issue of the I2C - that is the crux of the issue.

Whilst wiringPi is great, it doesn't read back the mapping status very well. The Foundation have now released raspi-gpio (sudo apt-get install raspi-gpio) using which "sudo raspi-gpio get" dumps out the mapping in a very readable manner.
Check the results from that to ensure that SCL0 and SDA0 are only mapped to GPIOs 28 and 29 and no where else.
...
Ah, except you say you have B rev2 - that may be the key. On the early revisions the pin usage was slightly different.
B Rev1 - I2C 1 on GPIOs 2 & 3. GPIOs 5 & 27 for LED and power.
B Rev2 - I2C 0 on GPIOs 0 & 1. GPIOs 5 & 21 for LED and power.
A+, B+, and B2 all revisions - I2C 0 on GPIOs 28 & 29. GPIOs 32 & 41 for LED and power.
You're going to have to amend camera_i2c to match the above.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

dand
Posts: 2
Joined: Wed Aug 05, 2015 4:16 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Aug 05, 2015 9:56 pm

YES! Thank you, oh, so very much!
I thought it was strange that you were enabling the I2C0 on pins 28 and 29, given http://www.raspberry-projects.com/pi/pi ... -b-io-pins. I had tried running it with I2C0 on pins 0 and 1. But I could not figure out the power pin, so I had kept it as 41. I guess one's gotta turn the peripheral ON before trying to interact with it :lol:

@jbeale: the instructions were great! There was only a couple of things I had to do that were not listed:
  • enable the camera, under raspi-config (I guess that goes without saying - just mentioning it in case someone who reads this later on doesn't know)
  • enable device tree, under raspi-config (oddly enough, it wasn't by default, and /dev/i2c-0 wouldn't show)
Thank you for the prompt response and for the great work! :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Thu Aug 06, 2015 9:17 am

Glad it's working - I'll edit the first post and try to find two minutes to make a camera_i2c_b_rev1 and camera_i2c_b_rev2.
I thought I had mentioned that it was only B+ and Pi2 pin usage, but perhaps I'd meant to and never did. I generally avoid developing on a Pi1 as it is so painfully slow to compile things, though they are useful to then deploy around the place.

The shutdown GPIO is actually turning the sensor and oscillator for the sensor on rather than the receiver peripheral (the bit built into the chip), but that's just semantics. I guess some may consider the sensor to be a peripheral to the Pi.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

tudor
Posts: 11
Joined: Wed Apr 09, 2014 12:07 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Sun Nov 01, 2015 10:06 pm

Thank you a lot for your work. It's awesome to have access to the raw stream!
My plans are for a 10bit/pixel x264 encoding (i hope a 320x240 stream would work).
The instructions are ok, but it doesn't work for me with the official DSI screen.
It would be a good idea to warn potential users that this can be an issue.
It would be nice to also put the "old" methods so that when it doesn't work there aren't 10 different instructions to try.
(i'm thinking about all the ways of saying that the arm cpu gets to drive the camera i2c bus)

A primitive way of evaluating auto-exposure would also help, the sensor datasheet is pretty long :-)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Nov 02, 2015 11:03 am

tudor wrote:Thank you a lot for your work. It's awesome to have access to the raw stream!
My plans are for a 10bit/pixel x264 encoding (i hope a 320x240 stream would work).
I'm not sure how you expect that to work. I'd expect x264 is going to want YUV data, not Bayer. Most of the spatial compression tricks it will want to pull just won't work on Bayer data as there isn't necessarily a correlation between the colour channels. The RAW10 packing is also very bizarre, so you might be better off repacking to RAW16.
tudor wrote:The instructions are ok, but it doesn't work for me with the official DSI screen.
It would be a good idea to warn potential users that this can be an issue.
This is why I2C-0 is normally considered to be controlled by the GPU and shouldn't be touched by the ARM! The display also uses it for detection and the touchscreen, so you've suddenly got two processors trying to access the same peripheral!
There is probably a way around this by specifying in config.txt that the display exists, but not loading the touchscreen driver. Needs a little bit of thought. There is no way to get the touchscreen working with this without doing hardware mods.
tudor wrote:It would be nice to also put the "old" methods so that when it doesn't work there aren't 10 different instructions to try.
(i'm thinking about all the ways of saying that the arm cpu gets to drive the camera i2c bus)
device-tree is nearly the only supported method - several people have expended a lot of effort upstreaming almost all the Pi drivers so that a stock kernel works, and I think those are going DT only.
As to the variation in which GPIOs and I2C bus to use, that could be coded into a table based on the Revision field from /proc/cpuinfo - there's a small exercise for the reader! The list of GPIOs/bus is all in https://github.com/raspberrypi/document ... t-blob.dts, and http://elinux.org/RPi_HardwareHistory#B ... on_History lists the board revisions. I can dig out the mapping if it isn't obvious.
tudor wrote:A primitive way of evaluating auto-exposure would also help, the sensor datasheet is pretty long :-)
But also those register settings don't set the sensor up to run AE - that is normally done on the GPU to provide more flexibility.
I can't provide any significant assistance on the register set due to NDAs, but there is a Linux OV5647 driver floating around the internet which may give you an alternate register set that may include AE running on the sensor.
Do ensure you set the frame height correctly, otherwise you may find you get no frames back as the line counter never hits your frame height before the CSI-2 stream signals end of frame (I can't remember if we trigger just on line count, or on frame end too).

This was a spare time project for me - I'm not employed to work on Pi. It isn't polished, and those out in the community are more than welcome to assist on any of the userspace bits. I will clean up my userland tree and do a Pull Request to get the rawcam stuff into the main repo - most people won't use it, but it just formalises things a bit more. I might even put together some basic docs. This is a not a project for novices to play with, so I have made lots of assumptions that those using it have some significant Linux experience and I don't think there is any real way around that.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

tudor
Posts: 11
Joined: Wed Apr 09, 2014 12:07 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 03, 2015 10:04 am

6by9 wrote:I'm not sure how you expect that to work. I'd expect x264 is going to want YUV data, not Bayer. Most of the spatial compression tricks it will want to pull just won't work on Bayer data as there isn't necessarily a correlation between the colour channels. The RAW10 packing is also very bizarre, so you might be better off repacking to RAW16.
As my wanted resolution for the video is around 320x240, all I need to do is get one red pixel and one green pixel from a line and a blue pixel from the next, jump a few pixels and repeat. The algorithm to produce yuv from this is trivial, i've already done it. At first I'll use 8bit/pixel from the packed data so we just have to ignore every 5th byte. Ffmpeg will happily take yuv422 if we give the resolution and fps. There is nothing particularly cpu intensive about this transformation and we keep the option of having full raw snapshots!

This project is just great! You have unlocked for us precisely the hardest parts!
There is absolutely no problem about the fact that there are rough edges. It's expected.
It's important to state the limitations before people try to use the software in order to limit the potential user-error cases.

About the auto-exposure, there are 4 million users, there is no reason that the difficulty of making an auto-exposure algorithm should be done by you or the Fundation... If I had time I would/will certainly find it very interesting to put the pixels in 256 bins and do a primitive histogram and change a few registers to get as good an exposure as possible. A simple white balance is also just 2 multiplications/pixel.

As I try to do some small encoding delay experiments the absolute fun would be to be able to show on the DSI screen video data before/after encoding :-)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 03, 2015 10:40 am

tudor wrote:As my wanted resolution for the video is around 320x240, all I need to do is get one red pixel and one green pixel from a line and a blue pixel from the next, jump a few pixels and repeat. The algorithm to produce yuv from this is trivial, i've already done it. At first I'll use 8bit/pixel from the packed data so we just have to ignore every 5th byte. Ffmpeg will happily take yuv422 if we give the resolution and fps. There is nothing particularly cpu intensive about this transformation and we keep the option of having full raw snapshots!
Fair enough - sounds plausible.
tudor wrote:About the auto-exposure, there are 4 million users, there is no reason that the difficulty of making an auto-exposure algorithm should be done by you or the Fundation... If I had time I would/will certainly find it very interesting to put the pixels in 256 bins and do a primitive histogram and change a few registers to get as good an exposure as possible. A simple white balance is also just 2 multiplications/pixel.
I'm not going to say too much, but there is something in the pipeline that may be of interest :D
tudor wrote:As I try to do some small encoding delay experiments the absolute fun would be to be able to show on the DSI screen video data before/after encoding :-)
It doesn't help that I don't have a display - I guess I ought to invest in one, and Farnell do claim to have stock. I will try to find a few moments to have a look into the firmware config options to enable just the display side without touchscreen.
Pi Towers have done the work to get the original A and B working with the display by hooking it into I2C-1 via jumpers, so it might be that those really wanting to go this route have to swap the GPU to using I2C-1 with the ARM having I2C-0, but it will probably require some surgery to the display flex (disconnecting the I2C lines).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 03, 2015 2:59 pm

6by9 wrote:
tudor wrote:As I try to do some small encoding delay experiments the absolute fun would be to be able to show on the DSI screen video data before/after encoding :-)
It doesn't help that I don't have a display - I guess I ought to invest in one, and Farnell do claim to have stock. I will try to find a few moments to have a look into the firmware config options to enable just the display side without touchscreen.
So I've looked at the source. The I2C is also used for the DSI to DPI bridge chip that is on the board, so it can't be removed totally. The touchscreen driver is also integrated in there, so not currently easy to disable just the touch screen. The firmware change should be trivial so I will look into it. It will of course mean that the touchscreen doesn't work.

As I suggested earlier, the other approach is to do something similar to this.
  • - Disconnect the I2C from the display connector.
  • Grab the dt-blob from github
  • alter the blob to set DISPLAY_SDA and DISPLAY_SCL to be 2&3 instead of 28&29 for your flavour of board.
  • add a section:

    Code: Select all

    [email protected]_I2C_PORT {
      type = "internal";
      number="1";
    };
    This appears to be absent from the version in the documentation for all revisions - someone needs a prod to update it.
  • compile to dt-blob.bin (see here)
  • copy to /boot/dt-blob.bin
  • Connect GPIOs 2&3 (header pins 3&5) to the display SDA and SCL pins.
  • Cross your fingers.
If you try that, then let me know how you get on.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 17, 2015 3:42 pm

I would like to connect an OV5640 image sensor to the CSI-2 interface. Basically, this sensor is compatible with the now obsolete OV5647 and by default, it is able to output the same Bayer format in RAW10.

But it also features a built-in ISP for Bayer to YUV and YUV to RGB conversion, many built-in pictures, lens and color corrections, built-in JPEG compression and A/F control... That is to say, most of GPU's ISP processing could be done by the sensor itself, and much more.

There are some caveats, though: besides the address on the I2C bus that is different (it looks like it can be configured in the OV5640 at power-up time), it seems that the GPU also checks the sensor model by reading its ID registers, I don't know if this will result into an error or not.

But I wonder if it would be easy to have both an I2C "pass-thru" to bypass the GPU sensor control and control what is sent to the sensor for configuring it from the ARM CPU, and a CSI "pass-thru" that bypasses most GPU ISP computation up to the optional H.264 compression, as this all can be done with the built-in sensor ISP?

This looks like a "simple-enough :mrgreen: " GPU firmware blob modification, as from what I understand from this thread, you can already get control over the I2C bus by using the gpio command and mux, and completely bypass the GPU up to the video RAM buffers that can then be accessed by the ARM CPU without copy, is it?

Michel

User avatar
jbeale
Posts: 3499
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 17, 2015 4:56 pm

Just a comment about using the Omnivision chip built-in ISP: have you ever used a cheap webcam or dashcam or Arduino-cam ? Have you compared the still and video image quality from those devices with the R-Pi camera? Having done those things myself, in my opinion the RPi image pipeline really is a significant improvement over the sensor's built-in processing.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 23874
Joined: Sat Jul 30, 2011 7:41 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 17, 2015 5:15 pm

jbeale wrote:Just a comment about using the Omnivision chip built-in ISP: have you ever used a cheap webcam or dashcam or Arduino-cam ? Have you compared the still and video image quality from those devices with the R-Pi camera? Having done those things myself, in my opinion the RPi image pipeline really is a significant improvement over the sensor's built-in processing.
What he said! In built ISP are usually a bit rubbish in comparison to properly tuned external ISP's
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Nov 17, 2015 5:28 pm

Well, there are many settings in the sensor itself, that are certainly difficult to use without proper vendor's support and corresponding NDA, whereas there are not many settings in the GPU ISP pipeline, but I agree that they do a decent job out of the box... So YMMV ;)

Anyway, it would be good to have more control on the I2C bus for things like sensor model identification, A/F control, etc., while still using the GPU ISP pipeline.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Nov 30, 2015 10:30 pm

6by9 wrote:
6by9 wrote:
tudor wrote:As I try to do some small encoding delay experiments the absolute fun would be to be able to show on the DSI screen video data before/after encoding :-)
It doesn't help that I don't have a display - I guess I ought to invest in one, and Farnell do claim to have stock. I will try to find a few moments to have a look into the firmware config options to enable just the display side without touchscreen.
So I've looked at the source. The I2C is also used for the DSI to DPI bridge chip that is on the board, so it can't be removed totally. The touchscreen driver is also integrated in there, so not currently easy to disable just the touch screen. The firmware change should be trivial so I will look into it. It will of course mean that the touchscreen doesn't work.
I've just had a fiddle with the firmware. The latest sudo rpi-update release should include a config.txt option of "disable_touchscreen". Setting that to 1 should configure the screen, but not set up the service that drives the touchscreen or backlight driver. I've not tested it extensively, but two pairs of eyes say it does the right thing. If you use it, please let me know if there are issues.

Getting both the screen to work with touchscreen and userspace access to the camera will require hardware mods to shift either display or camera to the other I2C bus.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Nov 30, 2015 10:39 pm

Hi,

Wow, thank you for the effort jbeale and 6by9 !
Great work, and I've managed to get this up and running in no time!
I'm trying to use this create a low latency tracker for well illuminated objects, since we know the ROI from the previous frame, we only need a couple hundred pixels update the information, so I hope to achieve really low latency when I don't have to decode a jpeg or h264 image

But I'm having a hard time changing to raspiraw source to achieve 720p with 60 fps.
I've tried adding
output->format->es->video.frame_rate.num = 60;
output->format->es->video.frame_rate.den = 1;
But it does not seem to change the callback timing, no matter what resolution I use

Do I need special sensor_regs for that?

Thanks,
Peter

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Mon Nov 30, 2015 11:36 pm

backinside wrote:I've tried adding
output->format->es->video.frame_rate.num = 60;
output->format->es->video.frame_rate.den = 1;
But it does not seem to change the callback timing, no matter what resolution I use

Do I need special sensor_regs for that?
Yes, you do. The port format only sets up the buffers. The I2C register set sets up the sensor to write data in a particular resolution and framerate. You need to amend those registers to make the sensor produce something other than [email protected]
If someone were to capture the I2C data for each of the modes that the GPU has configured, then I can legitimately add it into the app. Due to NDAs I can't really volunteer that information - a technicality, but I want to at least try to stay on the right side of the agreements and not drop Pi Towers in a legal quandary.

The alternative is that there is an OV5647 Linux kernel driver available on the web, and that would include a register set that could be integrated. I don't recall what modes that had configured.

NB I couldn't get the sensor up to 720P60 - [email protected] was the best I could manage. Reducing that to 1280x720 involved cropping the image on the sensor - that may be acceptable under some circumstances, but not great for general use.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Dec 01, 2015 1:46 pm

In order to use a sensor different from the now obsolete OV5647 (an OV5640), would it be possible to have a config.txt option to specify the I2C address and the ID register content to use by the firmware?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Tue Dec 01, 2015 2:05 pm

Squonk wrote:In order to use a sensor different from the now obsolete OV5647 (an OV5640), would it be possible to have a config.txt option to specify the I2C address and the ID register content to use by the firmware?
This thread is about allowing users to do all the work of grabbing frames from a sensor, not about getting the GPU firmware to change behaviour via config.txt.

Changing sensor will generally vary the register set in various ways, even if you choose another sensor from the same manufacturer. It will also need retuning of the ISP algorithms - see all the other threads where just changing the lens has significant impact due to differences in lens shading for one. Differences in red/blue sensitivities will need some tweaking of AWB. etc.
So, no, it is not as simple as asking for a different I2C address and ID register.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 02, 2015 8:05 am

6by9 wrote:This thread is about allowing users to do all the work of grabbing frames from a sensor, not about getting the GPU firmware to change behaviour via config.txt.
I agree that my post may be slightly off topic, however, this thread is the closest I could find, and still related to raw camera access with minimum GPU interaction.
6by9 wrote:Changing sensor will generally vary the register set in various ways, even if you choose another sensor from the same manufacturer.
The OV5640 is not a random sensor, it is an exact superset of the OV5647, sharing the same sensor matrix, analog amplifiers and compatible register set, with added built-in ISP and more importantly, non EOL soon.
6by9 wrote: It will also need retuning of the ISP algorithms - see all the other threads where just changing the lens has significant impact due to differences in lens shading for one. Differences in red/blue sensitivities will need some tweaking of AWB. etc.
Probably not true: there are already the standard and NoIR flavors of the Rpi camera that are not distinguishable from the GPU. However, there may have some tuning to perform to get the best possible picture, this is why I want to access the raw device in the first place.
Last edited by Squonk on Wed Dec 02, 2015 8:20 am, edited 1 time in total.

User avatar
rpdom
Posts: 15373
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 02, 2015 8:12 am

Squonk wrote:Probably not true: there are already the standard and NoIR flavors of the Rpi camera that are not distinguishable from the GPU.
The only real difference between the standard and the NoIR camera is that the IR filter isn't fitted to the NoIR as standard. It is supplied with the package as a piece of plastic that you can stick over the lens.

The sensors are identical.

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 02, 2015 8:25 am

rpdom wrote:
Squonk wrote:Probably not true: there are already the standard and NoIR flavors of the Rpi camera that are not distinguishable from the GPU.
The only real difference between the standard and the NoIR camera is that the IR filter isn't fitted to the NoIR as standard. It is supplied with the package as a piece of plastic that you can stick over the lens.

The sensors are identical.
Yes, the IR filter is just a piece of plastic, but color-wise (including AWB), this is a completely different matter.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7411
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 02, 2015 10:58 am

Squonk wrote:I agree that my post may be slightly off topic, however, this thread is the closest I could find, and still related to raw camera access with minimum GPU interaction.
Except you are referring to changes in the firmware, so the GPU is heavily involved. Changing the I2C address if you're using rawcam is just a mod to the userspace code that is on github - no firmware or config.txt changes required.
If you really want to continue discussion on config.txt options, then start a new thread. (If using just rawcam, then it can stay here).
Quonk wrote:The OV5640 is not a random sensor, it is an exact superset of the OV5647, sharing the same sensor matrix, analog amplifiers and compatible register set, with added built-in ISP and more importantly, non EOL soon.
Superset means more registers. Do those all default to the right settings to make it look like an OV5647? If not, then there is driver work required too.

I'm not sure why you are so bothered about OV5647 being EOL. RS say they currently have 10343 in stock, and Farnell have >1000. If you're basing a new product on the Pi Camera, then that should keep you going for a while.
My other guess is that you're trying to build a clone of the camera and hence are having issues as sourcing OV5647 due to EOL - that's not Pi Tower's concern. They will have planned what their next step is for the camera, but as is normal for them (having had so many delays on some products, eg display), they don't announce until they are ready to ship.

Bottom line: I am not going to make that sort of change to the firmware without Pi Towers having been convinced - take it up with them.
My advice to them is that they would need OmniVision to have been involved and to have confirmed that there are absolutely no differences. Otherwise the support burden is increased in order to support a camera module that they don't produce.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Squonk
Posts: 7
Joined: Tue Nov 17, 2015 7:58 am

Re: Raw sensor access / CSI-2 receiver peripheral

Wed Dec 02, 2015 1:35 pm

6by9 wrote:Superset means more registers. Do those all default to the right settings to make it look like an OV5647? If not, then there is driver work required too.
Yes it is! It defaults to OV5647 RAW Bayer output just liek the OV5647.
6by9 wrote:I'm not sure why you are so bothered about OV5647 being EOL. RS say they currently have 10343 in stock, and Farnell have >1000. If you're basing a new product on the Pi Camera, then that should keep you going for a while.
It will buy you another year, but not much more.
6by9 wrote:My other guess is that you're trying to build a clone of the camera and hence are having issues as sourcing OV5647 due to EOL - that's not Pi Tower's concern.
Not a clone: a camera with different lens, different FOV, M12 mounts, A/F...
6by9 wrote:Bottom line: I am not going to make that sort of change to the firmware without Pi Towers having been convinced - take it up with them.
OK, I will contact Pi Towers, thank you anyway.

Return to “Camera board”