backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: low-level interfacing to OV5647

Tue Mar 08, 2016 8:17 am

6by9 wrote:i2c_vc will pinmux i2c-0 to GPIOs 0&1. For Pi2 the camera is on 28&29, and the GPU will deal with that if you're using raspivid. Having two pins muxed to the same I2C function will fail. That is what my camera_i2c script is correcting, although I pushed some dt-overlays yesterday that allow configuring the GPIOs the ARM sets up for i2c0.
I would swear, I could run raspiraw and raspivid after that. And than run the camera_i2c script and raspiraw again.
But it does not work now :(
I'll run some more tests after a clean install
6by9 wrote:Do NOT use i2c_vc=on on Pi3. i2c0 is used by the GPU for additional functions, including controlling the camera power control GPIOs! Discussion ongoing (end of viewtopic.php?f=44&t=137848) over what can be arranged there. The camera I2C GPIOs have also moved.
Ok, I don't understand much of that thread yet, but let me know, if I can run same tests on the Pi3

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Tue Mar 08, 2016 9:49 am

backinside wrote:One more new topic in the thread, than I'll start finishing my old topics :)
I've managed to start a raspiraw when another instance was already running
After killing both of them, I can't start a capture
The output halts here:

pi@raspberrypi:~/userland-rawcam/userland $ ./build/bin/raspiraw
mmal: mmal_port_alloc: component:vc.ril.rawcam type:1 extra:0
mmal: mmal_port_alloc: vc.ril.rawcam:ctr:0: created at 0x5814e0
mmal: mmal_vc_component_create: vc.ril.rawcam

restarting the system seems to fix the problem
I've encountered this issue before, but with this method, I can reproduce the issue
Is there a way, to reset the camera if this happens?
Killing the task is a little dangerous. I think there is a double release mechanism lurking within the rawcam component that I haven't found. If it does double release then it locks the firmware up and a reboot is the only way to recover :(
The second instance should just fail on mmal_port_enable anyway - I suspect that is some error handling being missing from raspiraw as it was slightly hacked together code.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: low-level interfacing to OV5647

Tue Mar 08, 2016 9:55 am

6by9 wrote:
backinside wrote:One more new topic in the thread, than I'll start finishing my old topics :)
I've managed to start a raspiraw when another instance was already running
After killing both of them, I can't start a capture
The output halts here:

pi@raspberrypi:~/userland-rawcam/userland $ ./build/bin/raspiraw
mmal: mmal_port_alloc: component:vc.ril.rawcam type:1 extra:0
mmal: mmal_port_alloc: vc.ril.rawcam:ctr:0: created at 0x5814e0
mmal: mmal_vc_component_create: vc.ril.rawcam

restarting the system seems to fix the problem
I've encountered this issue before, but with this method, I can reproduce the issue
Is there a way, to reset the camera if this happens?
Killing the task is a little dangerous. I think there is a double release mechanism lurking within the rawcam component that I haven't found. If it does double release then it locks the firmware up and a reboot is the only way to recover :(
The second instance should just fail on mmal_port_enable anyway - I suspect that is some error handling being missing from raspiraw as it was slightly hacked together code.
Ok, so you're aware :) I'll stick with rebooting :)
Thanks :)

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Tue Mar 08, 2016 10:09 am

backinside wrote:
6by9 wrote:Do NOT use i2c_vc=on on Pi3. i2c0 is used by the GPU for additional functions, including controlling the camera power control GPIOs! Discussion ongoing (end of viewtopic.php?f=44&t=137848) over what can be arranged there. The camera I2C GPIOs have also moved.
Ok, I don't understand much of that thread yet, but let me know, if I can run same tests on the Pi3
At the moment you won't be able to get to power up the camera on Pi3 due to the control GPIOs moving. There is a driver for GPIOs via the GPU, but it didn't want to work for me when I tried it at the weekend.
You'll also need I2C mods as the I2C has also moved to pins 44&45. Fortunately that means that we can mux i2c1 to the appropriate pins and use that instead of i2c0. There are also now device tree overlays to be able to specify the GPIO pins for the I2C busses, which makes life easier.

I wasn't aware of any of the GPIO changes until just before the release, and haven't had time since to look at reworking any of the required stuff. It'll happen soon. (Whilst raspiraw is still on a private branch there's less pressure to sort it)
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: low-level interfacing to OV5647

Tue Mar 08, 2016 11:59 am

Ok, I have a version, that has everything, all the modes, exposure, flips and gain control
I have some wired issues, so I would love to get some help figuring these out (if you have time, if it's not under NDA, etc)
Mode 1, 2, 7 - Ok are Ok
Mode 3 - seems broken, It's like the buffer got overwritten
Mode 4, 5 and 6 - I've probably messed something up, it's like the resolution is not correct
But so far I have the Mode 7 I was interested in, so I'll try to push that one into production :)
https://github.com/6by9/userland/pull/1
I've created the first version of the pull request, let me know, what to fix :)

backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: low-level interfacing to OV5647

Wed Mar 09, 2016 1:13 pm

backinside wrote:Ok, I have a version, that has everything, all the modes, exposure, flips and gain control
I have some wired issues, so I would love to get some help figuring these out (if you have time, if it's not under NDA, etc)
Mode 1, 2, 7 - Ok are Ok
Mode 3 - seems broken, It's like the buffer got overwritten
Mode 4, 5 and 6 - I've probably messed something up, it's like the resolution is not correct
But so far I have the Mode 7 I was interested in, so I'll try to push that one into production :)
https://github.com/6by9/userland/pull/1
I've created the first version of the pull request, let me know, what to fix :)
I've got your feedback for the PR, and posted some answers.
I'm not sure, how it works, did you get notifications about them?
I would love to get your opinion about some questions, before I fix these issues

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Wed Mar 09, 2016 1:53 pm

backinside wrote:I've got your feedback for the PR, and posted some answers.
I'm not sure, how it works, did you get notifications about them?
I would love to get your opinion about some questions, before I fix these issues
Yes, I get notifications about comments posted as I'm one of the participants listed on the PR (same applies to issues).
I'll respond the PR. Are all your questions on there, or are there extras?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

backinside
Posts: 16
Joined: Fri May 15, 2015 10:07 pm

Re: low-level interfacing to OV5647

Thu Mar 10, 2016 2:43 pm

6by9 wrote:
backinside wrote:I've got your feedback for the PR, and posted some answers.
I'm not sure, how it works, did you get notifications about them?
I would love to get your opinion about some questions, before I fix these issues
Yes, I get notifications about comments posted as I'm one of the participants listed on the PR (same applies to issues).
I'll respond the PR. Are all your questions on there, or are there extras?
Cool, thx :) I've got all my answers now, I'll make the changes as soon as I can

Georg_PI
Posts: 9
Joined: Sun Jun 26, 2016 5:18 pm

Re: low-level interfacing to OV5647

Wed Jul 06, 2016 10:37 pm

Sorry for mumbling into your interresting conversation,

I do some research on an application with the Omnivision 5MP image sensor.
I need to use the image sensor with a very narrow ROI of only 8 to 32 full length lines.
This should give me a frame rate of more then 1000 Frames per second.
I have already been able to bypass the mysterious GPU on standard resolution images with 15 Fps and store the raw images in memory.
However, if you reconfigure the Omnivision image sensor to the narrow ROI, then nothing arrives on the Raspberry.
I guess this is due to the configuration of the MIPI interface.
Hence, I was wondering if somebody can advice me on how to program the MIPI image sensor interface on the Raspberry to accept this non standard resolution images.
Any useful advice will be most welcome.

Best regards

Georg P. Israel

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Wed Jul 06, 2016 10:45 pm

Georg_PI wrote:Sorry for mumbling into your interresting conversation,

I do some research on an application with the Omnivision 5MP image sensor.
I need to use the image sensor with a very narrow ROI of only 8 to 32 full length lines.
This should give me a frame rate of more then 1000 Frames per second.
I have already been able to bypass the mysterious GPU on standard resolution images with 15 Fps and store the raw images in memory.
However, if you reconfigure the Omnivision image sensor to the narrow ROI, then nothing arrives on the Raspberry.
I guess this is due to the configuration of the MIPI interface.
Hence, I was wondering if somebody can advice me on how to program the MIPI image sensor interface on the Raspberry to accept this non standard resolution images.
Any useful advice will be most welcome.
Welcome to the wonderful world of camera sensors - assumptions do not always follow.
If you have reprogrammed the sensor register config, have you also updated the resolution that the rawcam component on the Pi is expected to receive? If not then it'll be waiting for 1944 lines coming in and never getting them. That'll be the defines at https://github.com/6by9/userland/blob/r ... iraw.c#L58

How have you determined the correct register settings for the OV5647 for your setup? It is a very fickle sensor with multiple registers that will need tweaking. Can you post your amended tree (or at least raspiraw.c) to somewhere like github for a quick review?
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Georg_PI
Posts: 9
Joined: Sun Jun 26, 2016 5:18 pm

Re: low-level interfacing to OV5647

Wed Jul 06, 2016 10:58 pm

Hello,

this is great help.
I have a man that is looking into this issue,
and he seems to be terribly stuck. Due to this, I try to get the stuff rolling again.
I know very much, that the OV5647 register fiddling is very tedious and error prone.
This is the main reason of this research work.
I need to know what will be the real world frame rate of that sensor with an ROI of 8 to 32 lines and the minimal frame break time.
A good ROI to start with would be:

#define WIDTH 2592
#define HEIGHT 32

with a Frame break of say 24 lines, or possibly less.
I will ask my man to post here the settings. It would be very nice to get this off the ground.

Thank you very much for your advice.

Best regards

Georg P. Israel

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Thu Jul 07, 2016 1:50 pm

Georg_PI wrote:I have a man that is looking into this issue,
and he seems to be terribly stuck. Due to this, I try to get the stuff rolling again.
I know very much, that the OV5647 register fiddling is very tedious and error prone.
This is the main reason of this research work.
I need to know what will be the real world frame rate of that sensor with an ROI of 8 to 32 lines and the minimal frame break time.
A good ROI to start with would be:

#define WIDTH 2592
#define HEIGHT 32

with a Frame break of say 24 lines, or possibly less.
I will ask my man to post here the settings. It would be very nice to get this off the ground.
Post your code and it can be looked at.

You may be pushing things too far with only 32 lines. The peripheral generates interrupts at frame start, frame end, and optionally every N lines. It has one "pending" buffer register, so must pick up on frame start or a line interrupt and reprogramme that before frame end, otherwise it can't guarantee the buffers have been swapped. At 2592x32 you're in theory running at 900ish fps, so that's only 1.1ms per frame. I wouldn't like to guarantee that all the relevant buffer actions can be completed in that time frame.
Start with much slower timings/bigger frames and prove that you really have understood what is going on, and have correct register changes, then try speeding it up.

Please also be aware that the rawcam component will be deprecated when I finally get time to drive the CSI2 receiver via a Linux kernel driver, I'm just not finding the time to work on it at the moment :(
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Georg_PI
Posts: 9
Joined: Sun Jun 26, 2016 5:18 pm

Re: low-level interfacing to OV5647

Thu Jul 07, 2016 8:09 pm

Dear 6by9,

thank you very much for offering your very valuable help on this issue.
I do have a man that takes care of the code. I will ask him to post the code here and to discuss the details with you.

BTW, with 32 lines, I do expect something like 527 frames per second. That will mean 1.897ms per frame.
As much as I understand, if I run the OV5647 at full resolution with 15fps then I will have a total line time of 2700 pixel clock cycles. This includes 108 pixel clock for the line break time.
The total frame time has 1968 line times. This includes 24 line times for the frame break time.

If I assume that I have 32 lines, then that might still need the 24 frame break lines.
Due to this, I expect the total time for this to be:

(2592 + 108) * (32 + 24) = 151200 pixel clocks per ROI.
This will result to approximately 527 Frames per second.

One side question,
did you ever experiment with small ROI's with the Omnivision sensors ?
Do they have some hidden limitations that hinder us to use them for such an application?
I understand the possible limitations from the Raspberry for high frame rates. But are there known limitation from the Omnivision side ??

Well, I look very much forward to have this little application working.

Best regards

Georg P. Israel

Stefano Moratto
Posts: 2
Joined: Thu Oct 02, 2014 9:13 am

Re: low-level interfacing to OV5647

Thu Jul 07, 2016 11:50 pm

Hi,
I'm helping Georg, and this is the modification I made to raspiraw http://pastebin.com/QFwAgxu1.

It is basically the same program; I added command line parsing in order to get the image's height and the registry settings functions.
I'm able to get images having height from 1944 to 80 pixels. Below 80 the handler's function is never called.

Best,
Stefano

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Fri Jul 08, 2016 9:04 am

Stefano Moratto wrote:I'm helping Georg, and this is the modification I made to raspiraw http://pastebin.com/QFwAgxu1.

It is basically the same program; I added command line parsing in order to get the image's height and the registry settings functions.
I'm able to get images having height from 1944 to 80 pixels. Below 80 the handler's function is never called.
OK, that all looks vaguely sane although I haven't tried it myself. (I do object to a variable called "HEIGHT" though. Defines in capitals, variables in lower or mixed case, otherwise you confuse people. ;) )

Do you actually get the frame rate increase you're expecting with height 80? I know there are other registers which control related values and I'm not sure you have got them all.

I've checked the code handling the CSI2 peripheral. It has a slightly too simplistic approach to how many interrupts to produce per frame by taking height>>3 as the line interrupt frequency. In your case that will result in the crazy number of every 4 lines or 130usecs (each line is ~32.5us). It may be running an RTOS, but that is excessive. I guess I wasn't expecting people to be running at quite such a rate.
I will make a patch to the rawcam component to set the line interrupt rate to something more sane (probably 16). There are a couple of other patches waiting to be released, so I'll throw that at Pi Towers now in the hope it'll make an rpi-update release in the next day or so.

You can enable the GPU logging from the rawcam component by doing "vcgencmd set_logging level=0xc0", and "sudo vcdbg log msg" to read it. I doubt you'll be able to read it reliably whilst rawcam is running as the circular buffer will be written faster than the ARM is extracting it, so do a test run, stop it, then grab the logs.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Fri Jul 08, 2016 2:33 pm

Firmware update done - https://github.com/Hexxeh/rpi-firmware/ ... d0df7c8d38
If you "sudo rpi-update" it'll grab that release and you've got a chance of a more sensible interrupt rate.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Stefano Moratto
Posts: 2
Joined: Thu Oct 02, 2014 9:13 am

Re: low-level interfacing to OV5647

Sun Jul 10, 2016 7:47 pm

Thanks,
I'm going to test the new firmware.

Stefano

(BTW; I used HEIGHT in uppercase for laziness, I changed a #define into a variable ;-) )

malkauns
Posts: 10
Joined: Tue May 12, 2015 7:02 pm

Re: low-level interfacing to OV5647

Fri Sep 30, 2016 12:20 am

I'm trying to understand MIPI. Can someone please tell me what a SoT and EoT sequence looks like? I couldn't get it from the spec. My end goal is to grab frames at 60fps from this sensor using an FPGA.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Sat Oct 01, 2016 9:23 pm

malkauns wrote:I'm trying to understand MIPI. Can someone please tell me what a SoT and EoT sequence looks like? I couldn't get it from the spec. My end goal is to grab frames at 60fps from this sensor using an FPGA.
Please stick with your other thread at viewtopic.php?f=43&t=161420 - whilst this thread claims to be low level, you're lower still and messing with the electrical layer.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Ahmad.Zaklouta
Posts: 2
Joined: Mon Feb 13, 2017 10:54 am

Re: low-level interfacing to OV5647

Sun Mar 19, 2017 8:43 pm

Hi everyone,

What a great job!

I have a project to interface ov5647 with fpga for cube satellite. I have two problems. the first one is related to the registers set. What are the registers related to resolution? I need to capture one image per day. It should be small around 2 MP so it can be sent to earth. I checked the raspiraw and the different mode but I couldn't manage to understand how the resolution determined.

Second: I couldn't manage to understand how to capture one picture. in other words, how to program the shutter to take one picture.

I would appreciate any help.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4595
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: low-level interfacing to OV5647

Sun Mar 19, 2017 9:33 pm

Ahmad.Zaklouta wrote:I have a project to interface ov5647 with fpga for cube satellite. I have two problems. the first one is related to the registers set. What are the registers related to resolution? I need to capture one image per day. It should be small around 2 MP so it can be sent to earth. I checked the raspiraw and the different mode but I couldn't manage to understand how the resolution determined.
Sorry, I need to refer you to the comment at https://github.com/6by9/userland/blob/r ... odes.h#L30 other than to say that those arrays of register sets match the modes as specified in http://picamera.readthedocs.io/en/lates ... nsor-modes
You'll be getting Bayer raw 10 data out. Again the PiCamera docs do a good job at explaining the format at http://picamera.readthedocs.io/en/lates ... a-captures, or the V4L2 docs at https://www.linuxtv.org/downloads/v4l-d ... gb10p.html
Ahmad.Zaklouta wrote:Second: I couldn't manage to understand how to capture one picture. in other words, how to program the shutter to take one picture.
You don't get just one frame. You tell it to start streaming, wait for a frame end, then tell it to stop. The first frame out is not necessarily correctly exposed, so you probably want to drop it.
The other thing to consider is that something needs to determine appropriate exposure time and gain settings for your captures. That is not automatic using those register sets, so capturing just one frame is generally not going to be helpful.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
jbeale
Posts: 3301
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: low-level interfacing to OV5647

Mon Mar 20, 2017 9:46 pm

Ahmad.Zaklouta wrote:I have a project to interface ov5647 with fpga for cube satellite.
Have you done any measurements on the OV5647 to determine if "raw" data from this sensor actually gives you any more information than simply uncompressed (eg. raspistillyuv) especially given that you apparently intend to resample the image down to 2MB for transmission? See also https://www.raspberrypi.org/documentati ... /camera.md
There is a convenient software chain available for working with normal images, but with raw you have to do much more work.

Also I'm curious what the performance of this sensor is in space, relative to other sensors, given the much higher radiation environment there than on earth where it was designed for (eg. mobile phone sensor).

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 8 guests