cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

detecting camera without firmware stack

Tue Sep 14, 2021 11:26 am

assuming that i have already muxed an i2c controller to the i2c bus on the camera header

can i reasonably assume that every camera module uses smbus style registers?
what bus-address and register# should i read to identify which camera is connected? what value should i expect when reading it?

are there any risks to reading an address for one camera module, but that address has destructive effects on another module?
do i need to drive the CAMERA_SHUTDOWN pin in the right direction before the i2c will even respond?


for context, i'm wanting to automate things like "dtoverlay=ov5647" based on detection of the camera sensor on the i2c bus

naushir
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 108
Joined: Mon Apr 25, 2016 10:21 am

Re: detecting camera without firmware stack

Tue Sep 14, 2021 12:52 pm

for context, i'm wanting to automate things like "dtoverlay=ov5647" based on detection of the camera sensor on the i2c bus
This is indeed a useful features and we are looking into how to correctly implement it, so watch this space.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11629
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: detecting camera without firmware stack

Tue Sep 14, 2021 3:40 pm

For the cameras supported by the firmware it has just been implemented
https://github.com/Hexxeh/rpi-firmware/ ... c7d43c17c5

Add "camera_auto_detect=1" to config.txt and it should check for the supported and add the appropriate dtoverlay.

Note that sensors are NOT SMBus compatible - they are generally I2C.
They do vary as to how big the register index values are, where the ID register is, and various other things.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Tue Sep 14, 2021 7:37 pm

6by9 wrote:
Tue Sep 14, 2021 3:40 pm
For the cameras supported by the firmware it has just been implemented
https://github.com/Hexxeh/rpi-firmware/ ... c7d43c17c5

Add "camera_auto_detect=1" to config.txt and it should check for the supported and add the appropriate dtoverlay.

Note that sensors are NOT SMBus compatible - they are generally I2C.
They do vary as to how big the register index values are, where the ID register is, and various other things.
i'm wanting to do the same on my own firmware/bootloader, so i was hoping to get an actual answer on what registers to read, and what values to expect

User avatar
HermannSW
Posts: 4516
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: detecting camera without firmware stack

Tue Sep 14, 2021 8:32 pm

cleverca22 wrote:
Tue Sep 14, 2021 7:37 pm
i'm wanting to do the same on my own firmware/bootloader, so i was hoping to get an actual answer on what registers to read, and what values to expect
raspiraw answers all that!

v1:
https://github.com/raspberrypi/raspiraw ... 1020-L1024

v2:
https://github.com/raspberrypi/raspiraw ... #L722-L726

HQ:
https://github.com/raspberrypi/raspiraw ... #L902-L906
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Tue Sep 14, 2021 8:36 pm

HermannSW wrote:
Tue Sep 14, 2021 8:32 pm
raspiraw answers all that!
.i2c_addr = 0x36,
.i2c_addressing = 2,
.i2c_ident_length = 2,
.i2c_ident_reg = 0x300A,
.i2c_ident_value = 0x4756, // 0x5647 byte swapped
the way i parse that, is that you first write $i2c_addressing bytes of data ($i2c_ident_reg) to address $i2c_addr
you then read $i2c_ident_length bytes back, and expect it to exactly match $i2c_ident_value

the byte values are backwards, because its a LE number, but its simpler to just give the software the exact bytes it expects, and not encode that

looks easy enough to implement, and all of the code should also be in raspiraw if i need a reference

User avatar
HermannSW
Posts: 4516
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: detecting camera without firmware stack

Tue Sep 14, 2021 9:05 pm

cleverca22 wrote:
Tue Sep 14, 2021 8:36 pm
looks easy enough to implement, and all of the code should also be in raspiraw if i need a reference
This part of "probe_sensor()" does it:
https://github.com/raspberrypi/raspiraw ... #L390-L402
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Tue Sep 14, 2021 9:11 pm

yep, looks pretty simple

now i just need to crack open the old BCM2835-ARM-Peripherals.pdf and implement some baremetal i2c!

User avatar
Gavinmc42
Posts: 6059
Joined: Wed Aug 28, 2013 3:31 am

Re: detecting camera without firmware stack

Wed Sep 15, 2021 4:48 am

Just noticed the mode .h files.
Limited number of modes for imx477?
Or this just for basic settings and the other modes are DIY?

Baremetal setup by just copying those register settings.

Where is that hiding in the Libcamera stuff?
Libcamera now has a new detect app.
Libcamera is for 64bit Pis, so they say.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 5:08 am

Gavinmc42 wrote:
Wed Sep 15, 2021 4:48 am
Where is that hiding in the Libcamera stuff?
i think currently, its all in the kernel drivers, triggered by the dtoverlay=

i have linux able to boot on an rpi2 without any blobs involved
so if i just read the files HermannSW linked, i can implement auto-detection in the bootloader
then i just need to implement dtoverlay=, and the raw camera should just automagically work on boot, without any special config

getting hw accelerated debayer and awb/agc, is whole other ordeal though, that currently relies on the blobs to manage the ISP

User avatar
HermannSW
Posts: 4516
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: detecting camera without firmware stack

Wed Sep 15, 2021 6:50 am

cleverca22 wrote:
Wed Sep 15, 2021 5:08 am
getting hw accelerated debayer and awb/agc, is whole other ordeal though, that currently relies on the blobs to manage the ISP
Whether you need that depends on what you want to do.
If you only need frame for feature extraction (eg. for controlling robot), all you need to know is that 2x2 squares show rg/Gb pixels (top left), and that botton left "G" pixel is the brightest of the 4 pixels:
viewtopic.php?f=43&t=189661#p1192548
Image
So if you create (w/2)x(h/2) frame from wxh raw Bayer frame, you can use that as monochrome frame.
raspiraw captures raw10 (10bit per pixel), so you just need to extract say the odd row left of 2 pixels (the bright "G" pixels) 10bit values, do ">>2" and store the 8bit value into new (w/2)x(h/2) frame (can be done inplace). The new frame is GREY8 format then.

Alternatively you can use an Arducam monochrome global shutter camera (my favorite is the 25.99$ 0.3MP one (should be cheap enough to give it a try), provides 640x480@135fps, or what I prefer 320x240@204fps). They provide GREY8 frames by default. The Arducam libcamera framework is based on raspiraw and on github, so that should work without firmware/GPU for baremetal you are interested in:
https://github.com/ArduCAM/MIPI_Camera/tree/master/RPI

640x480@135fps slowmo taken with 0.3MP monochrome global shutter camera, (raspcatbot) robot moving with >2m/s, room dark, one bright desktop light only, in cable car mode:
Image
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 7:09 am

HermannSW wrote:
Wed Sep 15, 2021 6:50 am
Whether you need that depends on what you want to do.
If you only need frame for feature extraction (eg. for controlling robot), all you need to know is that 2x2 squares show rg/Gb pixels (top left), and that botton left "G" pixel is the brightest of the 4 pixels:
you still have agc problems then

the ISP is measuring the average brightness, in a hw accelerated manner
and libcamera uses that in a feedback loop, to adjust the shutter time (i2c regs) and analog gain (more i2c regs) in the camera

without the ISP to help out, youll either have fixed/manual brightness, or need to add some cpu intensive brightness calcs into whatever is processing the image

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29334
Joined: Sat Jul 30, 2011 7:41 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 8:58 am

Gavinmc42 wrote:
Wed Sep 15, 2021 4:48 am
Libcamera is for 64bit Pis, so they say.
Correct, but we are expecting it to be used on ALL Pi OS as its the future.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 9:22 am

jamesh wrote:
Wed Sep 15, 2021 8:58 am
Gavinmc42 wrote:
Wed Sep 15, 2021 4:48 am
Libcamera is for 64bit Pis, so they say.
Correct, but we are expecting it to be used on ALL Pi OS as its the future.
i installed libcamera on my pi2 (arm32 mode) a few days ago, and it just worked without any major issue (just minor ones, like LD_LIBRARY_PATH)

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 29334
Joined: Sat Jul 30, 2011 7:41 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 9:29 am

cleverca22 wrote:
Wed Sep 15, 2021 9:22 am
jamesh wrote:
Wed Sep 15, 2021 8:58 am
Gavinmc42 wrote:
Wed Sep 15, 2021 4:48 am
Libcamera is for 64bit Pis, so they say.
Correct, but we are expecting it to be used on ALL Pi OS as its the future.
i installed libcamera on my pi2 (arm32 mode) a few days ago, and it just worked without any major issue (just minor ones, like LD_LIBRARY_PATH)
That's good news. Lots of dev work being done on this, so I really would suggest people start moving to it. Whether that's applicable to this thread I doubt!
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Working in the Applications Team.

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 9:35 am

jamesh wrote:
Wed Sep 15, 2021 9:29 am
That's good news. Lots of dev work being done on this, so I really would suggest people start moving to it. Whether that's applicable to this thread I doubt!
the main goal of this thread, is to auto-detect the camera, and load the right overlays, when using the open firmware ive been working on
then libcamera will just automatically work out of the box

i'm now able to boot linux on a pi2, with working usb-host, and ntsc output, rigged to the linux console

ive not tested the CSI peripheral yet, and i know the ISP will be a major blocker for hw accel

from what ive heard, the blob is barely doing anything with the ISP now, its just exposing every control over the VCHIQ? protocol, so why is it still in blob land?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11629
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: detecting camera without firmware stack

Wed Sep 15, 2021 9:50 am

cleverca22 wrote:
Wed Sep 15, 2021 9:35 am
from what ive heard, the blob is barely doing anything with the ISP now, its just exposing every control over the VCHIQ? protocol, so why is it still in blob land?
ISP (and codec) hardware are both Broadcom IP which we can not publish. In both cases they take higher level controls to configure them, and the firmware then sets up the hardware to do the requested processing.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 221
Joined: Tue Jan 07, 2020 9:15 am

Re: detecting camera without firmware stack

Wed Sep 15, 2021 9:57 am

Yes, I was going to answer too! I guess it depends on what "barely doing anything" means. Besides the confidentiality issue, we're always going to need some "blob" on the GPU which generates some extra control data, kicks the ISP into life and deals with interrupts. I don't see any way to drive the VC4 ISP without that.

cleverca22
Posts: 4361
Joined: Sat Aug 18, 2012 2:33 pm

Re: detecting camera without firmware stack

Wed Sep 15, 2021 10:19 am

6by9 wrote:
Wed Sep 15, 2021 9:50 am
ISP (and codec) hardware are both Broadcom IP which we can not publish. In both cases they take higher level controls to configure them, and the firmware then sets up the hardware to do the requested processing.
one pipe-dream i have, is for that code to remain as a blob, but be a loadable module underneath the open firmware
so broadcom can keep its secrets, but the kernel around it is open, and users can modify anything else the firmware does

but i can see how broadcom may not like that, it makes it much more clear what code is even involved in the ISP, instead of trying to find one specific fish in a barrel of fish, with a blindfold on, your instead just handed a gift-wrapped fish!

Return to “Camera board”