nigel2021nz
Posts: 8
Joined: Mon May 10, 2021 4:41 am

Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Mon May 31, 2021 6:14 am

Hi there

I was wanting to use the arducam HQ camera module because of the M12 mount but I am having trouble building Libcamera.

I am following the instructions at https://www.raspberrypi.org/documentati ... /README.md

But I run into two issues

Firstly with the qcam test I get the following

Code: Select all

pi@raspberrypi:~/libcamera $ build/src/qcam/qcam
libEGL warning: DRI2: failed to authenticate
[0:01:49.472102855] [948]  INFO IPAManager ipa_manager.cpp:136 libcamera is not installed. Adding '/home/pi/libcamera/build/src/ipa' to the IPA search path
[0:01:49.542849583] [948]  INFO Camera camera_manager.cpp:294 libcamera v0.0.0+2639-3370b6d0
[0:01:49.555701256] [958]  WARN CameraSensor camera_sensor.cpp:196 'imx477 10-001a': Recommended V4L2 control 0x009a0922 not supported
[0:01:49.556240293] [958]  WARN CameraSensor camera_sensor.cpp:248 'imx477 10-001a': The sensor kernel driver needs to be fixed
[0:01:49.556378089] [958]  WARN CameraSensor camera_sensor.cpp:250 'imx477 10-001a': See Documentation/sensor_driver_requirements.rst in the libcamera sources for more information
[0:01:49.564493433] [958]  WARN CameraSensorProperties camera_sensor_properties.cpp:64 No static properties available for 'imx477'
[0:01:49.564644692] [958]  WARN CameraSensorProperties camera_sensor_properties.cpp:66 Please consider updating the camera sensor properties database
[0:01:49.565265766] [958]  WARN CameraSensor camera_sensor.cpp:376 'imx477 10-001a': Failed to retrieve the camera location
[0:01:49.796079099] [958]  INFO IPAProxy ipa_proxy.cpp:129 libcamera is not installed. Loading IPA configuration from '/home/pi/libcamera/src/ipa/raspberrypi/data'
[0:01:50.057740730] [948]  WARN V4L2 v4l2_pixelformat.cpp:180 Unsupported V4L2 pixel format XB24
[0:01:50.059032766] [948]  INFO Camera camera.cpp:905 configuring streams: (0) 800x600-BGR888
[0:01:50.059680599] [958]  INFO RPI raspberrypi.cpp:121 Mode: 1332x990 fmt RG10 Score: 1735.05 (best 1735.05)
[0:01:50.059944969] [958]  INFO RPI raspberrypi.cpp:121 Mode: 4056x3040 fmt RG12 Score: 1924.33 (best 1735.05)
[0:01:50.060047284] [958]  INFO RPI raspberrypi.cpp:121 Mode: 2028x1520 fmt RG12 Score: 1037.33 (best 1037.33)
[0:01:50.060222765] [958]  INFO RPI raspberrypi.cpp:121 Mode: 2028x1080 fmt RG12 Score: 1131.17 (best 1037.33)
[0:01:50.060485950] [958]  INFO RPI raspberrypi.cpp:121 Mode: 1332x990 fmt pRAA Score: 1235.05 (best 1037.33)
[0:01:50.060655283] [958]  INFO RPI raspberrypi.cpp:121 Mode: 4056x3040 fmt pRCC Score: 1424.33 (best 1037.33)
[0:01:50.060723154] [958]  INFO RPI raspberrypi.cpp:121 Mode: 2028x1520 fmt pRCC Score: 537.329 (best 537.329)
[0:01:50.060892987] [958]  INFO RPI raspberrypi.cpp:121 Mode: 2028x1080 fmt pRCC Score: 631.167 (best 537.329)
[0:01:50.061363857] [958]  INFO RPI raspberrypi.cpp:618 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected mode: 2028x1520-pRCC
Zero-copy enabled
[0:01:50.094459179] [958]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:01:50.094537382] [958]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
[0:01:50.094778438] [958]  INFO RPISTREAM rpi_stream.cpp:122 No buffers available for ISP Output0
qcam: ../src/ipa/raspberrypi/cam_helper_imx477.cpp:120: virtual RPiController::MdParser::Status MdParserImx477::Parse(libcamera::Span<const unsigned char>): Assertion `num_lines_ || buffer_size_bytes_' failed.
Aborted
as per previous post topics disable_touchscreen is set to 1 in config.txt.

The second issue I'm seeing is when I attempt to 'make' the libcamera-apps I get the following errors and compile fail.

Code: Select all

[ 55%] Building CXX object CMakeFiles/libcamera-vid.dir/libcamera_vid.cpp.o
[ 55%] Building CXX object CMakeFiles/libcamera-jpeg.dir/libcamera_jpeg.cpp.o
[ 59%] Building CXX object CMakeFiles/libcamera-hello.dir/libcamera_hello.cpp.o
[ 62%] Building CXX object CMakeFiles/libcamera-still.dir/libcamera_still.cpp.o
In file included from /home/pi/libcamera-apps/libcamera_jpeg.cpp:10:
/home/pi/libcamera-apps/libcamera_app.hpp: In member function ‘void LibcameraApp<OPTIONS>::StartCamera()’:
/home/pi/libcamera-apps/libcamera_app.hpp:367:37: error: ‘FrameDurations’ is not a member of ‘controls’
   if (!controls_.contains(controls::FrameDurations))
                                     ^~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make[2]: *** [CMakeFiles/libcamera-jpeg.dir/build.make:63: CMakeFiles/libcamera-jpeg.dir/libcamera_jpeg.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:88: CMakeFiles/libcamera-jpeg.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
In file included from /home/pi/libcamera-apps/libcamera_hello.cpp:11:
/home/pi/libcamera-apps/libcamera_app.hpp: In member function ‘void LibcameraApp<OPTIONS>::StartCamera()’:
/home/pi/libcamera-apps/libcamera_app.hpp:367:37: error: ‘FrameDurations’ is not a member of ‘controls’
   if (!controls_.contains(controls::FrameDurations))
                                     ^~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
[ 66%] Building CXX object CMakeFiles/libcamera-still.dir/yuv.cpp.o
make[2]: *** [CMakeFiles/libcamera-hello.dir/build.make:63: CMakeFiles/libcamera-hello.dir/libcamera_hello.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:224: CMakeFiles/libcamera-hello.dir/all] Error 2
[ 70%] Building CXX object CMakeFiles/libcamera-still.dir/dng.cpp.o
In file included from /home/pi/libcamera-apps/libcamera_encoder.hpp:8,
                 from /home/pi/libcamera-apps/libcamera_vid.cpp:14:
/home/pi/libcamera-apps/libcamera_app.hpp: In member function ‘void LibcameraApp<OPTIONS>::StartCamera()’:
/home/pi/libcamera-apps/libcamera_app.hpp:367:37: error: ‘FrameDurations’ is not a member of ‘controls’
   if (!controls_.contains(controls::FrameDurations))
                                     ^~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make[2]: *** [CMakeFiles/libcamera-vid.dir/build.make:63: CMakeFiles/libcamera-vid.dir/libcamera_vid.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:170: CMakeFiles/libcamera-vid.dir/all] Error 2
[ 74%] Building CXX object CMakeFiles/libcamera-still.dir/png.cpp.o
In file included from /home/pi/libcamera-apps/libcamera_still.cpp:15:
/home/pi/libcamera-apps/libcamera_app.hpp: In member function ‘void LibcameraApp<OPTIONS>::StartCamera()’:
/home/pi/libcamera-apps/libcamera_app.hpp:367:37: error: ‘FrameDurations’ is not a member of ‘controls’
   if (!controls_.contains(controls::FrameDurations))
                                     ^~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
make[2]: *** [CMakeFiles/libcamera-still.dir/build.make:63: CMakeFiles/libcamera-still.dir/libcamera_still.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:197: CMakeFiles/libcamera-still.dir/all] Error 2
make: *** [Makefile:130: all] Error 2


I'm wondering if this is because the Arducam HQ camera module is not supported via Libcamera or do I have another issue?
The Arducam HQ module does work with Picamera.

I'm running Linux raspberrypi 5.10.39-v7l+ #1421 SMP Tue May 25 11:02:26 BST 2021 armv7l GNU/Linux

Any advice much appreciated.

kind regards
Nigel.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 199
Joined: Tue Jan 07, 2020 9:15 am

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Mon May 31, 2021 8:36 am

Hi, thanks for the question. I think there are 2 distinct things going on here.

Firstly, as you have guessed, 3rd party imx477 camera boards (also 3rd party imx219 boards) are not supported (except on the compute module). I don't know if dmesg will report anything in that case, but you could have a look. But if you're using a compute module, or an official imx477, and it's still not working, please let us know.

Secondly, there's been a small change in the libcamera API which is causing the build error in libcamera-apps (libcamera has not yet reached an officially stable API version yet). You would need to edit libcamera_app.hpp and replace "FrameDurations" by "FrameDurationLimits". This change seems to have happened a few days ago (to be fair, I did know it was "on its way"). I'll patch it up in our official repo in the next day or so.

nigel2021nz
Posts: 8
Joined: Mon May 10, 2021 4:41 am

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Tue Jun 01, 2021 4:51 am

Hi, thanks for the reply.

I can confirm everything is fine with a Pi v2 IMX219 camera thanks. Libcamera working well.
I will try with a pi HQ camera when I get a chance.

One further question related to the Innomaker IMX327 raw camera module.

I can get that to run with libcamera-hello and libcamera-vid at 30 fps. Is it possible to achieve a higher frame rate?
With the innomaker driver it runs at 60fps raw output thru v4l2.
I'm guessing maybe the IMX290 driver max's out at 30fps maybe? Do you know of any work around for this?
Tho even if the camera was running 1920x1080 @ 60fps I would hit a wall with the ISP throughput?

Sorry, that was more than one question.

kind regards
Nigel.

therealdavidp
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 199
Joined: Tue Jan 07, 2020 9:15 am

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Tue Jun 01, 2021 9:50 am

Hi again

So with the imx327 we use this open source driver: https://github.com/raspberrypi/linux/bl ... c/imx290.c

We've had to modify it so that it works at all, but it wasn't originally developed by us. It's not terribly clear exactly what the limits on framerates are, and I'm not entirely sure if it's handling them all correctly. I don't suppose Innomaker publish the register sets their driver uses so that someone could compare? It sounds like more work is needed here, but I don't know of any easy solutions.

1080p60 would be fine for the Pi 4 ISP (you'd have to disable the colour denoise feature to get that throughput, however). It probably somewhat exceeds what you might get from the the h.264 video encoder, though I think people sometimes get close.

nigel2021nz
Posts: 8
Joined: Mon May 10, 2021 4:41 am

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Wed Jun 02, 2021 5:08 am

Let me see what I can find out re the Innomaker driver for the 327. Unfortunately I don't think they expose the register set tho.

thanks
Nigel.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11475
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Wed Jun 02, 2021 2:57 pm

therealdavidp wrote:
Tue Jun 01, 2021 9:50 am
So with the imx327 we use this open source driver: https://github.com/raspberrypi/linux/bl ... c/imx290.c
I have datasheets for the IMX327 and IMX290. Frame rates are largely fixed based on resolution, but there is a quirk with the imx290 driver that dropping from 4 to 2 lanes drops the framerate by a factor of 4 instead of 2.
therealdavidp wrote:We've had to modify it so that it works at all, but it wasn't originally developed by us. It's not terribly clear exactly what the limits on framerates are, and I'm not entirely sure if it's handling them all correctly. I don't suppose Innomaker publish the register sets their driver uses so that someone could compare? It sounds like more work is needed here, but I don't know of any easy solutions.
Don't hold your breath. They're ignoring my issue regarding them breaking the GPLv2 licence on modules for which Raspberry Pi hold the copyright - https://gitee.com/inno-maker/cam-mipi32 ... ues/I22HTM
therealdavidp wrote:1080p60 would be fine for the Pi 4 ISP (you'd have to disable the colour denoise feature to get that throughput, however). It probably somewhat exceeds what you might get from the the h.264 video encoder, though I think people sometimes get close.
You'll struggle to get 1080p60 from the H264 block, particularly as it reuses the ISP for input format conversion.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11475
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Fri Jun 18, 2021 9:02 am

FYI I've merged changes to the imx290/327 driver to enable 60fps in all configurations, and therealdavidp has them running happily under libcamera at that rate - https://github.com/raspberrypi/linux/pull/4398
https://github.com/raspberrypi/linux/pu ... 1aa87ccd2c commit text gives the history of why 2 lane mode ended up only doing 30fps.

I may even build up the enthusiasm to upstream those patches as they are generic, but they need a little reworking as one of my patches is a fixup to an earlier one.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

nigel2021nz
Posts: 8
Joined: Mon May 10, 2021 4:41 am

Re: Libcamera and Arducam 12MP IMX477 Mini High Quality Camera Module for Raspberry Pi

Fri Jun 25, 2021 9:25 am

Hi 6by9

Thanks for letting me know, that's great.

kind regards
Nigel.

Return to “Camera board”