RussC243
Posts: 11
Joined: Sun Apr 22, 2018 11:06 am

Resolved: Onboard camera not working with OpenCV, USB camera works fine

Fri Nov 16, 2018 8:25 pm

Hi,
I am using the onboard V2 camera connected to a Pi 3B+ running the latest sketch image. After entering modprobe bcm2835-v4l2, I can successfully take pictures/video using raspistill and raspivid. The images look great!

I built OpenCV-3.4.3, loaded it to the Pi, plugged in a USB camera for a test and the code below works fine for the USB camera. I can see canny images, detect lines etc. When using the onboard camera, the program hangs when creating the VideoCapture object.

public static void main(String[] args) {
System.loadLibrary("opencv_java343");
Mat mat = Mat.eye(3, 3, CvType.CV_8UC1);
System.out.println("mattt = " + mat.dump());//test the lib. Always prints OK

int camID = 1; //0 is USB cam, 1 is onboard cam
VideoCapture camera = new VideoCapture(camID); //hangs here if onboard cam, USB cam OK
System.out.println("made it past VideoCapture on camera ID " + camID);
//do neat stuff with OpenCV

I suspect the issue may be related to the pixel format. The USB cam default is YUYV. The onboard cam default is YU12. Before running the program, I set the format with v4l2-ctl to YUYV and read back the set value for the onboard camera but the program still hangs when creating the VideoCapture object. So, I ctrl+c the program and notice the setting is back to the default.

I am using java with Netbeans version 9. V4l2 info pasted in below.

Any suggestions how to resolve issue would be great. Perhaps how to enable logging and where the log files would be written to.
Thanks,
Russ

[email protected]:~/NetBeansProjects/VisionPi/dist $ v4l2-ctl --list-formats-ext -d /dev/video0
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUYV 4:2:2
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 352x288
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 176x144
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 160x120
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.200s (5.000 fps)

Index : 1
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.067s (15.000 fps)

[email protected]:~/NetBeansProjects/VisionPi/dist $ v4l2-ctl --list-formats-ext -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'YU12'
Name : Planar YUV 4:2:0
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 1
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUYV 4:2:2
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 2
Type : Video Capture
Pixel Format: 'RGB3'
Name : 24-bit RGB 8-8-8
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 3
Type : Video Capture
Pixel Format: 'JPEG' (compressed)
Name : JFIF JPEG
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 4
Type : Video Capture
Pixel Format: 'H264' (compressed)
Name : H.264
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 5
Type : Video Capture
Pixel Format: 'MJPG' (compressed)
Name : Motion-JPEG
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 6
Type : Video Capture
Pixel Format: 'YVYU'
Name : YVYU 4:2:2
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 7
Type : Video Capture
Pixel Format: 'VYUY'
Name : VYUY 4:2:2
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 8
Type : Video Capture
Pixel Format: 'UYVY'
Name : UYVY 4:2:2
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 9
Type : Video Capture
Pixel Format: 'NV12'
Name : Y/CbCr 4:2:0
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 10
Type : Video Capture
Pixel Format: 'BGR3'
Name : 24-bit BGR 8-8-8
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 11
Type : Video Capture
Pixel Format: 'YV12'
Name : Planar YVU 4:2:0
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 12
Type : Video Capture
Pixel Format: 'NV21'
Name : Y/CrCb 4:2:0
Size: Stepwise 32x32 - 3280x2464 with step 2/2

Index : 13
Type : Video Capture
Pixel Format: 'BGR4'
Name : 32-bit BGRA/X 8-8-8-8
Size: Stepwise 32x32 - 3280x2464 with step 2/2
Last edited by RussC243 on Sun Nov 18, 2018 8:49 am, edited 1 time in total.

il_diavolo
Posts: 136
Joined: Mon Dec 02, 2013 7:56 pm

Re: Onboard camera not working with OpenCV, USB camera works fine

Fri Nov 16, 2018 10:28 pm

Yes, I have the same problem......

viewtopic.php?f=43&t=225539&p=1384459#p1384459

Best of luck with it, I never got any response to the above post. Now working through all my programs to use picamera for the onboard camera, which works, and OpenCV for the USB camera. It's a PITA.

For what it's worth the primary error seems to be "VIDIOC_QBUF: Invalid argument". Despite extensive Googling I have not found a valid solution. The only thing I haven't done is to rebuild OpenCV as that's a PITA as well.

RussC243
Posts: 11
Joined: Sun Apr 22, 2018 11:06 am

Re: Onboard camera not working with OpenCV, USB camera works fine

Sat Nov 17, 2018 3:11 am

IL_dialovo
I think the problem you posted may be a bit different. You said "cap = cv2.VideoCapture(index)" followed by "success,". i.e. VideoCapture returns. I am hanging on this line: "VideoCapture camera = new VideoCapture(camID);" ...never printing the line below it. In your case, you pass then fail on the read. Regardless , I will see what I can learn about VIDIOC_QBUF to see if it is related to my issue. If I make any progress, I will post.

I have to assume there are many many people using the onboard pi camera with OpenCV, Strecth and it is working fine. If anyone can confirm this with any details on the compiler options used to build OpenCV, that would help greatly.

Thanks,
Russ

RussC243
Posts: 11
Joined: Sun Apr 22, 2018 11:06 am

Re: Onboard camera not working with OpenCV, USB camera works fine

Sun Nov 18, 2018 8:48 am

With some help from this forum and the OpenCV forum, my issue is resolved. I will describe my understanding of the issue. The correctness of my description may be a bit lacking as I am new to Linux , Java and OpenCV.

My problem was that I installed gstreamer because I incorrectly thought it was a needed dependency when building OpenCV. (During CMAKE I saw warnings like “failed to find gstreamer”. So I installed gstreamer and the warnings went away so I thought I was one step closer to getting a usable build.)

Two of the imports I have in my Netbeans project are
import org.opencv.videoio.VideoCapture;
import org.opencv.videoio.Videoio;

When you import create VideoCapture object, Videoio 1st tries to use GStreamer regardless of if it is installed or not. If GStreamer is not installed, Videoio tries v4l2. The later is what you want because gstreamer is known to be buggy and not work well with the onboard CSI MIPI Pi camera.

After uninstalling a handful of gstreamer components, NOT loading V4l2 with sudo modprobe bcm2835-v4l2, setting environment variable OPENCV_VIDEOIO_DEBUG=1, then a reboot, I get the following in the SSH putty session:
[ WARN:0] VIDEOIO(createGStreamerCapture(index)): result=(nil) isOpened=-1 ...
[ WARN:0] VIDEOIO(cvCreateCameraCapture_V4L(index)): trying ...
[ WARN:0] VIDEOIO(cvCreateCameraCapture_V4L(index)): result=(nil) ...
When gstreamer was installed, the program would hang on the createGStreamerCapture step. With gstreamer still uninstalled and loading v412, I get the following:
[ WARN:0] VIDEOIO(createGStreamerCapture(index)): result=(nil) isOpened=-1 ...
[ WARN:0] VIDEOIO(cvCreateCameraCapture_V4L(index)): trying ...
[ WARN:0] VIDEOIO(cvCreateCameraCapture_V4L(index)): result=0x6523c7a8 ...

Then camera opens, OpenCV runs fine, I can see live video OpenCV canny images from onboard Pi camera in an xming window on my laptop. At 640x480 I see about 15fps with a few OpenCV test functions applied to each frame(grayscale, blur, canny). This is with Netbeans 9, OpenCV 3.4.3 and Raspbian Stretch.
Thanks for the help.
Russ

peterAgainAgain
Posts: 2
Joined: Mon May 27, 2019 5:36 am

Re: Resolved: Onboard camera not working with OpenCV, USB camera works fine

Mon May 27, 2019 6:30 am

RussC243 your solution is to NOT do a stack of things which suggests the default set-up works fine, but I can't get to stage 1. If you're attending to this perhaps you could answer a few questions? Which version of opencv did you get working and why did you choose it? Did you need to disable ffmpeg to prevent the segmentation fault issue? If so, how? You seem knowledgeable about image processing; any suggestions on cmake parameters for opencv on the pi?

Return to “Camera board”