Video capture software - Ubuntu Arm to Rpi?


21 posts
by Steep » Tue Aug 21, 2012 11:11 pm
This is a beginners question really, I have a video camera I use with my telescope ( a Point Grey Chameleon ) for which Point Grey have written capture software for the Beagle board. I'd like to try and make it work on my RPi, what would I need to do to convert it from Ubuntu Arm7 to Debian Arm?

I have no Linux system except the RPi but could if needed install on a spare PC.
Posts: 145
Joined: Fri May 18, 2012 8:03 pm
by ghans » Wed Aug 22, 2012 2:13 pm
If it is open source , it should just be a configure / make away.
Else the company will have to pay a programmer / linux guy to port it over , and then think about
how to get back this additional expense (from customers like you perhaps ?) .

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 4449
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by Steep » Thu Aug 23, 2012 12:48 am
Thanks, I guess my first port of call should be PG, I'll try them.
Posts: 145
Joined: Fri May 18, 2012 8:03 pm
by ghans » Fri Aug 24, 2012 6:00 am
Was it open source anyway ?

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 4449
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by trbotha » Wed Dec 19, 2012 4:15 pm
The Point Grey API (Flycapture) is not open source and I have querried them whether they can make it available on ARMv6, but no luck. However, Point Grey USB cams use IIDC standard used for 1394 (firewire) cameras. I have succesfully obtained images from two Point Grey chameleon cameras simultaneously using the Rpi and the libdc1394 library which is open source.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by ghans » Wed Dec 19, 2012 9:50 pm
Good to know , but i don't understand the second part of your post:
Is there a USB Device "category" or standard for FireWire
cameras ?


ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 4449
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by trbotha » Thu Dec 20, 2012 8:04 am
There is no USB device standard for Firewire interface. However the Point Grey USB cams dont use the standard Webcam interface, they are compliant with IIDC 1934 DCAM interface. The libdc1394 library thus uses libusb (specific versions are required for both libraries latest libs work fine) to communicate with the USB Point Grey Cameras. Thus, any command issued via the lidc1394 API is transmitted via USB and the Point Grey USB cameras understands this command. There may be some commands that are specific to the 1394 bus that are not capable on the USB bus. One I believe is the synching of the cameras using the bus. However, the Point Grey cameras both support Software and Hardware triggers for synching. Thus, this is only applicable to Point Grey USB cameras (there may be others check the list of IIDC DCAM compliant hardware). Any other USB cameras should be compliant with the same standard used on most webcams.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by rasmus25 » Mon Feb 04, 2013 4:00 pm
Hi, I am also considering the Chameleon+RPi combo. Were you able to get any video out of the camera or just still frames? The RPi USB interface seems to be a bit slow. I wonder if 15 FPS at 800x600 would be doable on Pi?

Some day I would like to use Pi's hardware encoding capabilities to record hours of video from the Point Grey Chameleon.
Posts: 13
Joined: Mon Feb 04, 2013 3:53 pm
by trbotha » Mon Feb 04, 2013 6:39 pm
I can capture video as well. I currently capture two chameleon cams using as fast as possible sofware synching (required for stereo vision) so not consistent fps streaming per say. However, I can currently capture two chameleons at 640x480 grayscale resolution at around 25-30fps. The biggest problem I had was not the USB speed, but rather the SD memory card write speed since I started out writing raw images to the SD card and that was the limit. I make use of hardware H264 encodingto speed up capturing. I transform the gray to raw rgb (since the encoder only except rgb and yuv input streams no gray as yet) then hardware encode to h264 and add the h264 header afterwards. The limit is then really the conversion from gray to rgb and other slight cpu computing part. If you were to use colour the slowest bit will most likely be the bayer image to rgb conversion. Not sure if bayer inputs are accpeted by the harware encoding.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by ghans » Mon Feb 04, 2013 6:55 pm
Can you share with us how you realised it ?
How did you use OpenMAX and how did you interface
the camera in your setup ? Did you create custom software ?

ghans
• Don't like the board ? Missing features ? Change to the prosilver theme ! You can find it in your settings.
• Don't like to search the forum BEFORE posting 'cos it's useless ? Try googling : yoursearchtermshere site:raspberrypi.org
Posts: 4449
Joined: Mon Dec 12, 2011 8:30 pm
Location: Germany
by trbotha » Mon Feb 04, 2013 7:07 pm
I'm willing to share my code, since alot of it based on other people's work. However, I'll just write some comments before I do so. I used OMX based on the example hello_video just wrote a c++ class for it and for the point grey cams. Some of the code is hardware coded since I only spend a few hours on the code. The camera class needs alot of change settings code since I mainly plugged the cameras into my laptop set them up with flycapture and saved the settings on the cameras. It should be easy to add functions to change exposure etc.. And again I'm using software triggering not streaming but this should be an easy mod aswell.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by rasmus25 » Tue Feb 05, 2013 10:09 am
Good job! I would be very much interested in the code. I am not in a hurry, as I haven't ordered the camera yet, but it is encouraging to know that the idea has potential. I guess the color conversion could not be avoided anyways, even with typical webcams the color format of raw images is not quite what encoders expect.
Posts: 13
Joined: Mon Feb 04, 2013 3:53 pm
by trbotha » Wed Feb 06, 2013 4:44 pm
I have uploaded the source code to https://github.com/trbotha/PointGrey_RPi

Again note that the code is not complete. For the code to be used the camera needs to be setup to work in software trigger mode, the prototype function is provided for this but is commented out. The camera can be setup using Flycapture software from PointGrey simply save the changes on the first memmory channel of the camera. I will add code later to setup the camera from the source code when I review the program later, since I will use the program later in a project, for now people can play around with the code if they want. In main.cpp the program only capture 100 frames using a for loop simply change this to a while loop with some kind of keyhit function to exit.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by trbotha » Thu Feb 14, 2013 9:31 am
I had time to work on the code again, the cameras can be set into streaming mode aswell, the shutter opening time can also be specified. Other settings can easily be added as well using the dc1394 library api. The code also auto sets up the amount of cameras on the bus ie.e 1 or 2 cameras. More may be possible depending on bus limitations however only upto two have been tested thus far. Two chaemleons cameras were tested at streaming 30fps succesfully, this I believe is the limit of the cameras as well. I have not tested colour as yet, but bayer conversion algorithms are available from the dc1394 library as well.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by rasmus25 » Tue Feb 26, 2013 5:26 pm
I received my camera yesterday and experimented with color video. Results are good.

trbotha, I forked your code to https://github.com/rasmus25/PointGrey_RPi and modified it to get color video out. I used libdc1394 debayerization functions to get RGB images. Current code is hard-coded to receive 1280x960 bayer images from camera and use downsample debayerization to get a 640x480 image out of it, which is then encoded in GPU. I used downsampling, because it seems to me that this is the easiest way to do color pixel binning (using Chameleons built-in pixel binning produces grayscale output, not bayer). Framerate is around 17.5, as expected from streaming 1280x960 video over USB2.0, CPU usage is around 70%. This is probably because debayerization runs on CPU. It should not be too hard to write an OpenGL shader that does downsampling debayerization on the GPU.
Posts: 13
Joined: Mon Feb 04, 2013 3:53 pm
by RomanG » Sun Apr 07, 2013 5:00 pm
Hi trbotha,

I have Point Grey Research FireFly MV FMVU-03MTM mono camera and I would like to get it going on pi.
Was very happy to find this thread (the only one dealing with PointGrey cameras on rpi).

Trying to compile your code on my rpi (Raspbian 3.6.11+ #371 PREEMPT) I run in to some missing libraries (ilclient and camwire).

Using your code, how can be video stream from the camera captured?
Just saved to the file or can it be also continuous live stream?

Thanks,
Roman
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada
by RomanG » Mon Apr 08, 2013 4:16 am
I compiled and installed camwire and then I was able to compile your GPU-Encode.
When I run it as a regular user, I get Segmentation fault error:
Code: Select all
pi@RPi /usr/src/PointGrey_RPi $ ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
libdc1394 warning: usb: Failed to open device for config ROM
libdc1394 warning: Failed to get config ROM from usb device
libdc1394 error: No cameras found.

Segmentation fault


When I run it as root, the GPU-encode is able to query camera for parameters and then displays prompt, Press S to start and stop recording
Code: Select all
pi@RPi /usr/src/PointGrey_RPi $ sudo ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
Using camera with GUID 49712223531685979
Shutter value of cam 0 is 480
Get Camera Features
------ Features report ------
OP   - one push capable
RC   - readout capable
O/OC - on/off capable
AC   - auto capable
MC   - manual capable
ABS  - absolute capable
-----------------------------
Brightness:
        RC  MC  (active is: MAN)  AC  (active is: MAN)
        min: 1 max 255
        current value is: 134
Exposure:
        O/OC  MC  (active is: MAN)  AC  (active is: MAN)
        Feature: OFF  min: 7 max 62
        current value is: 15
Sharpness:
        NOT AVAILABLE
White Balance:
        NOT AVAILABLE
Hue:
        NOT AVAILABLE
Saturation:
        NOT AVAILABLE
Gamma:
        RC  O/OC  MC  (active is: MAN)
        Feature: OFF  min: 0 max 1
        current value is: 0
Shutter:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 1 max 2048
        current value is: 480
        absolute settings:
         value: 0.029988
         min: 0.000062
         max: 0.127948
Gain:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 16 max 64
        current value is: 64
        absolute settings:
         value: 12.041193
         min: 0.000000
         max: 12.041193
Iris:
        NOT AVAILABLE
Focus:
        NOT AVAILABLE
Temperature:
        NOT AVAILABLE
Trigger:
        RC  O/OC
        Feature: ON
        AvailableTriggerModes: 384 387
        AvailableTriggerSources: 576 579 580
        Polarity Change Capable: True
        Current Polarity: NEG
        current mode: 384

        current source: 580
Trigger Delay:
        RC  O/OC  MC  (active is: MAN)  ABS
        Feature: OFF  min: 0 max 4095
        current value is: 0
        absolute settings:
         value: 0.000000
         min: 0.000000
         max: 65.000000
White Shading:
        NOT AVAILABLE
Frame Rate:
        RC  O/OC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        Feature: OFF  min: 466 max 2258
        current value is: 480
        absolute settings:
         value: 30.000000
         min: 4.604834
         max: 30.430416
Zoom:
        NOT AVAILABLE
Pan:
        RC  O/OC  MC  (active is: AUTO)  AC  (active is: AUTO)
        Feature: ON  min: 0 max 112
        current value is: 56
Tilt:
        NOT AVAILABLE
Optical Filter:
        NOT AVAILABLE
Capture Size:
        NOT AVAILABLE
Capture Quality:
        NOT AVAILABLE
 nSlice = 480
 nStride = 640
Bitrate is = 5000000
encode to idle...
enabling port buffers for 200...
enabling port buffers for 201...
encode to executing...

Filename is file_Cam0.h264

Press S to start and stop recording.


When I press S, the session hangs (I am doing this trough ssh).
I have to restart the session and kill the GPU-Encode.

Running it directly on pi yields same results.

The video file *.h264 is created, but it has 0b size.

Any suggestions?

Thanks a lot,
Roman
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada
by trbotha » Thu May 09, 2013 5:31 pm
Hi RomanG

Sorry for the late reply, If you still having problems try to remove any function in main.cpp with the word "terminal" in it i.e. reset_terminal and set_conio_terminal etc. These functions are used to provide getch characteristics such that carriage return (enter) does not need to be entered after a letter. I have experienced at times that the the program does not get the character input. If you remove these lines that will remove this problem, however, you will then need to type 's' or 'S' and then 'Enter' for both start and stop.

Hope this helps
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm
by RomanG » Sat May 11, 2013 11:26 pm
Thanks trbotha,

I will try as you suggested.

Roman
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada
by RomanG » Sat May 11, 2013 11:37 pm
OK, I commented out any functions related to terminal and also any calls to these functions.

When I run GPU-Encode as sudo, I get following:
Code: Select all
pi@RPi /usr/src/PointGrey_RPi $ sudo ./GPU-Encode
No File name supplied will use deafult = file*.mp4
string is shutter
shutter is 10.000000
Using camera with GUID 49712223531685979
Shutter value of cam 0 is 480
Get Camera Features
------ Features report ------
OP   - one push capable
RC   - readout capable
O/OC - on/off capable
AC   - auto capable
MC   - manual capable
ABS  - absolute capable
-----------------------------
Brightness:
        RC  MC  (active is: MAN)  AC  (active is: MAN)
        min: 1 max 255
        current value is: 146
Exposure:
        O/OC  MC  (active is: MAN)  AC  (active is: MAN)
        Feature: OFF  min: 7 max 62
        current value is: 7
Sharpness:
        NOT AVAILABLE
White Balance:
        NOT AVAILABLE
Hue:
        NOT AVAILABLE
Saturation:
        NOT AVAILABLE
Gamma:
        RC  O/OC  MC  (active is: MAN)
        Feature: OFF  min: 0 max 1
        current value is: 0
Shutter:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 1 max 2048
        current value is: 480
        absolute settings:
         value: 0.029988
         min: 0.000062
         max: 0.127948
Gain:
        RC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        min: 16 max 64
        current value is: 64
        absolute settings:
         value: 12.041193
         min: 0.000000
         max: 12.041193
Iris:
        NOT AVAILABLE
Focus:
        NOT AVAILABLE
Temperature:
        NOT AVAILABLE
Trigger:
        RC  O/OC
        Feature: ON
        AvailableTriggerModes: 384 387
        AvailableTriggerSources: 576 579 580
        Polarity Change Capable: True
        Current Polarity: NEG
        current mode: 384

        current source: 580
Trigger Delay:
        RC  O/OC  MC  (active is: MAN)  ABS
        Feature: OFF  min: 0 max 4095
        current value is: 0
        absolute settings:
         value: 0.000000
         min: 0.000000
         max: 65.000000
White Shading:
        NOT AVAILABLE
Frame Rate:
        RC  O/OC  MC  (active is: MAN)  AC  (active is: MAN)  ABS
        Feature: OFF  min: 466 max 2258
        current value is: 480
        absolute settings:
         value: 30.000000
         min: 4.604834
         max: 30.430416
Zoom:
        NOT AVAILABLE
Pan:
        RC  O/OC  MC  (active is: AUTO)  AC  (active is: AUTO)
        Feature: ON  min: 0 max 112
        current value is: 56
Tilt:
        NOT AVAILABLE
Optical Filter:
        NOT AVAILABLE
Capture Size:
        NOT AVAILABLE
Capture Quality:
        NOT AVAILABLE
 nSlice = 480
 nStride = 640
Bitrate is = 5000000
encode to idle...
enabling port buffers for 200...
enabling port buffers for 201...
encode to executing...

Filename is file_Cam0.h264

Press S to start and stop recording.


Pressing S seems to start recording (the CPU load goes up) and pressing S again stops recording (CPU load goes down). It does not hang up the RPi

However, the *.mp4 file has 0 bytes size, same as before.

Roman
Posts: 41
Joined: Fri Aug 17, 2012 11:56 am
Location: Ottawa, Canada
by trbotha » Sun May 12, 2013 2:25 pm
The last problem that it could be is that of the triggering, since I see the default in my code seems to use software triggering. Try and umcomment line 67 and comment line 68 in main.cpp. This will swicth the mode to streaming mode instead of software triggering. The software triggering is not neccessary since I use it to sync two cameras for stereovision and is therefore not needed in your code.

If you wish to use streaming look at rtmp servers, I have personally not used it before, you should be able to use the file being created for the server.
Posts: 8
Joined: Wed Dec 19, 2012 4:08 pm