Kinect - Would it ever work with a Raspberry Pi?


112 posts   Page 2 of 5   1, 2, 3, 4, 5
by maverick7170 » Mon Jun 04, 2012 2:22 pm
Ok, thanks for the heads up! I'll have to pick up a powered hub.

Have you had any luck using the Kinect as a webcam for a simple test? On Arch, loading the uvcvideo module and running luvcview works well with the PS3 webcam. The frame rates are not very good at higher resolution, but that seems to be consistent with other reports on the forum.
Posts: 8
Joined: Mon Nov 21, 2011 1:04 am
by practisevoodoo » Mon Jun 04, 2012 2:30 pm
None at all, apparently if I can get the gspca_kinect module onto the pi then I can use the kinect as a regular webcam. If it is possible to decode the depth image back into distances then that might be enough for my purposes but I've not managed it yet. Basically any time I try to get image data off the kinect my code locks up so at the moment my plan is to go through the freenect library and find the exact line where things lock.
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by AjTech » Tue Jun 05, 2012 9:17 am
Hi all i should hopefully have my pi in 5 weeks so i will be able to work on getting my kinect hooked up to it.

I will document how i go.
Posts: 2
Joined: Sun Jun 03, 2012 11:31 am
by theunit » Tue Jun 05, 2012 2:34 pm
A few people have mentioned the issue of only the Motor showing up, no Audio and Camera.

In my experience you MUST put the Kinect through a powered hub to get these to show up consistently (as well as the Kinect external power supply). The Kinect has it's own USB Hub, the motor, the audio and the camera. I read somewhere that the Motor is actually powered by the USB Bus and NOT the external power supply. This seems a ridiculous design, especially for an inductive load like a motor, but as soon as i plugged it in via hub everything showed up.

My choice of Kinect library is libfreenect: https://github.com/OpenKinect/libfreenect

Compiles no problem on the RPi once you have installed the dependencies.
Posts: 31
Joined: Fri May 11, 2012 7:52 am
by practisevoodoo » Tue Jun 05, 2012 3:37 pm
Yes it compiles and the motor and accelerometer work but are you able to get anything from the cameras? Because I'm not.
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Plum » Sun Jun 10, 2012 12:09 pm
So what I did until now to get my Kinect running (using the current Debian image):

1. powered externally and attached to an externally powered USB hub. Otherwise the cameras are not detected by lsusb
2. compiled libfreenect - worked great without any changes
Code: Select all
apt-get install git-core cmake libglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev
git clone git://github.com/OpenKinect/libfreenect.git
cd libfreenect
mkdir build
cd build
cmake ..
make
make install

3. compiled also opencv as image processing library
Code: Select all
cd opencv
mkdir release
cd release
cmake -D BUILD_opencv_gpu=OFF -D BUILD_TESTS=OFF -D BUILD_PERF_TESTS=OFF -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
make

4. when I run "tiltdemo" the system's wireless connection (USB dongle) is getting crappy and the raspberry pi takes a very long time (several seconds) to get the tilt data from the camera
5. when I run the demos accessing the camera device it takes forever during initialisation of the camera...

Any ideas how to solve 4 and 5?
Posts: 4
Joined: Sun Jun 10, 2012 12:04 pm
by practisevoodoo » Sun Jun 10, 2012 4:20 pm
Ok but are you actually getting the image data off the camera? You say that it takes forever to initialise the camera, how long are we talking? I've been leaving it for a couple of hours and still not getting a response.
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Plum » Sun Jun 10, 2012 6:06 pm
I lose my network connection (which will never get up after that)... I isolated the "critical" line in the code and found out it is an timed out (very slow) libusb call. It seems the camera doesn't answer or very delayed (because sometimes it gets over the call after a few minutes and then hangs on the next line).

I'm currently installing another i386 based debian system on another machine to try if my hardware is working properly.
Posts: 4
Joined: Sun Jun 10, 2012 12:04 pm
by Plum » Sun Jun 10, 2012 6:12 pm
Works on the i386 based system ==> so kinect hardware is ok!
Posts: 4
Joined: Sun Jun 10, 2012 12:04 pm
by Plum » Mon Jun 11, 2012 6:05 pm
Update (sorry about posting again): I think I have isolated the problem. I attached the board to an ethernet and also noticed that although low cpu load my system starts to "hang" during initilization of kinect (yes it initializes without USB WiFi attached - although power supply for my USB hub is able to support 5A). If I start one of my test tools for kinect the vmstat shows me a high interrupt load (~20000), which seems to be some kind of limit.

cat /proc/interrupts gives me
Code: Select all
           CPU0
  3:     549629   ARMCTRL  BCM2708 Timer Tick
 65:          2   ARMCTRL  ARM Mailbox IRQ
 66:          1   ARMCTRL  VCHIQ doorbell
 75:  235616157   ARMCTRL  dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
 77:      14910   ARMCTRL  bcm2708_sdhci (dma)
 83:         10   ARMCTRL  uart-pl011
 84:      49731   ARMCTRL  mmc0
Err:          0


Is this limit of 20k somehow CPU-specific or is it somewhere configurable in the kernel, because I think this may be the source - or at least one of them - of my problem. I still get no picture data.stream_process inside cameras.c of libfreenect tells me invalid magic. I will investigate this issue now, but help with my interrupt problem would be highly appreciated.
Posts: 4
Joined: Sun Jun 10, 2012 12:04 pm
by Dimi » Fri Jun 22, 2012 7:50 am
Hello!

I also could compile everything but there seems no way to activate the color or depth stream. Somehow the OpenGL seems to make Problems. Tiltdemo is working without problems, also writing and compiling my own code is working, as long as I´m not using the GL components.

I got a "Write Reg 0x0105 <= 0x00 Error" when I try to start the ./glview.

Any clue?

thx Dimi
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by theunit » Fri Jun 22, 2012 9:47 am
Dimi wrote:Somehow the OpenGL seems to make Problems.


This is the conclusion I came to. I wrote some code to access the depth stream without GL but I haven't tested it yet (I was borrowing a Kinect and i no longer have it)

Were you able to access the depth and colour streams with your non-GL code? I think I read somewhere there is a problem with freeglut on the RPi.
Posts: 31
Joined: Fri May 11, 2012 7:52 am
by practisevoodoo » Fri Jun 22, 2012 10:19 am
I modified the tiltdemo code to try and grab the depth and camera images and it still doesn't work. As soon as the code gets to either of the camera calls it will just hang.
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Dimi » Fri Jun 22, 2012 10:25 am
@theunit
You can send me your code and I can try it out.
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by Dimi » Fri Jun 22, 2012 12:42 pm
theunit wrote:
Dimi wrote:Somehow the OpenGL seems to make Problems.


This is the conclusion I came to. I wrote some code to access the depth stream without GL but I haven't tested it yet (I was borrowing a Kinect and i no longer have it)

Were you able to access the depth and colour streams with your non-GL code? I think I read somewhere there is a problem with freeglut on the RPi.


can you send me the code and I will try to run it?

I´m not using freeglut, but libglut3-dev. Seems if there is something missing in the libs for our debian version and thats why is hanging and waiting there?
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by practisevoodoo » Fri Jun 22, 2012 1:33 pm
Unless I am very badly mistaken the libraries don't use GLUT to get hold of the image data from the kinect, just to display the data once it is collected.
The problem isn't in GLUT, the problem is that the raspberry can't get the image data to display in the first place.

Try running this code to see what I mean
http://pastebin.com/a6NgR8L2
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Dimi » Tue Jun 26, 2012 10:52 am
practisevoodoo wrote:
Try running this code to see what I mean
http://pastebin.com/a6NgR8L2


Hey, sorry for the delay. I tried your code, but no luck. It stops with errors at the point he tries to access the video stream. In some other posts I´ve read that it could be a problem with the libusb, but I couldn´t compile it myself yet.

cheers
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by practisevoodoo » Tue Jun 26, 2012 11:50 am
I have seen people claiming that they have gotten it working by compiling lsusb and I have been getting in touch of them to confirm that by ' working ' they mean that they can actually get the image data and so far by 'working ' they have either meant that the pi is now detected and they can run tilt demo or they have not replied. At the moment I don't think that anyone has actually gotten the kinect working on the pi although I would love to be proved wrong
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Dimi » Tue Jun 26, 2012 11:58 am
ah, ok! good to know! :) and I was wondering why it isn´t working with my device ;)
maybe its because the hardware limitations? although the depth-calculating is happening with the kinect hardware, it could be possible, that it just won´t run on our Pis?

Now I´m wondering if an overclocked Pi, would change this :?:
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by practisevoodoo » Tue Jun 26, 2012 12:45 pm
I really doubt that the pi can't get the data from the kinect from a hardware standpoint. At the moment I can't even get the normal camera data off the device and that is just a bog standard webcam and those DO work on the pi. I'm planning on going through the freenect code to find out exactly where in the internals it is getting stuck but I just haven't had the time yet.
Posts: 19
Joined: Thu May 10, 2012 12:52 pm
by Dimi » Tue Jun 26, 2012 12:48 pm
good luck for that! let us then know your progress :)
Posts: 13
Joined: Fri Jun 22, 2012 7:45 am
by nadirkazan » Fri Jun 29, 2012 12:59 am
I'm working on inexpensive LiDAR,

http://www.kickstarter.com/projects/110 ... ar-project

so follow the progress
Posts: 4
Joined: Sat Jun 16, 2012 5:33 am
by theunit » Wed Jul 04, 2012 3:17 pm
I have just got the ASUS Xtion sensor working on the raspberry pi using OpenNI. The OpenNI SimpleRead example works and outputs the depth of the middle pixel. I can't see any reason why the Kinect would not also work with OpenNI.

There are a few things you must do to make OpenNI compile (and compilation takes a loooong time):

This helped me with some of it:
http://www.kinecthesia.com/2012/02/lights-camera-xtion-getting-asus-xtion.html

That link says edit RedistMaker but that just redirects to Redist_OpenNi.py which is the file you need to edit. Change the machinetype to "arm" and also remove the line which adds the "-j" flag to make. For some reason it thinks the RPi has 0 cores and "-j0" doesn't work.

Edit CFLAGS in Build/Common/Platform.Arm for the raspberry pi:
Code: Select all
CFALGS += -mtune=arm1176jzf-s -mfpu=vfp -mfloat-abi=softfp

(and remove the line referring to armv9 and cortex-a8)

The same changes need to be made to compile the PrimeSense/Sensor parts as well.
Posts: 31
Joined: Fri May 11, 2012 7:52 am
by theunit » Wed Jul 04, 2012 4:26 pm
I tried compiling the Kinect drivers for OpenNI from https://github.com/avin2/SensorKinect/ but the compile failed due to what seems like some missing header files.
Posts: 31
Joined: Fri May 11, 2012 7:52 am
by AnalogArsonist » Sun Jul 08, 2012 4:35 pm
theunit wrote:I tried compiling the Kinect drivers for OpenNI from https://github.com/avin2/SensorKinect/ but the compile failed due to what seems like some missing header files.


theunit, did you use the ubuntu binaries or the ARM binaries? I'm wondering if we have to use the ARM since technically that's what the RPi is. I'm a *nix noob so I'm a little out of my league but wanted to try the OpenNI drivers as well.

Thanks, Ben
Posts: 5
Joined: Sat Mar 10, 2012 3:45 am