Re: Kinect - Would it ever work with a Raspberry Pi?
I've managed to compile the freenect libraries (didn't bother with the demos etc) as well as libusb from source. I've managed to get a hacked version of the tiltdemo.c program to work, by not using the sync commands but using a proper contex from the device. I will see if I can get video later today (going to have to write my own OpenGL / EGL wrapper) as the glut demos will not work under the pi.
I'm uploading a video of the tilt working to youtube now will add a link once done.
I'm uploading a video of the tilt working to youtube now will add a link once done.
-
- Posts: 19
- Joined: Thu May 10, 2012 12:52 pm
Re: Kinect - Would it ever work with a Raspberry Pi?
Can you post the code?
Re: Kinect - Would it ever work with a Raspberry Pi?
Let me tidy it up a bit first, it's a major hack
Here is the video of it working http://www.youtube.com/watch?v=UfmiPEVQ ... e=youtu.be

Here is the video of it working http://www.youtube.com/watch?v=UfmiPEVQ ... e=youtu.be
Re: Kinect - Would it ever work with a Raspberry Pi?
jmacey, I found the freenect tiltdemo example to work fine without any hacks. I did write some code to grab a depth image from kinect and save it to file - this seemed like the most basic program - but I have not yet tested it. No-one has reported success of grabbing depth images from the Kinect with libfreenect yet (that I know of). I wish the Asus Xtion worked with freenect, OpenNI is so bulky, especially for the RPi.
Re: Kinect - Would it ever work with a Raspberry Pi?
For some reason I didn't work for me but once I used the freenect_set_led rather than the sync versions it was all fine. Am working on a video demo now will report back later
Re: Kinect - Would it ever work with a Raspberry Pi?
Here is my first blog post about getting the tilt stuff working, http://jonmacey.blogspot.co.uk/2012/07/ ... inect.html will start on the video stuff later.
Re: Kinect - Would it ever work with a Raspberry Pi?
as long as you wont try to use the depth cam (maybe opengl components) the tilt demo works quite fine without haking it.
Re: Kinect - Would it ever work with a Raspberry Pi?
So far I can get the depth and the video camera's to initialise ok, I can see the IR glowing, however there seems to be an issue that the freenect_set_video_callback and freenect_set_depth_callback don't properly register my callbacks and I just grab the initial zero buffer.
I'm going to see If I can dig deeper into the freenect source and see what is going on, I think the main issues are to do with sync speed, I'm getting data to and from as the debug is working ok
However getting this (which is from the usb_libusb10.c file)
Will dig deeper, btw the code I'm using is based on my blog post here http://jonmacey.blogspot.co.uk/2010/12/ ... esign.html
I'm going to see If I can dig deeper into the freenect source and see what is going on, I think the main issues are to do with sync speed, I'm getting data to and from as the debug is working ok
Code: Select all
Control cmd=0016 tag=0000 len=000a: 18
Control reply: 18
start_lines: 0
end_lines: 0
cropping_lines: 0
Control cmd=0004 tag=0001 len=000a: 18
Control reply: 18
dcmos_emitter_distance: 7.500000
dcmos_rcmos_distance: 2.300000
reference_distance: 120.000000
reference_pixel_size: 0.104200
Control cmd=0016 tag=0002 len=000a: 18
Control reply: 18
ax: 1657
bx: 29
cx: 6
dx: 466
ay: 13
by: 2683
cy: 588
dy: 30
dx_start: 890
dy_start: 11433
dx_beta_start: 10062
dy_beta_start: 130801
dx_beta_inc: 170
dy_beta_inc: 136
dxdx_start: 2095599
dxdy_start: 631
dydx_start: 693
dydy_start: 2096048
dxdxdx_start: 134215779
dydxdx_start: 134217220
dxdxdy_start: 134217092
dydxdy_start: 134217003
dydydx_start: 134217189
dydydy_start: 134214840
Control cmd=0016 tag=0003 len=000a: 18
Code: Select all
iso_callback(): failed to resubmit transfer after successful completion: -1
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
iso_callback(): failed to resubmit transfer after successful completion: -1
iso_callback(): failed to resubmit transfer after successful completion: -1
Re: Kinect - Would it ever work with a Raspberry Pi?
So it does seem to be a sync / timeout error, turning on USB debug gives this (and I've changed the timeout values in the kinect lib to loads of different values to no avail)
Enough code for one day will look again tomorrow
Code: Select all
[Stream 80] Invalid magic 0000
libusb: 0.734725 debug [submit_iso_transfer] need 1 32k URBs for transfer
libusb: 0.735193 error [submit_iso_transfer] submiturb failed error -1 errno=4008
libusb: 0.735427 debug [submit_iso_transfer] first URB failed, easy peasy
iso_callback(): failed to resubmit transfer after successful completion: -1
libusb: 0.735697 debug [libusb_handle_events_timeout_completed] doing our own event handling
libusb: 0.735879 debug [handle_events] poll() 4 fds with timeout in 60000ms
libusb: 0.736095 debug [handle_events] poll() returned 1
libusb: 0.736298 debug [reap_for_handle] urb type=0 status=0 transferred=0
libusb: 0.736479 debug [handle_iso_completion] handling completion status 0 of iso urb 1/1
libusb: 0.736671 debug [handle_iso_completion] last URB in transfer --> complete!
libusb: 0.736849 debug [disarm_timerfd]
libusb: 0.736987 debug [usbi_handle_transfer_completion] transfer 0x14840cc has callback 0x4023792c
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
[Stream 80] Invalid magic 0000
libusb: 0.737425 debug [submit_iso_transfer] need 1 32k URBs for transfer
libusb: 0.741147 error [submit_iso_transfer] submiturb failed error -1 errno=4008
libusb: 0.741431 debug [submit_iso_transfer] first URB failed, easy peasy
iso_callback(): failed to resubmit transfer after successful completion: -1
libusb: 0.741712 debug [libusb_handle_events_timeout_completed] doing our own event handling
libusb: 0.741907 debug [handle_events] poll() 4 fds with timeout in 60000ms
libusb: 0.742115 debug [handle_events] poll() returned 1
libusb: 0.742324 debug [reap_for_handle] urb type=0 status=0 transferred=0
libusb: 0.742856 debug [handle_iso_completion] handling completion status 0 of iso urb 1/1
libusb: 0.743074 debug [handle_iso_completion] last URB in transfer --> complete!
libusb: 0.743253 debug [disarm_timerfd]
libusb: 0.743407 debug [usbi_handle_transfer_completion] transfer 0x1485b44 has callback 0x4023792c
libusb: 0.743603 debug [submit_iso_transfer] need 1 32k URBs for transfer
libusb: 0.744082 error [submit_iso_transfer] submiturb failed error -1 errno=4008
libusb: 0.744313 debug [submit_iso_transfer] first URB failed, easy peasy
iso_callback(): failed to resubmit transfer after successful completion: -1
Re: Kinect - Would it ever work with a Raspberry Pi?
Just a few confirmations:
1) Kinect, despite coming with its own power plug, needs to be connected to the pi using a powered USB hub. Else, only the motor will be listed in lsusb, and libfreenect will not find a kinect device.
2) Not all powered USB hubs will work. Tried 4 hubs (3 different models), only one did work. This is the model of which I have two instances, the other instance does not work, so I cannot recommend any model.
3) When driving the motor, the pi reports ethernet problems. dmesg shows "eth0: kevent 4 may have been dropped"
4) Setting LEDs works. Also, the infrared laser diode can be switched on using freenect_start_depth.
5) freenect_process_events never calls the registered depth callback. Depth images do not reach the pi.
So it seems that Kinect depth sensing will not work with the pi.
1) Kinect, despite coming with its own power plug, needs to be connected to the pi using a powered USB hub. Else, only the motor will be listed in lsusb, and libfreenect will not find a kinect device.
2) Not all powered USB hubs will work. Tried 4 hubs (3 different models), only one did work. This is the model of which I have two instances, the other instance does not work, so I cannot recommend any model.
3) When driving the motor, the pi reports ethernet problems. dmesg shows "eth0: kevent 4 may have been dropped"
4) Setting LEDs works. Also, the infrared laser diode can be switched on using freenect_start_depth.
5) freenect_process_events never calls the registered depth callback. Depth images do not reach the pi.
So it seems that Kinect depth sensing will not work with the pi.
-
- Posts: 11
- Joined: Sat Jul 14, 2012 11:36 pm
Re: Kinect - Would it ever work with a Raspberry Pi?
Have experienced similar....
libfreenect can control leds and motor, but seems the callbacks in examples never get run, just end up with empty arrays where there should be depth/rgb data.
Compiled OpenNI and SensorKinect on Raspbian, but again no progress, running SampleNiSimpleRead just gives:
I am still hopeful though, seems people have got it working with not only the beagleboard-xm, but also the more rpi similar plain beagleboard:
http://www.kinecthesia.com/2011/04/great-success.html
http://www.kinecthesia.com/2011/10/kine ... -come.html
libfreenect can control leds and motor, but seems the callbacks in examples never get run, just end up with empty arrays where there should be depth/rgb data.
Compiled OpenNI and SensorKinect on Raspbian, but again no progress, running SampleNiSimpleRead just gives:
Code: Select all
UpdateData: a timeout has occurred waiting for new data.
http://www.kinecthesia.com/2011/04/great-success.html
http://www.kinecthesia.com/2011/10/kine ... -come.html
Re: Kinect - Would it ever work with a Raspberry Pi?
I actually hacked out all the threading code in the libraries, and it seems to be a USB bottleneck, I'm presuming it's just USB timeouts / async type problem.
There was an issue with the mac version and there had to be a patch for libUSB to make it work, my suspicion is that there is a similar problem with the pi, but I don't know enough about usb programming.
I will have another look next week when I get a chance.
There was an issue with the mac version and there had to be a patch for libUSB to make it work, my suspicion is that there is a similar problem with the pi, but I don't know enough about usb programming.
I will have another look next week when I get a chance.
Re: Kinect - Would it ever work with a Raspberry Pi?
This guys says he had success by "Manually build[ing] [Libusb] with the libusb0.1-compact"
see http://programmingwhilestupid.com/kinec ... pberry-pi/
Hope you have success with this. I have a Kinect sensor working with my PandaBoard, but I'd love to be able to replace it with a tiny inexpensive R-Pi.
see http://programmingwhilestupid.com/kinec ... pberry-pi/
Hope you have success with this. I have a Kinect sensor working with my PandaBoard, but I'd love to be able to replace it with a tiny inexpensive R-Pi.
Re: Kinect - Would it ever work with a Raspberry Pi?
any prove for this?
Re: Kinect - Would it ever work with a Raspberry Pi?
trying now but now luck so far
Re: Kinect - Would it ever work with a Raspberry Pi?
A bit more detail.
I've built both libusb and libusb-compat, It seems that the compat lib is for using older USB devices, however when linking the kinect lib it only seems to use the libusb.
I've hacked things to use just the compat lib and it built but when running it complains that libusb_init is not found. This is part of the normal lib-usb library, so I seems that it needs both.
With both installed I still get the usual errors of iso_callback() failed and [stream 80] Invalid magic 0000 and no data passed to the image block.
I've left a message on the blog post to ask for more details, see what he says
Jon
I've built both libusb and libusb-compat, It seems that the compat lib is for using older USB devices, however when linking the kinect lib it only seems to use the libusb.
I've hacked things to use just the compat lib and it built but when running it complains that libusb_init is not found. This is part of the normal lib-usb library, so I seems that it needs both.
With both installed I still get the usual errors of iso_callback() failed and [stream 80] Invalid magic 0000 and no data passed to the image block.
I've left a message on the blog post to ask for more details, see what he says
Jon
Re: Kinect - Would it ever work with a Raspberry Pi?
thx to theunit, I could also get the Xtion with the Pi working, see the images attached.theunit wrote:I have just got the ASUS Xtion sensor working on the raspberry pi using OpenNI......
the SimpleRead-Sample works much faster when its just in console mode (after booting) while the SimpleViewer-Sample is really slow.
I thought the Xtion (and the Kinect) would simply calculate everything on the hardware and then just send the results just to the windows output? But it seems, that the sample is calculating a lot of things by itself.
cheers
dimi
- Attachments
-
- xtion+pi_2.jpg (43.36 KiB) Viewed 8421 times
-
- xtion+pi_1.jpg (63.25 KiB) Viewed 8421 times
Re: Kinect - Would it ever work with a Raspberry Pi?
Nice work Dimi!
I see you got the SimpleViewer working, I thought there were issues with glut or something, it didn't work on mine. Did you do anything different? Which distro? I was using Debian squeeze, I am going to update to wheezy soon though.
I see you got the SimpleViewer working, I thought there were issues with glut or something, it didn't work on mine. Did you do anything different? Which distro? I was using Debian squeeze, I am going to update to wheezy soon though.
Re: Kinect - Would it ever work with a Raspberry Pi?
thx for your work in the first place!
yes the normal SimpleViewer isn´t compiling because of some glut errors, but I used the java-based one (org.OpenNI.Samples.SimpleViewer.SimpleViewer) this one is compiling and viewing just the depth-image. but again, reeeallyyy slow! maybe 1-2fps. and just when debian is started (with startx, obviously
)
currently I am on squeezy, some things wasn´t working in wheezy, even it was quite a little bit faster.
yes the normal SimpleViewer isn´t compiling because of some glut errors, but I used the java-based one (org.OpenNI.Samples.SimpleViewer.SimpleViewer) this one is compiling and viewing just the depth-image. but again, reeeallyyy slow! maybe 1-2fps. and just when debian is started (with startx, obviously

currently I am on squeezy, some things wasn´t working in wheezy, even it was quite a little bit faster.
Re: Kinect - Would it ever work with a Raspberry Pi?
I didn't even think to try the java samples. I would guess that the glut/c++ implementation would run a bit faster (if it worked).
-
- Posts: 11
- Joined: Sat Jul 14, 2012 11:36 pm
Re: Kinect - Would it ever work with a Raspberry Pi?
Could either of you with Xtions post the output of lsusb -vv?
Many thanks.
Many thanks.
-
- Posts: 11
- Joined: Sat Jul 14, 2012 11:36 pm
Re: Kinect - Would it ever work with a Raspberry Pi?
Maybe if I explain my reasoning. The PrimeSense Sensor driver defaults to using bulk endpoint transfers when compiled on arm, from GlobalDefaults.ini:
The output of lsusb -vv with kinect attached reports only isochronous endpoints:
Maybe the success with asus Xtion devices is because they support data transfers with less strict time constraints (bulk), where as the kinects iso transfers mean that on a device like the Pi, all frames end up being dropped as it can get no where near the 30 per second that seems the only option in both the libfreenect and PrimSense mod drivers? (Very small understanding of USB communication, may be totally wrong!)
Those with working Xtions, have you modified any of the settings in /usr/etc/primesense/GlobalDefaults.ini? What do the SamplesConfig.xml files you are using to run the samples look like, default? Apart from the usb transfer mode, I would be very interested in what resolution your settings files are causing, as again from GlobalDefaults.ini it seems when compiled on arm they differ:
The avin2 SensorKinect driver breaks if you set any other resolution than VGA and fps other than 30.
I have tried modifying XnDeviceSensorIO.cpp in avin2s mod to use bulk transfers, always fails. Figure I was being optimistic thinking you could choose the transfer mode even after a particular endpoint had been enumerated/was set in firmware.
Also, from what I have seen so far, compiling libusb-1.0.x or libusb-compat from source has absolutely no effect, but I may be doing it wrong, seems it is only libusb-1.0.x that is used by both drivers, at least for the camera.
Thanks for any responses.
Code: Select all
; USB interface to be used. 0 - FW Default, 1 - ISO endpoints, 2 - BULK endpoints. Default: Arm - 2, other platforms - 1
;UsbInterface=2
Code: Select all
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x0bc0 2x 960 bytes
Those with working Xtions, have you modified any of the settings in /usr/etc/primesense/GlobalDefaults.ini? What do the SamplesConfig.xml files you are using to run the samples look like, default? Apart from the usb transfer mode, I would be very interested in what resolution your settings files are causing, as again from GlobalDefaults.ini it seems when compiled on arm they differ:
Code: Select all
; 0 - QVGA, 1 - VGA, 4 - QQVGA. Default: Arm - 4, other platforms - 0
;Resolution=1
I have tried modifying XnDeviceSensorIO.cpp in avin2s mod to use bulk transfers, always fails. Figure I was being optimistic thinking you could choose the transfer mode even after a particular endpoint had been enumerated/was set in firmware.
Also, from what I have seen so far, compiling libusb-1.0.x or libusb-compat from source has absolutely no effect, but I may be doing it wrong, seems it is only libusb-1.0.x that is used by both drivers, at least for the camera.
Thanks for any responses.
Re: Kinect - Would it ever work with a Raspberry Pi?
Hey hitcher 404,
I didn´t modified GlobalDefaults.ini or the SamplesConfig.xml.
When i digg depper the SimpleRead-Sample, there is the full depth image information coming with 30fps and just the middle point is shown on screen. So in console (without startx) information are coming really fast.
And here is the ouput of my lsusb -v (-w isn´t possible)
hope you can work with it.
cheers
dimi
I didn´t modified GlobalDefaults.ini or the SamplesConfig.xml.
When i digg depper the SimpleRead-Sample, there is the full depth image information coming with 30fps and just the middle point is shown on screen. So in console (without startx) information are coming really fast.
And here is the ouput of my lsusb -v (-w isn´t possible)
Code: Select all
Bus 001 Device 005: ID 1d27:0600
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
idVendor 0x1d27
idProduct 0x0600
bcdDevice 0.01
iManufacturer 2 PrimeSense
iProduct 1 PrimeSense Device
iSerial 0
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 69
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 500mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 3
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x0b70 2x 880 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x135c 3x 860 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 1
Transfer Type Isochronous
Synch Type None
Usage Type Data
wMaxPacketSize 0x1040 3x 64 bytes
bInterval 1
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 1
bNumEndpoints 3
bInterfaceClass 255 Vendor Specific Class
bInterfaceSubClass 0
bInterfaceProtocol 0
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x82 EP 2 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0200 1x 512 bytes
bInterval 1
Device Qualifier (for other device speed):
bLength 10
bDescriptorType 6
bcdUSB 2.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 64
bNumConfigurations 1
Device Status: 0x0000
(Bus Powered)
cheers
dimi
Re: Kinect - Would it ever work with a Raspberry Pi?
Hello everyone I have finally got my pi so i will see what i can do with the kinect running ont he pi.
Keep up the great work with kinect on pi.
Keep up the great work with kinect on pi.
Re: Kinect - Would it ever work with a Raspberry Pi?
Good Luck AjTech and keep us informed about your progress!AjTech wrote:Hello everyone I have finally got my pi so i will see what i can do with the kinect running ont he pi.
Keep up the great work with kinect on pi.