User avatar
jbeale
Posts: 3345
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

blue-sky dreaming: only slightly ridiculous feature request

Thu Dec 12, 2013 10:17 pm

In the fullness of time, when JamesH or someone gets everything else done, it would be awfully neat to be able to readout only some selected scanlines from the sensor.

Why? Because then you could get a very high framerate if you only need a few scanlines (assuming no other limitations besides the OmniVision chip).

And why is that useful? Because I'd like one of these (but cheaper than the $400 Neato XV-11 robot vacuum it's built into):
A Low-Cost Laser Distance Sensor (triangulation-based scanning LIDAR) http://www.robotshop.com/media/files/PD ... epaper.pdf

JustThisGuy
Posts: 114
Joined: Thu Jan 05, 2012 11:22 pm

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 12:03 am

Wouldn't -roi be of use here?
Any conversation about a sufficiently complex subject is indistinguishable from babble.

User avatar
jbeale
Posts: 3345
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 12:14 am

-roi works at a higher level, it is a digital zoom into the given frame. Behind the scenes, I suspect the full video sensor area is still being read out, or at least it may as well be, because right now you are still limited to 30 fps no matter what. I am suggesting something closer to bare-metal programming of the sensor chip where you read out selected scanlines at the maximum refresh rate. If you are bandwidth limited to some fixed number of pixels per second, and you can do 1080 scanlines at 30 fps, you should be able to do 108 scanlines at 300 fps and 10 scanlines at over 3000 fps. The whitepaper I linked above describes a LIDAR application using 10 scanlines at 4000 fps (although I don't know what specific camera they use). I believe the R-Pi camera should be capable of a similar level of performance, given the right interface code. I think this is a very intriguing application myself, but I admit it is a sort of "niche" use.

poing
Posts: 1131
Joined: Thu Mar 08, 2012 3:32 pm

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 12:24 am

I didn't read the whole paper, but why do you need 4000 fps?

User avatar
jbeale
Posts: 3345
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 12:52 am

poing wrote:I didn't read the whole paper, but why do you need 4000 fps?
You don't really need any particular framerate, but the faster it is, the more useful it is. This is a triangulation scheme; so the position of the laser dot as seen by the camera (which is offset by some baseline from the laser source) determines the distance to the reflecting object. Each frame generates just one distance ranging value (unless you've got multiple beam intersections, like through a glass window, or windowscreen, at edges, etc). You put the laser+camera on a spinning platform so you can sweep out 360 degrees. If your camera does 4k fps and your assembly rotates at 10 Hz (600 rpm), you can then acquire 400 points per rotation, which gives you slightly better than 1 degree angular resolution for your range map. Those are the parameters of the "Neato Robotics" XV-11 robotic vacuum cleaner which maps out the borders of the room it is in, in real time.

If your frame rate is much slower, you must take data points much more slowly and it becomes progressively less useful for building a real-time range map. If your robot is supposed to avoid an obstacle which is a person walking by, you want to get at least some data points before the person has been and gone. Likewise if you are also tilting your rotation axis to sweep out an area and build up a 3D range map, your full scan time becomes extremely long if you have to rotate slowly. With more data you can also do averaging to improve your signal to noise ratio.

Instead of tilting or translating the spinning scanner, you can also have it stationary and detect passing objects, like boxes on a conveyor belt, people in a hallway, or cars on a road. With a fast scan you can acquire many depth-map slices and even build up a partial 3D model as the object passes through the scan plane. With a slow scan, you may get only a few points or miss the object completely, if it is moving quickly.

Laser scanners are well known industrial tools. I have read that Google's self-driving car uses a Velodyne HDL-64e lidar scanner costing $70000. That's a high-end device, but even the cheap ones are in the thousands. There is only one "consumer" lidar I know of, part of the $400 XV-11 vacuum robot. I suspect a Raspberry Pi based device could be cheaper than that.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 19530
Joined: Sat Jul 30, 2011 7:41 pm

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 9:20 am

The sensor only bins to 2x2, as you would expect, but it might be possible to skip multiple lines/pixels on readout. Not sure though - looking at the datasheet there is nothing obvious. I'll keep looking.

EDIT: registers 0x3814 and 0x3815 are the ones, but there are upper limits to frame rate based on the speed of the sensors ADC's, and they top out fairly soon once you get above 120fps or so (depends on the sensor, not sure of exact figure for the 5647)
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
jbeale
Posts: 3345
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: blue-sky dreaming: only slightly ridiculous feature requ

Fri Dec 13, 2013 2:58 pm

jamesh wrote:The sensor only bins to 2x2, as you would expect, but it might be possible to skip multiple lines/pixels on readout. Not sure though - looking at the datasheet there is nothing obvious. I'll keep looking.

EDIT: registers 0x3814 and 0x3815 are the ones, but there are upper limits to frame rate based on the speed of the sensors ADC's, and they top out fairly soon once you get above 120fps or so (depends on the sensor, not sure of exact figure for the 5647)
I see. The camera they used must have had a different architecture to permit the higher frame rate. Thanks for looking into it, anyway!

steveglennon
Posts: 2
Joined: Thu Jan 14, 2016 12:08 am
Location: Louisville, CO

Re: blue-sky dreaming: only slightly ridiculous feature requ

Thu Jan 14, 2016 12:28 am

I would still love to see 120FPS at QVGA or even QQVGA supported on the Pi. I'm looking at some computer vision applications and having lots of success with the Pi2, running at 90fps. I believe faster frame rate would be more advantageous than higher resolution - hence the happy tradeoff to 320x240 or even 160x120.

I know this is probably not mainstream, but if jamesh or 6x9 might be willing I could contribute some updates to raspicam and raspicam_cv add the 90fps (I have working now) and 120fps support.

Thanks for even considering this....

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5124
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: blue-sky dreaming: only slightly ridiculous feature requ

Thu Jan 14, 2016 9:09 am

Holy thread resurrection Batman!

I think this was before the binned and 90fps modes were added, so we hadn't found the readout skip registers. With those you can skip an arbitrary number of pixels in both directions between odd and even pixels (different to maintain the Bayer pattern). IIRC 90fps mode is binned and then skip 3, skip 5 in both directions (there was a thread which got into lots of detail as to whether skip 3/5 was better than 7/1 with lots of confusion over how Bayer patterns work).

The problem I hit was that above 90fps the tuner algorithms ended up taking longer than the frame period, and I wasn't intending to go hacking around in there. Someone else has become involved in the camera firmware and felt like it wouldn't be too bad, so if and when he sorts that then we might revisit even higher FPS modes on OV5647.

Actually your request is subtly different from jbeale's original (I think). AIUI His request was for effectively 2952x2 so that he could get a rotating line array. That sort of mode won't be added as it just gets too onerous to support all the different potential requests and use cases.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
jbeale
Posts: 3345
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: blue-sky dreaming: only slightly ridiculous feature requ

Thu Jan 14, 2016 4:46 pm

6by9 wrote:Actually your request is subtly different from jbeale's original (I think). AIUI His request was for effectively 2952x2 so that he could get a rotating line array. That sort of mode won't be added as it just gets too onerous to support all the different potential requests and use cases.
Well I did say it was slightly ridiculous :-). Also, since I made the original request there have been other developments in the LIDAR field, with simple systems like Lidar-Lite ( http://pulsedlight3d.com/ ) now around $100, so no worries there.

Currently I'm getting a lot of use out of the existing RPi camera for normal photo/video applications. There are various other dev-board type cameras (ARM-based systems, even Arducam, etc.) but to my knowledge nothing is even close to as well supported, capable, or useful as the RPi camera.

steveglennon
Posts: 2
Joined: Thu Jan 14, 2016 12:08 am
Location: Louisville, CO

Re: blue-sky dreaming: only slightly ridiculous feature requ

Thu Jan 14, 2016 4:58 pm

jbeale wrote:Currently I'm getting a lot of use out of the existing RPi camera for normal photo/video applications. There are various other dev-board type cameras (ARM-based systems, even Arducam, etc.) but to my knowledge nothing is even close to as well supported, capable, or useful as the RPi camera.
I've been looking at a lot of possible platforms for small, lightweight, inexpensive camera/vision development, including somewhat expensive CV cameras from PointGrey. Nothing else I have seen comes close to RPi for small, lightweight, and access to uncompressed camera data fast and with low overhead. Awesome platform for computer vision. I kind of got sick of people who thought that the act of doing something on the RPi made it (and them) clever, but in the realm of computer vision I think it is a fantastic platform.

I know there is lots of BRCM and OV proprietary data that prevents us digging in too far, but I will help anywhere I can.

Return to “Camera board”

Who is online

Users browsing this forum: No registered users and 7 guests