Cheapest High speed camera up to 900 FPS!!

[mhamdy]

Hi Guys,
I have been thinking and planning about this and i decided to join here and discuss with you guys here and find out what you think.
I have found CMOS sensor produced by toshiba, 13 Mp which can goes up to 900 fps in bright mode and costs Sample Price 1,500 Yen ( almost 13 dollars per piece) (source for the price is : http://www.bloomberg.com/bb/newsarchive ... nEwYU.html)

Lets get to the technical details of it,
(http://toshiba.semicon-storage.com/info ... 82&lang=en)
Property Value Unit
Optical size 1/3.07 inch -
Pixel size (µm) 1.12 micrometer
Number of pixels 13M pixels
Output pixels 4208(H)×3120(V) -
Data formats RAW8
RAW10 -
Frame rate (full) 30 (normal mode: progressive) fps
Frame rate (1080p) 120 (normal mode: progressive) / 240 (bright mode: interlace) fps
Frame rate (720p) 120 (normal mode: progressive) / 240 (bright mode: interlace) fps
Frame rate (QVGA) 480 (normal mode: progressive) / 900 (bright mode: interlace) fps
Frame rate (VGA) 240 (normal mode: progressive) / 480 (bright mode: interlace) fps
Power supply voltage (analog) 2.8 +/- 0.2 V
Power supply voltage (digital) 1.2 +/- 0.1 and 1.8 +/- 0.1 V
Power supply voltage (IO) 1.7 to 3.0 V
Operational Temperature(Topr) -20 to 60 degC
Storage Temperature(Tstg) -40 to 85 degC
I/F (serial) CSI-2 4lanes
Control interface I2C
Progressive scan Progressive scan
Rest you can find over website
For the system block diagram


Now regarding Raspberry PI 2

So lets 1st discuss Raspberry PI 2 camera interface, CSI-2. It is a system for low voltage 1.2 V applications, allowing data rates of up to 800 Mbps per lane with 1 Gbps set as a practical limit. In practice, the data rate can vary a lot and depends upon the quality of the interconnections. A maximum of four physical data lanes are allowable in this specification, however two are available for the Raspberry Pi
This is a high-speed data communication bus and noise is of huge concern to the design engineer. Although this type of serial communication generates negligible crosstalk, the specification suggests using minimum clock rates for the camera module. The CSI transmission clock is source synchronous and the main processor may produce it instead to avoid noise interference on the camera module. The data transmission supports a wide range of data types such as RGB, RAW, YUV, generic, or byte based programmer defined. (Source for this and pic down is http://www.petervis.com/Raspberry_PI/Ra ... tions.html )



I think, the budget for this project if numbers are right, would be maximum 45$, saying that i would make c-mount and install my nikon lens. Would be cool and cheap 900 fps camera right?

[gkreidl]

A simple calculation will show you that such frame rates are not possible with the RPi and also not for the H264 encoder. And who will write the software for the GPU?

[jamesh]

The Pi CSI-2 interface is 2 lanes, not four. The chip does have a 4 lane interface, but it's not on the CSI connector. The Compute Module can handle 4 lanes.

And as above, this requires access to the GPU/ISP software which is closed source.

[mhamdy]

I think at qvga 900fps is supported by RPI 2, I have done the calculations, I think you can do it too, many online calculators

[mhamdy]

I think the real issue is access to GPU/ISP, that's why I posted here, did someone manage before?

[RaTTuS]

viewtopic.php?f=43&t=109137 is going to be your best bet

[jbeale]

I think the real issue is access to GPU/ISP, that's why I posted here, did someone manage before?

To date the only available camera that works on the R-Pi is the one being sold for that purpose now, the OmniVision OV5647.
An added wrinkle is that the OV5647 sensor is scheduled to become obsolete viewtopic.php?f=43&t=106055&p=731659 and we have been told plans to address that are being made, but they have not yet been revealed.

If you look through the past 2-3 years of postings in this forum, you will see many people proposing to develop, or hoping someone would develop an interface for various other camera sensors. That has not come to pass, because the camera interface is a non-public, closed source part of the GPU, and the Raspberry Pi Foundation's goals were satisfied with the one sensor available. Broadcom apparently allows that level of access to the GPU/image pipeline hardware only to significantly large companies, and under NDA.

However, just a few days ago we did get basic access to the CSI-2 interface
viewtopic.php?f=43&t=109137 (but remember that's just the raw data.)

[mhamdy]

Thanks, at least there is a hope

[jbeale]

I think there is a hope to interface new 2-lane MIPI CSI-2 interface devices to the R-Pi (or 4-lane with the much less hobbyist-friendly compute module) BUT with the important limitation that the ISP (image pipeline) is still not publicly available and there is no current prospect that it will be. That means you can get raw data at relatively low speed (dictated by the memory bus clock and ARM clock rate) but I do not see any path to 900 FPS at this time.

[mhamdy]

the question is why the new release of pi 2 with higher spec and surprise surprise you still have to work with old spec, as you said OV5647 will become obsolete, anyways 1G ram wasnt really that good option for high speed camera, i think i have to drop raspberry pi for my project and go to another higher spec open GPU.

[riklaunim]

If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.

[rpiswag]

Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.

[riklaunim]

Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.

Due to intellectual property that still has value or not only Broadcom has the rights to it so it would have to be a decision of multiple vendors. And some ARM licensing issues may arise here and there.

[jamesh]

Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.

Of course Brcm makes money on it - quite a lot of money or they wouldn't be selling them at all - the Pi isn't the only device out there with a VC4 in it. It's used in Roku and Sky media boxes for a start.

There are very few if any mobile SoC with ISP's and GPU's that a fully open source. None I suspect.

[rpiswag]

WOW! I am truly amazed! It is great that the Raspberry Pi Foundation helps kids get inspired to go into computer. For instance the pi is what got me into computers and I am happy for that. What about a Kickstarter project to buy those binary blobs and release them into the world? How many million would one need for that to happen?

[riklaunim]

How many million would one need for that to happen?

None. You don't sell things that bring money. Not to mention that it may not be doable legally as there may be many owners of IP (AFAIK this is one of reasons nvidia can't opensource to much).

You want more open source system? Use Intel based one AMD is getting there too as their open source GPU driver actually works on par with closed and it will be even better as AMD is cleaning up duplication between drivers.

[mhamdy]

If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.

No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well

[pluggy]

If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.

No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well

http://www.mini-itx.com/store/asus

By the time you've bought the workarounds for a Pi, you'll be up to the price of this, a stick of RAM and a USB stick to boot from.

Shock, horror, scandal, it would even run real Windows, accept an old school VGA monitor, play Flash in a browser, plug in real SATA drives, and probably leave you change out of 20 watts.

Heretic, burn him at the stake......

[riklaunim]

No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well

A 10W TDP CPU isn't very high, and you could look for even lower. The total price would be quite low too as the imaging solutions are already there and supported. And you already have high level SDK and libraries to avoid "complex processing" or the need for raw low level access to some hardware.

Raspberry Pi doesn't have such solutions and will never have. It's not a high performance flexible computing platform. If you dump a lot of money then some software could be made, you will have to hack low RAM amount (so no RAM buffering), to slow storage (no high framerates), to slow interfaces (no high framerate) and may may other limitation just to get a fraction of performance of a fast sensor - which also won't be cheap - and if you want to make your own camera you will have to pay even more for creating one from a bare sensor (duplicating the cost big companies pay do to the same thing).

So it's your decision - use $100 Intel PC/laptop/tablet with $1500 fast machine vision camera or invest $100 000 or more to get a slow camera with basic electronics connected to $35 Raspberry.

[Lee Tracey]

I'm a newcomer to this forum and not really an expert on the Rasberry. What has triggered my interest is the 900 FPS aspect as I am actively researching this function as related to a miniature video recorder. I have seen a claim that a newco is developing a camera with a 4K sensor that can be windowed down to provide 1000 FPS at 1080p and or the same camera or a version that can deliver 1000 FPS at VGA ( 640 X 480). The CMOSIS CM 12000 sensor has been hinted at. The 4K windowed down to 1080 and delivering 1000 FPS at that resolution might be a possibility in the multi-thousand dollar machine vision market but I venture to propose impossible for the domestic pleasure market. My ambition is to develop a 2016 version of the Casio ZR200 camera as I am convinced that if I can locate the right sensor then I can raise the cash. The ZR200 used a 16MP sensor and could take stills at full 2K HD of 1920 X 1080p and also make video at 30 FPS at 640 X 480 and also 1000 FPS video at 244 X 64. Today I think the 2016 version could upgrade the video at 30 FPS to 1920 X 1080p and the 1000 FPS to 640 X 480, and as such and with the other features of the ZR200 would be a multi-market world beater. I would love to hear from anybody else interested in this project or can shed any technical light on any aspect, especially choice of sensor. Why did Casio not proceed with the ZR200 in 2012? That was the year of auto focus (AF) and every camera manufacturer was turning out cameras with the feature - Until Honeywell dropped a bombshell on them claiming a patent. Honeywell brought a test case against Minolta and won a $150 million settlement. Casio, a small player in the camera market, panicked and rather than fight Honeywell closed the camera plant and dropped out of the camera game. Anybody interested in the activity of the "Newco" please get in touch and I will provide a link. Lee Tracey - [email protected]

[HermannSW]

viewtopic.php?f=43&t=109137 is going to be your best bet

Yes it is, "raspiraw" is cool and the only option allowing to capture camera image sequences without GPU code access.

The trick is to store the captured frames under "/dev/shm" ramdisk.
That way you can store raspiraw frames with a save rate of 1 (all frames).
Because of /dev/shm size you can record videos up to 4s on Pi Zero and up to 9s on Pi 2B.
Then after recording completed, post processing can happen.
viewtopic.php?f=43&t=199204

No new camera is needed, the good old Raspberry v1 camera can record videos with up to 645fps:

• 640x240 stretched to 640x480 at 180fps
• 640x120 stretched to 640x240 at 360fps (patch for you to try here)
  
• 640x64 stretched to 640x128 at 600fps (only Pi 2B and 3B, not Pi Zero), here
  

If processing the frames on the fly and no storing is needed (like for robot control), then of course the high fps time is unlimited:
viewtopic.php?f=43&t=189661#p1231151

[HermannSW]

Here are some more 600fps videos, starting from this posting:
viewtopic.php?f=43&t=109137&start=300#p1243429

Candle light video with camera rotated 90°, animated .gif with 25fps, 24x slower than real:


You can use 600fps mode for power line frequency detection as well by direct into lamp video.
In Germany we have 50Hz, with repeating pattern all 600/(50*2)=6 frames in 600fps video.
If repeating pattern happens every 5 frames then power line frequency is 60Hz.


2000rpm gear motor validation by counting frame number per 1 motor axis rotation (here between 16 and 17):


The new modes (360fps, 600fps) qualify Raspberry v1 camera as high speed camera (>250fps):
https://en.wikipedia.org/wiki/High-speed_camera

[jamesh]

Here are some more 600fps videos, starting from this posting:
viewtopic.php?f=43&t=109137&start=300#p1243429

Candle light video with camera rotated 90°, animated .gif with 25fps, 24x slower than real:


You can use 600fps mode for power line frequency detection as well by direct into lamp video.
In Germany we have 50Hz, with repeating pattern all 600/(50*2)=6 frames in 600fps video.
If repeating pattern happens every 5 frames then power line frequency is 60Hz.


2000rpm gear motor validation by counting frame number per 1 motor axis rotation (here between 16 and 17):

The weird labyrinth like artefacts on the images is down to a inadequate/too simplistic debayer algorithm IIRC. Not sure what you are using for that, but might be worth looking in to more sophisticated ones.

[HermannSW]

> The weird labyrinth like artefacts on the images is down to a inadequate/too simplistic debayer algorithm IIRC.
>
Good to know.

> Not sure what you are using for that, but might be worth looking in to more sophisticated ones.
>
I use 6by9's patched version of dcraw, that allows to process frames captured with "raspiraw -hd ...":
https://github.com/6by9/RPiTest

[6by9]

I'm not sure it is demosaicing at fault. I was fiddling yesterday trying to get a 320x240 mode running and was observing similar issues through the ISP. The standard mode 7 (VGA @ 90fps) doesn't suffer it, but as soon as you increase the skipping factor further on the sensor it throws it off, and it didn't seem stable.
I also haven't worked out why HermannSW's modes also appear to change the Bayer order, and I suspect that may be linked.