mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 2:24 pm

Hi Guys,
I have been thinking and planning about this and i decided to join here and discuss with you guys here and find out what you think.
I have found CMOS sensor produced by toshiba, 13 Mp which can goes up to 900 fps in bright mode and costs Sample Price 1,500 Yen ( almost 13 dollars per piece) (source for the price is : http://www.bloomberg.com/bb/newsarchive ... nEwYU.html)

Lets get to the technical details of it,
(http://toshiba.semicon-storage.com/info ... 82&lang=en)
Property Value Unit
Optical size 1/3.07 inch -
Pixel size (µm) 1.12 micrometer
Number of pixels 13M pixels
Output pixels 4208(H)×3120(V) -
Data formats RAW8
RAW10 -
Frame rate (full) 30 (normal mode: progressive) fps
Frame rate (1080p) 120 (normal mode: progressive) / 240 (bright mode: interlace) fps
Frame rate (720p) 120 (normal mode: progressive) / 240 (bright mode: interlace) fps
Frame rate (QVGA) 480 (normal mode: progressive) / 900 (bright mode: interlace) fps
Frame rate (VGA) 240 (normal mode: progressive) / 480 (bright mode: interlace) fps
Power supply voltage (analog) 2.8 +/- 0.2 V
Power supply voltage (digital) 1.2 +/- 0.1 and 1.8 +/- 0.1 V
Power supply voltage (IO) 1.7 to 3.0 V
Operational Temperature(Topr) -20 to 60 degC
Storage Temperature(Tstg) -40 to 85 degC
I/F (serial) CSI-2 4lanes
Control interface I2C
Progressive scan Progressive scan
Rest you can find over website :D :D :D
For the system block diagram
Image

Now regarding Raspberry PI 2

So lets 1st discuss Raspberry PI 2 camera interface, CSI-2. It is a system for low voltage 1.2 V applications, allowing data rates of up to 800 Mbps per lane with 1 Gbps set as a practical limit. In practice, the data rate can vary a lot and depends upon the quality of the interconnections. A maximum of four physical data lanes are allowable in this specification, however two are available for the Raspberry Pi
This is a high-speed data communication bus and noise is of huge concern to the design engineer. Although this type of serial communication generates negligible crosstalk, the specification suggests using minimum clock rates for the camera module. The CSI transmission clock is source synchronous and the main processor may produce it instead to avoid noise interference on the camera module. The data transmission supports a wide range of data types such as RGB, RAW, YUV, generic, or byte based programmer defined. (Source for this and pic down is http://www.petervis.com/Raspberry_PI/Ra ... tions.html )

Image

I think, the budget for this project if numbers are right, would be maximum 45$, saying that i would make c-mount and install my nikon lens. Would be cool and cheap 900 fps camera right?

gkreidl
Posts: 6130
Joined: Thu Jan 26, 2012 1:07 pm
Location: Germany

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 3:47 pm

A simple calculation will show you that such frame rates are not possible with the RPi and also not for the H264 encoder. And who will write the software for the GPU?
Minimal Kiosk Browser (kweb)
Slim, fast webkit browser with support for audio+video+playlists+youtube+pdf+download
Optional fullscreen kiosk mode and command interface for embedded applications
Includes omxplayerGUI, an X front end for omxplayer

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24130
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 3:49 pm

The Pi CSI-2 interface is 2 lanes, not four. The chip does have a 4 lane interface, but it's not on the CSI connector. The Compute Module can handle 4 lanes.

And as above, this requires access to the GPU/ISP software which is closed source.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 4:04 pm

I think at qvga 900fps is supported by RPI 2, I have done the calculations, I think you can do it too, many online calculators

mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 4:06 pm

I think the real issue is access to GPU/ISP, that's why I posted here, did someone manage before?

User avatar
RaTTuS
Posts: 10498
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK
Contact: Twitter YouTube

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 4:09 pm

viewtopic.php?f=43&t=109137 is going to be your best bet
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

User avatar
jbeale
Posts: 3516
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 4:21 pm

mhamdy wrote:I think the real issue is access to GPU/ISP, that's why I posted here, did someone manage before?
To date the only available camera that works on the R-Pi is the one being sold for that purpose now, the OmniVision OV5647.
An added wrinkle is that the OV5647 sensor is scheduled to become obsolete viewtopic.php?f=43&t=106055&p=731659 and we have been told plans to address that are being made, but they have not yet been revealed.

If you look through the past 2-3 years of postings in this forum, you will see many people proposing to develop, or hoping someone would develop an interface for various other camera sensors. That has not come to pass, because the camera interface is a non-public, closed source part of the GPU, and the Raspberry Pi Foundation's goals were satisfied with the one sensor available. Broadcom apparently allows that level of access to the GPU/image pipeline hardware only to significantly large companies, and under NDA.

However, just a few days ago we did get basic access to the CSI-2 interface
viewtopic.php?f=43&t=109137 (but remember that's just the raw data.)

mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 4:34 pm

Thanks, at least there is a hope :D

User avatar
jbeale
Posts: 3516
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 5:36 pm

I think there is a hope to interface new 2-lane MIPI CSI-2 interface devices to the R-Pi (or 4-lane with the much less hobbyist-friendly compute module) BUT with the important limitation that the ISP (image pipeline) is still not publicly available and there is no current prospect that it will be. That means you can get raw data at relatively low speed (dictated by the memory bus clock and ARM clock rate) but I do not see any path to 900 FPS at this time.

mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 6:02 pm

the question is why the new release of pi 2 with higher spec and surprise surprise you still have to work with old spec, as you said OV5647 will become obsolete, anyways 1G ram wasnt really that good option for high speed camera, i think i have to drop raspberry pi for my project and go to another higher spec open GPU.

riklaunim
Posts: 265
Joined: Tue Apr 22, 2014 7:34 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 8:13 pm

If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.

rpiswag
Posts: 804
Joined: Mon May 19, 2014 10:04 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 9:46 pm

Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.
A computer's power can't be just measured Gigahertz. It is the same thing with us humans.

riklaunim
Posts: 265
Joined: Tue Apr 22, 2014 7:34 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 10:08 pm

rpiswag wrote:Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.
Due to intellectual property that still has value or not only Broadcom has the rights to it so it would have to be a decision of multiple vendors. And some ARM licensing issues may arise here and there.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24130
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 10:13 pm

rpiswag wrote:Why can't the pi be open source and open hardware? It has a old gpu and I would be amazed if Broadcom made any money on still in 2015(not including the pi).The only reason I own a Beagle Bone Black was because it is a open source and it is open hardware as well.
Of course Brcm makes money on it - quite a lot of money or they wouldn't be selling them at all - the Pi isn't the only device out there with a VC4 in it. It's used in Roku and Sky media boxes for a start.

There are very few if any mobile SoC with ISP's and GPU's that a fully open source. None I suspect.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

rpiswag
Posts: 804
Joined: Mon May 19, 2014 10:04 pm

Re: Cheapest High speed camera up to 900 FPS!!

Tue May 05, 2015 10:51 pm

WOW! I am truly amazed! It is great that the Raspberry Pi Foundation helps kids get inspired to go into computer. For instance the pi is what got me into computers and I am happy for that. What about a Kickstarter project to buy those binary blobs and release them into the world? How many million would one need for that to happen?
A computer's power can't be just measured Gigahertz. It is the same thing with us humans.

riklaunim
Posts: 265
Joined: Tue Apr 22, 2014 7:34 pm

Re: Cheapest High speed camera up to 900 FPS!!

Wed May 06, 2015 6:23 am

rpiswag wrote:How many million would one need for that to happen?
None. You don't sell things that bring money. Not to mention that it may not be doable legally as there may be many owners of IP (AFAIK this is one of reasons nvidia can't opensource to much).

You want more open source system? Use Intel based one ;) AMD is getting there too as their open source GPU driver actually works on par with closed and it will be even better as AMD is cleaning up duplication between drivers.

mhamdy
Posts: 6
Joined: Tue May 05, 2015 1:44 pm

Re: Cheapest High speed camera up to 900 FPS!!

Wed May 06, 2015 12:37 pm

riklaunim wrote:If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.
No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well :)

User avatar
pluggy
Posts: 3635
Joined: Thu May 31, 2012 3:52 pm
Location: Barnoldswick, Lancashire,UK
Contact: Website

Re: Cheapest High speed camera up to 900 FPS!!

Wed May 06, 2015 1:02 pm

mhamdy wrote:
riklaunim wrote:If you really need fast framerate and good support for it then just pick some mini-ITX Intel based mini PC and use one of machine vision cameras that does the job. No need for vendor specific hardware hacking or limitations. Currently for example sensors made by CMOSIS offer very fast framerate (available in Ximea, Point Grey, Basler and other cameras). You get a camera with support and warranty, software with good SDK and some third party apps to play with.
No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well :)
http://www.mini-itx.com/store/asus

By the time you've bought the workarounds for a Pi, you'll be up to the price of this, a stick of RAM and a USB stick to boot from.

Shock, horror, scandal, it would even run real Windows, accept an old school VGA monitor, play Flash in a browser, plug in real SATA drives, and probably leave you change out of 20 watts.

Heretic, burn him at the stake......
Don't judge Linux by the Pi.......
I must not tread on too many sacred cows......

riklaunim
Posts: 265
Joined: Tue Apr 22, 2014 7:34 pm

Re: Cheapest High speed camera up to 900 FPS!!

Wed May 06, 2015 6:54 pm

mhamdy wrote:No way going on intel mother board pass, expensive high power consumption, complex processing, my target was arm simple low cost project but i got to say i respect companies like Ximea , they are doing good job as well :)
A 10W TDP CPU isn't very high, and you could look for even lower. The total price would be quite low too as the imaging solutions are already there and supported. And you already have high level SDK and libraries to avoid "complex processing" or the need for raw low level access to some hardware.

Raspberry Pi doesn't have such solutions and will never have. It's not a high performance flexible computing platform. If you dump a lot of money then some software could be made, you will have to hack low RAM amount (so no RAM buffering), to slow storage (no high framerates), to slow interfaces (no high framerate) and may may other limitation just to get a fraction of performance of a fast sensor - which also won't be cheap - and if you want to make your own camera you will have to pay even more for creating one from a bare sensor (duplicating the cost big companies pay do to the same thing).

So it's your decision - use $100 Intel PC/laptop/tablet with $1500 fast machine vision camera or invest $100 000 or more to get a slow camera with basic electronics connected to $35 Raspberry.

Lee Tracey
Posts: 1
Joined: Fri Dec 18, 2015 11:05 pm

Re: Cheapest High speed camera up to 900 FPS!!

Fri Dec 18, 2015 11:41 pm

I'm a newcomer to this forum and not really an expert on the Rasberry. What has triggered my interest is the 900 FPS aspect as I am actively researching this function as related to a miniature video recorder. I have seen a claim that a newco is developing a camera with a 4K sensor that can be windowed down to provide 1000 FPS at 1080p and or the same camera or a version that can deliver 1000 FPS at VGA ( 640 X 480). The CMOSIS CM 12000 sensor has been hinted at. The 4K windowed down to 1080 and delivering 1000 FPS at that resolution might be a possibility in the multi-thousand dollar machine vision market but I venture to propose impossible for the domestic pleasure market. My ambition is to develop a 2016 version of the Casio ZR200 camera as I am convinced that if I can locate the right sensor then I can raise the cash. The ZR200 used a 16MP sensor and could take stills at full 2K HD of 1920 X 1080p and also make video at 30 FPS at 640 X 480 and also 1000 FPS video at 244 X 64. Today I think the 2016 version could upgrade the video at 30 FPS to 1920 X 1080p and the 1000 FPS to 640 X 480, and as such and with the other features of the ZR200 would be a multi-market world beater. I would love to hear from anybody else interested in this project or can shed any technical light on any aspect, especially choice of sensor. Why did Casio not proceed with the ZR200 in 2012? That was the year of auto focus (AF) and every camera manufacturer was turning out cameras with the feature - Until Honeywell dropped a bombshell on them claiming a patent. Honeywell brought a test case against Minolta and won a $150 million settlement. Casio, a small player in the camera market, panicked and rather than fight Honeywell closed the camera plant and dropped out of the camera game. Anybody interested in the activity of the "Newco" please get in touch and I will provide a link. Lee Tracey - [email protected]

User avatar
HermannSW
Posts: 1653
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Cheapest High speed camera up to 900 FPS!!

Wed Dec 06, 2017 11:45 am

RaTTuS wrote:
Tue May 05, 2015 4:09 pm
viewtopic.php?f=43&t=109137 is going to be your best bet
Yes it is, "raspiraw" is cool and the only option allowing to capture camera image sequences without GPU code access.

The trick is to store the captured frames under "/dev/shm" ramdisk.
That way you can store raspiraw frames with a save rate of 1 (all frames).
Because of /dev/shm size you can record videos up to 4s on Pi Zero and up to 9s on Pi 2B.
Then after recording completed, post processing can happen.
viewtopic.php?f=43&t=199204

No new camera is needed, the good old Raspberry v1 camera can record videos with up to 645fps:
  • 640x240 stretched to 640x480 at 180fps
  • 640x120 stretched to 640x240 at 360fps (patch for you to try here)
    Image
  • 640x64 stretched to 640x128 at 600fps (only Pi 2B and 3B, not Pi Zero), here
    Image

If processing the frames on the fly and no storing is needed (like for robot control), then of course the high fps time is unlimited:
viewtopic.php?f=43&t=189661#p1231151
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

User avatar
HermannSW
Posts: 1653
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Cheapest High speed camera up to 900 FPS!!

Fri Dec 08, 2017 1:53 pm

Here are some more 600fps videos, starting from this posting:
viewtopic.php?f=43&t=109137&start=300#p1243429

Candle light video with camera rotated 90°, animated .gif with 25fps, 24x slower than real:
Image

You can use 600fps mode for power line frequency detection as well by direct into lamp video.
In Germany we have 50Hz, with repeating pattern all 600/(50*2)=6 frames in 600fps video.
If repeating pattern happens every 5 frames then power line frequency is 60Hz.
Image

2000rpm gear motor validation by counting frame number per 1 motor axis rotation (here between 16 and 17):
Image

The new modes (360fps, 600fps) qualify Raspberry v1 camera as high speed camera (>250fps):
https://en.wikipedia.org/wiki/High-speed_camera
Last edited by HermannSW on Fri Dec 08, 2017 2:38 pm, edited 1 time in total.
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 24130
Joined: Sat Jul 30, 2011 7:41 pm

Re: Cheapest High speed camera up to 900 FPS!!

Fri Dec 08, 2017 2:05 pm

HermannSW wrote:
Fri Dec 08, 2017 1:53 pm
Here are some more 600fps videos, starting from this posting:
viewtopic.php?f=43&t=109137&start=300#p1243429

Candle light video with camera rotated 90°, animated .gif with 25fps, 24x slower than real:
Image

You can use 600fps mode for power line frequency detection as well by direct into lamp video.
In Germany we have 50Hz, with repeating pattern all 600/(50*2)=6 frames in 600fps video.
If repeating pattern happens every 5 frames then power line frequency is 60Hz.
Image

2000rpm gear motor validation by counting frame number per 1 motor axis rotation (here between 16 and 17):
Image
The weird labyrinth like artefacts on the images is down to a inadequate/too simplistic debayer algorithm IIRC. Not sure what you are using for that, but might be worth looking in to more sophisticated ones.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed. Here's an example...
“I think it’s wrong that only one company makes the game Monopoly.” – Steven Wright

User avatar
HermannSW
Posts: 1653
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Cheapest High speed camera up to 900 FPS!!

Fri Dec 08, 2017 2:43 pm

> The weird labyrinth like artefacts on the images is down to a inadequate/too simplistic debayer algorithm IIRC.
>
Good to know.

> Not sure what you are using for that, but might be worth looking in to more sophisticated ones.
>
I use 6by9's patched version of dcraw, that allows to process frames captured with "raspiraw -hd ...":
https://github.com/6by9/RPiTest
⇨https://stamm-wilbrandt.de/en/Raspberry_camera.html

https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://stamm-wilbrandt.de/github_repo_i420toh264
https://github.com/Hermann-SW/fork-raspiraw
https://twitter.com/HermannSW

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 7531
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Cheapest High speed camera up to 900 FPS!!

Fri Dec 08, 2017 4:05 pm

I'm not sure it is demosaicing at fault. I was fiddling yesterday trying to get a 320x240 mode running and was observing similar issues through the ISP. The standard mode 7 (VGA @ 90fps) doesn't suffer it, but as soon as you increase the skipping factor further on the sensor it throws it off, and it didn't seem stable.
I also haven't worked out why HermannSW's modes also appear to change the Bayer order, and I suspect that may be linked.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Return to “General discussion”