On Hands and Knees Begging for a proper-Good Camera

[htcohio]

Negativity?

I apologize if I come across that way. I'm just trying to understand if there is a future road map for the camera CSI port and official cameras.

This is for hobby, makers and development. Also, the current platform with RPI it's proven and works really well.

It just seems like there are quite a few alternative options rpif could consider and I'm just interested in what the community thoughts are.

[HermannSW]

Here is a Starvis demo video. (Imx290).

https://youtu.be/6gwt7mFOxBs

Wow, now I understand why you want WDR/low light support -- a picture says more than 1000 words, from 2:22min of that video:



I cannot answer your questions directed to RPF, and I am not sure whether 6by9 or Jamesh are allowed to answer (I am working for IBM, and for IBM products only product management is allowed to make any future directed statements) -- but they will have a product management as well.


I just realized that your interest is in IMX290, and v2 camera was IMX219 -- in last package I got with samples from Arducam I did not only get 2xov5647 and 2xov9281 stereo cameras for stereo hat, I got a 16MP color rolling shutter IMX298 camera as well. Bigger number than IMX290, but maybe a bit comparable feature wise.

IMX298 allows for [email protected]:
https://www.uctronics.com/index.php/ard ... ry-pi.html


What I learned from v1 and v2 cameras is, that you can increase framerate by reducing vertical resolution, see raspiraw diagram at bottom. As long as you keep below 200fps raspivid has same behavior (I was able to capture [email protected] with v1 camera with raspivid, that is restricted to 90fps for 640x480 -- I added a mode8 to userland fork and built my own version of raspivid). When back at home tomorrow evening I will run Arducam "list_format" for the IMX298 and see what video modes are exposed. And I will do some framerate measurements for modes in your 1-2MP target range for the IMX298 with Arducam Mipi driver.

<EDIT>
Not needed, found the available video modes of IMX298 list_format command at 2:14min od Arducam demo linked in product description. Only modes in your target range are 1280x720=1MP(0.88) and 1920x1080=2MP. Will see what the framerate is with those modes:
https://www.youtube.com/watch?v=XJ2VrwXMhy4

</EDIT>


With raspiraw you can get up to 750fps/1007fps with v1/v2 camera (cannot be used for streaming because of post processing needed):
https://www.raspberrypi.org/forums/view ... 5#p1320034

[jbeale]

The official RPi team doesn't talk much about future product plans. The only person who even really drops hints is Eben, and they're pretty vague at that. The rest of us can speculate freely, but I'm not sure if it's useful.

For what it's worth, I have used quite a few RPi cameras, mostly as home-security devices. For that and most other non-FPV applications the latency in milliseconds is not relevant. The only possible benefit to what you propose for me would be low-light performance, but you can do even better with current IP cameras. For example if I wanted a 2MP camera good in low light, I would get a Dahua IPC-HDW4233 for about $50 (or the 4MP HDW4433 for a little more). Those cameras have a weatherproof case for mounting anywhere, it's all ready to plug into a PoE port, and it just works. To match that starting with a $25 Pi + $25 camera I still need to add IR illuminator, some kind of case, and the PoE solution so it would be a lot more work and still end up more expensive.

[jamesh]

As always, we do not comment on future products, roadmaps, or timescales.

So you can ask away, but we cannot say anything. We won't even say if what you are suggesting is likely, unlikely, mad, or sensible.

[Pilotnbr1]

I get it.. With 25 million pi sold to date, the drone crowd is probably a small niche market. The pi are always fairly lower priced bits of hardware as well so they have to be low margin, high volume, sales that guide decisions.
Seems to me the v2 cam seems long in the tooth, circa 2016.. Just from a marketing standpoint I would image when the stockpile from the current run on v2 get low, thats when its time for a v3. I know the numbers have to work and I know you guys in the know cant really say.

At any rate, thanks for the responses- this kind of interaction with current devs of a product I own is a rare privilege

[6by9]

Where it is feasible to support alternative suppliers, we do. Hence why there are so many alternative cases and HATs out there. There is no need for RPT to make them all.
It is possible for third parties to make modules and drivers for the Pi should they wish so, but there is a potentially significant support burden.
We are working with the libcamera project to try and fit in with the standard approach for Linux systems and complex camera setups, so that should hopefully reduce the overhead (https://mobile.twitter.com/gsholling/st ... 681280?p=v)

v2 sensor was introduced 2014 as OV5647 went EOL. Supply information from Sony isn't something I can share.
Yes it's getting a little old, but still just as functional as when introduced.

[narpat007]

Yes, we need it desperately , please help us

[jamesh]

Yes, we need it desperately , please help us

Please read all previous posts in this thread.

[fruitoftheloom]

Yes, we need it desperately , please help us

The next version of the Camera when it is released will be advertised on the Official Blog:

https://www.raspberrypi.org/blog/

Begging and putting stupid emojis in a post is just plainly irritating and annoying, especially as you have already posted the same previously in this post

[htcohio]

Here is a Starvis demo video. (Imx290).

https://youtu.be/6gwt7mFOxBs

Wow, now I understand why you want WDR/low light support -- a picture says more than 1000 words, from 2:22min of that video:



I cannot answer your questions directed to RPF, and I am not sure whether 6by9 or Jamesh are allowed to answer (I am working for IBM, and for IBM products only product management is allowed to make any future directed statements) -- but they will have a product management as well.


I just realized that your interest is in IMX290, and v2 camera was IMX219 -- in last package I got with samples from Arducam I did not only get 2xov5647 and 2xov9281 stereo cameras for stereo hat, I got a 16MP color rolling shutter IMX298 camera as well. Bigger number than IMX290, but maybe a bit comparable feature wise.

IMX298 allows for [email protected]:
https://www.uctronics.com/index.php/ard ... ry-pi.html


What I learned from v1 and v2 cameras is, that you can increase framerate by reducing vertical resolution, see raspiraw diagram at bottom. As long as you keep below 200fps raspivid has same behavior (I was able to capture [email protected] with v1 camera with raspivid, that is restricted to 90fps for 640x480 -- I added a mode8 to userland fork and built my own version of raspivid). When back at home tomorrow evening I will run Arducam "list_format" for the IMX298 and see what video modes are exposed. And I will do some framerate measurements for modes in your 1-2MP target range for the IMX298 with Arducam Mipi driver.

<EDIT>
Not needed, found the available video modes of IMX298 list_format command at 2:14min od Arducam demo linked in product description. Only modes in your target range are 1280x720=1MP(0.88) and 1920x1080=2MP. Will see what the framerate is with those modes:
https://www.youtube.com/watch?v=XJ2VrwXMhy4

</EDIT>


With raspiraw you can get up to 750fps/1007fps with v1/v2 camera (cannot be used for streaming because of post processing needed):
https://www.raspberrypi.org/forums/view ... 5#p1320034

@HermannSW,

Thank you for the info!


Are you able to stream 720p H.264 60fps?

If so, do you have control over the GOP keyframes? That is probably the single most important parameter when using it for a UAV or robotic application because there are typically complex scene changes and momentary breakups, if for example on a webcam or IP camera the "GOP is set to "1" if it's set up 30fps, that equals 30 frames you have to wait before the video recovers.

Raspivid allows down to every 2 individual frames so at 60fps Video recovers within 100ms.

[HermannSW]

Are you able to stream 720p H.264 60fps?

Likely.

I did run list_format for UC-554 13MP rolling shutter IMX135 first, these are the available modes:

Code: Select all

mode: 0, width: 1280, height: 720, pixelformat: pRAA, desc: (null)
mode: 1, width: 1920, height: 1080, pixelformat: pRAA, desc: (null)
mode: 2, width: 2080, height: 1536, pixelformat: pRAA, desc: (null)
mode: 3, width: 3264, height: 2464, pixelformat: pRAA, desc: (null)
mode: 4, width: 4192, height: 3120, pixelformat: pRAA, desc: (null)

For the UC-572 16MP rolling shutter IMX298 these are the available modes as in screenshot:

Code: Select all

mode: 0, width: 1280, height: 720, pixelformat: pRAA, desc: (null)
mode: 1, width: 1920, height: 1080, pixelformat: pRAA, desc: (null)
mode: 2, width: 4672, height: 3496, pixelformat: pRAA, desc: (null)

I did run "./video" that captures 10 seconds of .h264 video onto SD card.
On a static view it gives"Total frame count = 742" for "TimeElapsed = 10.000118".
I heavily moved my hand in view and the frame number dropped to 59fps:

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ./video
Open camera...
Setting the resolution...
Current resolution is 1280x720
Notice:You can use the list_format sample program to see the resolution and control supported by the camera.
Start preview...
Enable Software Auto Exposure...
Enable Software Auto White Balance...
Start video encoding...
Total frame count = 592
TimeElapsed = 10.000099
Stop video encoding...
Close camera...
[email protected]:~/MIPI_Camera/RPI $

So whether you can achieve 60fps depends on scene and action.

If so, do you have control over the GOP keyframes? That is probably the single most important parameter when using it for a UAV or robotic application because there are typically complex scene changes and momentary breakups, if for example on a webcam or IP camera the "GOP is set to "1" if it's set up 30fps, that equals 30 frames you have to wait before the video recovers.

Raspivid allows down to every 2 individual frames so at 60fps Video recovers within 100ms.

I don't know, these are the frame types of recorded video:

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ffprobe -loglevel quiet -show_frames test.h264 | grep pict_type > out
[email protected]:~/MIPI_Camera/RPI $ wc --lines out
591 out
[email protected]:~/MIPI_Camera/RPI $ sort out | uniq -c
     10 pict_type=I
    581 pict_type=P
[email protected]:~/MIPI_Camera/RPI $ 

Besides "video.c" there is "video2stdout.c" as well, so you can pipe video remotely eg. using netcat.

[htcohio]

Are you able to stream 720p H.264 60fps?

Likely.

I did run list_format for UC-554 13MP rolling shutter IMX135 first, these are the available modes:

Code: Select all

mode: 0, width: 1280, height: 720, pixelformat: pRAA, desc: (null)
mode: 1, width: 1920, height: 1080, pixelformat: pRAA, desc: (null)
mode: 2, width: 2080, height: 1536, pixelformat: pRAA, desc: (null)
mode: 3, width: 3264, height: 2464, pixelformat: pRAA, desc: (null)
mode: 4, width: 4192, height: 3120, pixelformat: pRAA, desc: (null)

For the UC-572 16MP rolling shutter IMX298 these are the available modes as in screenshot:

Code: Select all

mode: 0, width: 1280, height: 720, pixelformat: pRAA, desc: (null)
mode: 1, width: 1920, height: 1080, pixelformat: pRAA, desc: (null)
mode: 2, width: 4672, height: 3496, pixelformat: pRAA, desc: (null)

I did run "./video" that captures 10 seconds of .h264 video onto SD card.
On a static view it gives"Total frame count = 742" for "TimeElapsed = 10.000118".
I heavily moved my hand in view and the frame number dropped to 59fps:

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ./video
Open camera...
Setting the resolution...
Current resolution is 1280x720
Notice:You can use the list_format sample program to see the resolution and control supported by the camera.
Start preview...
Enable Software Auto Exposure...
Enable Software Auto White Balance...
Start video encoding...
Total frame count = 592
TimeElapsed = 10.000099
Stop video encoding...
Close camera...
[email protected]:~/MIPI_Camera/RPI $

So whether you can achieve 60fps depends on scene and action.

If so, do you have control over the GOP keyframes? That is probably the single most important parameter when using it for a UAV or robotic application because there are typically complex scene changes and momentary breakups, if for example on a webcam or IP camera the "GOP is set to "1" if it's set up 30fps, that equals 30 frames you have to wait before the video recovers.

Raspivid allows down to every 2 individual frames so at 60fps Video recovers within 100ms.

I don't know, these are the frame types of recorded video:

Code: Select all

[email protected]:~/MIPI_Camera/RPI $ ffprobe -loglevel quiet -show_frames test.h264 | grep pict_type > out
[email protected]:~/MIPI_Camera/RPI $ wc --lines out
591 out
[email protected]:~/MIPI_Camera/RPI $ sort out | uniq -c
     10 pict_type=I
    581 pict_type=P
[email protected]:~/MIPI_Camera/RPI $ 

Besides "video.c" there is "video2stdout.c" as well, so you can pipe video remotely eg. using netcat.

Hello,

Thank you for the information, I didn't realize you had replied again sorry for the delay.

I think that the thing we are trying to emphasize is that yes there are all types of different video pipelines and configurations available but, there is just that "little bit more" we are able to achieve with "raspivid" pipeline through a raw socket and injected frames of a fixed size (1024 bytes)

Example:
nice -n -9 raspivid -w $WIDTH -h $HEIGHT -fps $FPS -b $BITRATE -g $KEYFRAMERATE -t 0 $EXTRAPARAMS -o - | nice -n -9 /root/wifibroadcast/tx_rawsock -p 0 -b $VIDEO_BLOCKS -r $VIDEO_FECS -f $VIDEO_BLOCKLENGTH -t $VIDEO_FRAMETYPE -d $VIDEO_WIFI_BITRATE -y 0 $NICS

On the recieving end we use the RPI specific hello_video driver for playback (modified and referenced somewhat here)
viewtopic.php?t=156495

Overall, just sticking to all hardware and drivers specific to the Raspberry Pi allow us to achieve video latency, and smooth, stutter free video that would be widely accepted for long range UAV drone or other applications.

Of course there are any number of ways that video can be transmitted and some alternative cameras as well but, each of them seem to have that one or two drawbacks compared to the official pi cam...

There is just some unknown magic that happens at the GPU level that can't quite be achieved with the above-referenced alternatives.
*It gets close and it's fairly usable...

I am just personally curious what we might expect in the future considering that the broadcom team that originally developed the SOC GPU side of things was shut down, does that pretty much mean the system and MIPI CSI-2 is kind of stuck as-is to some extent?

Thanks for the replies