How do you compute the maximum combination of rtsp input video that the RPI can simultaneously service.

[cmisip]

I managed to get 1920x1080 motion vector detection in zoneminder with a ring buffer of 150 RGB frames ( it had to be big to avoid buffer overruns but I am also running all this on an sd card instead of a hard disk) at 15 fps ( I have throttled the source to 15 fps for now). Its early so there are probably bugs yet (web UI is messed up yet) and I haven't obviously done a lot of peformance testing. It shouldn't be possible to run this configuration except I cheated and resized the video to 640x360 prior to motion vector extraction thereby reducing the size of each RGB frame. It is also simultaneously encoding mp4 at 1920x1080 resolution. I will be trying next to build a function for I420 to jpeg which would be able to save jpegs at 1920x1080 and I think since it would be faster than the current RGB to jpeg function. . Currently jpegs are stored at 640x360. I am seeing CPU utilization of 40% and memory use at 25%. I think I could possibly be doing two streams at this resolution just given the CPU and memory usage. I am not sure if the RPI video bandwidth will allow this though. My initial goal was 4 640x360 streams or 3 704x420 streams or 2 1270x720 or 1 1920x1080 stream combinations. What are the theoretical limits for video bandwidth along with their corresponding gpu mem settings? How does input fps affect the calculations. I am hoping for 20 fps input rtsp streams running near realtime.

Thanks,
Chris

[cmisip]

I am going to try to answer my own question here. Correct me if I am wrong.

The video decoder and encoder share the same block. They share the same resource but not the same bandwidth. The decoder can do 1920x1080X60 or 124 Mpixels/s.

1080P60 decode is aided by automatically overclocking when the codec detects a "hard stream". Multiple decodes won't be seen as hard, so you may want to manually overclock.

Code: Select all

Decoding 1 rtsp stream at 1920x1080x25 or  1920x1080x25 = 51.84 Mpixels/s

The video encoder can do 1920x1080x30 or 62.2 Mpixels/s

Code: Select all

Encoding 640x360x25 = 5.76 Mpixels/s 

vc.ril.isp has a bandwidth of 200 Mpixels/s:

Code: Select all

Resizing and converting to RGB with vc.ril.isp from 1920x1080 to 640x368 = 51.84 Mpixels/s

The JPEG encoder block can service 120 MPixels/s:

Code: Select all

Encoding 640x360 RGB to JPEG = 5.76 Mpixels/s

If the above is correct, it seems that the maximum simultaneous 1920x1080x25 streams for decoding possible is 2. At 1270x720x25, it is 5. At 704x480x25 it is 15. It seems that I will not reach the hardware bandwidth limit for any of the combinations I hope possible. The JPEG encoder serializes client requests but it won't max out until there are 20 simultaneous service requests at 640x360 so for my purposes, it may as well be considered parallel.

I am still not sure how the GPU_MEM factors into this.

Thanks,
Chris

[ejolson]

I am going to try to answer my own question here. Correct me if I am wrong.

The video decoder and encoder share the same block. They share the same resource but not the same bandwidth. The decoder can do 1920x1080X60 or 124 Mpixels/s.

Code: Select all

Decoding 1 rtsp stream at 1920x1080x25 or  1920x1080x25 = 51.84 Mpixels/s

The video encoder can do 1920x1080x30 or 62.2 Mpixels/s

Code: Select all

Encoding 640x360x25 = 5.76 Mpixels/s 

vc.ril.isp has a bandwidth of 200 Mpixels/s:

Code: Select all

Resizing and converting to RGB with vc.ril.isp from 1920x1080 to 640x368 = 51.84 Mpixels/s

The JPEG encoder block can service 120 MPixels/s:

Code: Select all

Encoding 640x360 RGB to JPEG = 5.76 Mpixels/s

If the above is correct, it seems that the maximum simultaneous 1920x1080x25 streams for decoding possible is 2. At 1270x720x25, it is 5. At 704x480x25 it is 15. It seems that I will not reach the hardware bandwidth limit for any of the combinations I hope possible. The JPEG encoder serializes client requests but it won't max out until there are 20 simultaneous service requests at 640x360 so for my purposes, it may as well be considered parallel.

I am still not sure how the GPU_MEM factors into this.

Thanks,
Chris

It seems well reasoned. It is actually possible to encode multiple video streams simultaneously using the same GPU? It will be interesting to hear your progress. Have you found that the GPU jpeg encoders and decoders are reliable enough?

[cmisip]

It is possible to run components in parallel processes. And in each process, you can chain multiple components. I was able to run 4 cameras simultaneously in 640x360 last year when I started looking into this. I'm just continuing the project, sans the multiple cameras which have shipped out.

I have just been running the code with the mmal jpeg encoder for a day but it seems to be doing fine so far. I will continue to test.

Chris

[cmisip]

I seem to be getting some tearing with the jpegs produced by the jpeg encoder. Don't know what is causing this.

I'll start a new thread.

Chris