Streaming mjpg from Raspberry Cam

[TonAu]

Code: Select all

/opt/vc/bin/vcgencmd version
May 24 2013 16:51:23
Copyright (c) 2012 Broadcom
version 6676ab07e56aeec897f23d09b8fb87b55cbec981 (clean) (release)

So a bit older. But it is the very first model B.

I tried again, so started without any options (no x no y no d) with the default 1000ms and after 800 frames or so it went black again. I stopped and restarted mjpg-streamer again and now it has been working well for hours. So rather random

The link you gave describes the problem exactly.

[jacksonliam]

I think it's a general raspberry pi camera issue rather than anything I can fix, unfortunately! Running rpi-update will put you on the newer GPU firmware. https://github.com/Hexxeh/rpi-update/

[TarjeiB]

How did you achieve this >~7fps? I've been playing with the code with aim to write an mjpeg streamer input plugn and in a simple capture->encode loop the fastest i seem to get is about 12fps. Resolution makes very little difference here, and it requires around a 100ms delay between captures else the picture is too dark or just black.


For anyone who is interested, here's my work on an mjpeg streamer plugin for raspicam https://github.com/jacksonliam/mjpg-streamer

Try this:

Code: Select all

$ raspistill -t 999999 -tl 50 -o /tmp/mjpg/test.jpg -n -w 960 -h 720&
$ mjpg_streamer -i 'input_file.so -f /tmp/mjpg -d 0'&

This seems to give ~15fps (but I haven't measured it exactly). It may be as low as 10 or maybe even 7, as you're experiencing.
If I go lower than -tl 50 I get the problem that the images darken significantly or are completely black, probably because of the raspistill not having time to adjust for light variations.

I'll check out your plugin, that looks absolutely fantastic - being able to stream mjpg "properly" without stopping by the filesystem would be perfect.

[Lori]

Thanks for the raspi input plugin really useful!
I am working on something similar myself, so an actually working example helps me a lot xD

I took a quick look at the code, and if you change the delay var to 50 instead of 1000 it gives me a fps of ~15 frames.
usleep(1000 * 1000) (ln 765) gives an 1 second delay, that takes the fps down a lot

Edit: 15fps at 640x480, at 1280x720 it's ~13fps

[jacksonliam]

You can specify that delay on the command line at the same place as the resolution, so no need to compile! I'maafraid any delay less than 100 just gives me a black image.

[mikerr]

Try this:

Code: Select all

$ raspistill -t 999999 -tl 50 -o /tmp/mjpg/test.jpg -n -w 960 -h 720&
$ mjpg_streamer -i 'input_file.so -f /tmp/mjpg -d 0'&

I'll check out your plugin, that looks absolutely fantastic - being able to stream mjpg "properly" without stopping by the filesystem would be perfect.

/tmp is on the SD card better to use /run/shm which is a 25MB ramdisk included in raspbian.

[TarjeiB]

/tmp is on the SD card better to use /run/shm which is a 25MB ramdisk included in raspbian.

My initial instructions also included making /tmp into a ramdisk

[Lori]

You can specify that delay on the command line at the same place as the resolution, so no need to compile! I'maafraid any delay less than 100 just gives me a black image.

oh ofc *facepalm*
You could take a look at tasanakorn's code: http://www.raspberrypi.org/phpBB3/viewt ... 2&p=356842
He used RaspiVid instead of RaspiStill (I think that one has a less heavy encoder so it's faster, and video needs lesser adjusting for lighting and such(?)). But his code is kind of messy.. -though the functionality is similar to yours, only your code is better readable

[TarjeiB]

oh ofc *facepalm*
You could take a look at tasanakorn's code: http://www.raspberrypi.org/phpBB3/viewt ... 2&p=356842
He used RaspiVid instead of RaspiStill (I think that one has a less heavy encoder so it's faster, and video needs lesser adjusting for lighting and such(?)). But his code is kind of messy.. -though the functionality is similar to yours, only your code is better readable

But that will require CPU reencoding to JPG, since the raspivid provides h264. For mjpg-streaming with no CPU usage it's just raspistill that will do (for now)

[Lori]

oh ofc *facepalm*
You could take a look at tasanakorn's code: http://www.raspberrypi.org/phpBB3/viewt ... 2&p=356842
He used RaspiVid instead of RaspiStill (I think that one has a less heavy encoder so it's faster, and video needs lesser adjusting for lighting and such(?)). But his code is kind of messy.. -though the functionality is similar to yours, only your code is better readable

But that will require CPU reencoding to JPG, since the raspivid provides h264. For mjpg-streaming with no CPU usage it's just raspistill that will do (for now)

Yup you're right.. (i tried it out ) ..runs at 100% cpu with not-so-high fps
The interesting thing about working with vid is that it does not require the worker thread to do something, so you can put OpenCV detection there, for example. (Without interrupting the streaming fps)
I guess I'll try another aproach

btw, I found a little mistake in the input_init function: You might want to assign the plugin_number with:

Code: Select all

plugin_number = plugin_no;

Not sure if that helps much with the black frames, but the other plugins have this line too.

[jacksonliam]

oh ofc *facepalm*
You could take a look at tasanakorn's code: http://www.raspberrypi.org/phpBB3/viewt ... 2&p=356842
He used RaspiVid instead of RaspiStill (I think that one has a less heavy encoder so it's faster, and video needs lesser adjusting for lighting and such(?)). But his code is kind of messy.. -though the functionality is similar to yours, only your code is better readable

But that will require CPU reencoding to JPG, since the raspivid provides h264. For mjpg-streaming with no CPU usage it's just raspistill that will do (for now)

Yup you're right.. (i tried it out ) ..runs at 100% cpu with not-so-high fps
The interesting thing about working with vid is that it does not require the worker thread to do something, so you can put OpenCV detection there, for example. (Without interrupting the streaming fps)
I guess I'll try another aproach

btw, I found a little mistake in the input_init function: You might want to assign the plugin_number with:

Code: Select all

plugin_number = plugin_no;

Not sure if that helps much with the black frames, but the other plugins have this line too.

Thanks, added. For some reason the test_picture plugin I based this off doesn't have that line :/

[TonAu]

Hi,

After a couple of restarts with added parameters like "input_raspicam.so -ex off" during stopping the streamer with CTRL-C I see :

Code: Select all

 
DBG(/home/pi/mjpg-streamerTA/plugins/input_raspicam/input_raspicam.c, worker_cleanup(), 865): cleaning up ressources allocated by input thread
 o: cleaning up ressources allocated by server thread #00
Segmentation fault
pi@raspberry

Then during restarting with a normal commandline the cam hangs at

Code: Select all

 
 i: Starting Camera
 DBG(/home/pi/mjpg-streamerTA/plugins/input_raspicam/input_raspicam.c, worker_thread(), 530): Host init, starting mmal stuff

[jacksonliam]

Yeah If it doesn't clean up properly you're gunna have to restart your Pi to make the camera work again. I had that problem a couple of times during writing this.

No idea why it would segfault though, perhaps everything isn't getting cleaned up before mjpeg_streamer kills the plugin :/ I have managed to replicate the issue though so I'll take a closer look.

[ds18s20]

Could someone clarify on the load consequences of using the camera module to other functions. I read the module uses the GPU's i2c interface for the bitmap payload; what other function would a constant i2c use "kill" so to speak if one constantly streams from the camera (one day when v4l is released and true full frame streaming becomes a reality)?

For example; if you constantly copy large data back and forth from a big USB stick to SD that nearly kills your Ethernet performance since it's all hooked to the same USB hub. Is there a similar consequence when talking about constant full payload using the camera module?

[custom]

I have the same issue. Images are getting darker after a a few seconds. I'm using libjpeg62-dev.

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 3.6.11+ #484 PREEMPT Mon Jun 24 15:45:35 BST 2013 armv6l GNU/Linux

root@raspberrypi:~# /opt/vc/bin/vcgencmd version
Jun 24 2013 15:58:41 
Copyright (c) 2012 Broadcom
version b03d7b5780a1f8610bb40394615df57c693726a1 (clean) (release)

[jacksonliam]

I have the same issue. Images are getting darker after a a few seconds. I'm using libjpeg62-dev.

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 3.6.11+ #484 PREEMPT Mon Jun 24 15:45:35 BST 2013 armv6l GNU/Linux

root@raspberrypi:~# /opt/vc/bin/vcgencmd version
Jun 24 2013 15:58:41 
Copyright (c) 2012 Broadcom
version b03d7b5780a1f8610bb40394615df57c693726a1 (clean) (release)

Is that using my raspicam plugin for mjpeg streamer experimental?
If so, try with "input_raspicam.so -d 2000"
That should give a two second delay between frames captured at 640x480. If they still get dark I'm not sure what can be done, its a firmware issue IIRC.

[custom]

Yes, I'm using your plugin.

I tried with the following command and it also happens after 10-20 seconds.

Code: Select all

./mjpg_streamer -o "output_http.so -w ./www" -i "input_raspicam.so -d 2000 -hf -vf"

[Lori]

I have the same issue. Images are getting darker after a a few seconds. I'm using libjpeg62-dev.

Code: Select all

root@raspberrypi:~# uname -a
Linux raspberrypi 3.6.11+ #484 PREEMPT Mon Jun 24 15:45:35 BST 2013 armv6l GNU/Linux

root@raspberrypi:~# /opt/vc/bin/vcgencmd version
Jun 24 2013 15:58:41 
Copyright (c) 2012 Broadcom
version b03d7b5780a1f8610bb40394615df57c693726a1 (clean) (release)

I have libjpeg8-dev, and I don't have problems with black images or whatsoever. Maybe that is the problem?

*That or imagemagick?

[jacksonliam]

I'm pretty sure the version of libjpeg has very little to do with it as I don't think mjpeg streamer should be touching the image. You could try using a bigger image size 1920 X 1080 works for me! Do you get the same issues using raspistill time lapse mode to store Jpeg?

[custom]

I'm pretty sure the version of libjpeg has very little to do with it as I don't think mjpeg streamer should be touching the image. You could try using a bigger image size 1920 X 1080 works for me! Do you get the same issues using raspistill time lapse mode to store Jpeg?

The images are also getting dark with 1920x1080. I did a short timelapse test (50 frames) and I did not this get this problem. But I'll do a longer test tomorrow.

[jbeale]

Is there a difference in the "image getting darker over time" problem when you operate the Pi headless, vs. with a display attached?

[prateep]

Hi all,

I'm trying out some of the examples provided in this thread and I'm seeing what seems to be very slow behavior.

- latest Raspbian
- latest firmware
- raspistill outputting to /run/

In the output folder, i'm consistently seeing periods of time where the output file is 0 bytes. I'm interpreting that to mean that i'm getting < 1fps

Any thoughts would be appreciated

Code: Select all

pi@raspi2 /run/stream $ pwd
/run/stream

pi@raspi2 /run/stream $ ps -ef | grep raspistill
pi        4953  4952  1 21:45 pts/1    00:00:07 raspistill -rot 180 -n -w 640 -h 480 -q 5 -o /run/stream/pic.jpg -th 0:0:0 -tl 100 -t 9999999

pi@raspi2 /run/stream $ while true; do ls -laF pic.jpg ; sleep 1; done
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6383 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 0 Jul  7 21:57 pic.jpg
-rw-r--r-- 1 pi pi 6298 Jul  7 21:57 pic.jpg

[TarjeiB]

Hi all,

In the output folder, i'm consistently seeing periods of time where the output file is 0 bytes. I'm interpreting that to mean that i'm getting < 1fps

You may want to test it out with not overwriting the same file, this may skew your results in the ls - bit as it could be just about to write the file again. Try with the filename set to pic%d.jpg to get an incremental number.

[JamesRonald]

Liam,

Good work and thank you for making input_raspicam available. I'm currently running it on my RoombaPi robot and I am getting about 8.5 FPS. I am not seeing the picture go dark over time as others have reported. I am running up to date raspbian on a Rev b Version 2 with the camera mounted upside down for ease of mounting.

mjpg_streamer -o "output_http.so" -i "input_raspicam.so -vf -hf -d 80"

- Jim

[jacksonliam]

No problem, glad it's working okay for you! You might be able to squeeze a few more fps or use a higher resolution for a bit more detail without issue, I'm going to spend time on this again soon, though might wait for the v4l2 driver to see what's possible with that...