Streaming mjpg from Raspberry Cam


207 posts   Page 1 of 9   1, 2, 3, 4, 5 ... 9
by TarjeiB » Mon May 27, 2013 11:19 am
I managed to get a mjpg-stream from the Raspberry Cam by (ab)using the timelapse function and using mjpg-streamer. The CPU usage is very low (~1%) but it's not at all optimal, the speed is quite bad. Does someone have any idea on how to make a higher FPS solution?

First make sure you have a /tmp or other location that writes to memory instead of thje SD card:

/etc/default/tmpfs:
Code: Select all
# mount /tmp as a tmpfs.  Defaults to no; set to yes to enable (/tmp
# will be part of the root filesystem if disabled).  /tmp may also be
# configured to be a separate mount in /etc/fstab.
RAMTMP=yes

Reboot.

Then run raspistill in timelapse mode with the desired parameters:

Code: Select all
raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50&


This outputs a JPG picture to /tmp/mjpg/test.jpg every half second. Experiment with the -tl parameter to see how fast you can make it go.

And mjpg_streamer with the desired parameters:

Code: Select all
mjpg_streamer -i 'input_file.so -f /tmp/mjpg -r'


This reads the current image in /tmp/mjpg, outputs it to the mjpg-stream, then deletes it (-r). As soon as raspistill writes a new image, it reads that, outputs it, and deletes it. Etc, etc.

Access the stream by putting
Code: Select all
http://<raspberrypi>:8080/?action=stream
in a compatible browser, like Chrome, and Firefox, or VLC.

This produces as high quality a MJPG-stream you want, but due to the writing and deleting of images, even on ramdisk, it's not giving much of a FPS. Hopefully someone has a better idea.
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by pi-man-uk » Mon May 27, 2013 11:53 am
on first look - works a treat for me, low cpu usage, v good quality and wide browser support.
well done. many thanks.
Posts: 62
Joined: Tue May 08, 2012 1:41 pm
by pi-man-uk » Tue May 28, 2013 7:37 am
Quick question. As I said, this method works great for my project. One problem at present.

I want the stream to run forever. Currently the -t 999999 switch seems to knock out the raspistill command after a period of time. Is that right and is there a way to have it running without timeout. I tried putting a really big number in there but command didn't seem to run after that.

Thanks for any insight.
Posts: 62
Joined: Tue May 08, 2012 1:41 pm
by TarjeiB » Tue May 28, 2013 7:53 am
The -t parameter is in milliseconds, and there's a limit (I think someone mentioned ~26 days?) Don't know how many 9's that work out to.

The binary shouldn't need -t when it has -tl in my opinion, and the old bug that made -1 work for eternal loop would have been nice to have back :)

In any case, if you need it to run longer, you can always use a bash script that just loops it forever.

Code: Select all
#!/bin/bash
while :
do
raspistill -tl 500 -t 999999 -vf -w 960 -h 720 -o /tmp/mjpg/test.jpg -n -q 50
sleep 1
done


I haven't tested that code so can't promise it's bug free :)

*EDIT* Tested and 999999999 works as maximum.
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by pi-man-uk » Tue May 28, 2013 8:01 am
Thanks T - very clear and helpful... will ssh in and give this a try later... as you say shame the -1 trick doesn't work.. that'd do it easily.
Posts: 62
Joined: Tue May 08, 2012 1:41 pm
by jamesh » Tue May 28, 2013 9:21 am
-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...

-t and -tl are different options - you often still want the app the finish, even when in timelapse mode. So useful to have both options.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 16357
Joined: Sat Jul 30, 2011 7:41 pm
by ppumkin » Tue May 28, 2013 9:42 am
Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all
raspivid -o -  \
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg



I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.

I will see what performance I get. then will try to overclock see if it helps. boost your technique and compare mine too.

This is prooving to be a bit of a pain to stream to a browser:(
Posts: 82
Joined: Tue May 29, 2012 10:22 pm
by pi-man-uk » Tue May 28, 2013 10:13 am
Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?
Posts: 62
Joined: Tue May 08, 2012 1:41 pm
by texy » Tue May 28, 2013 10:54 am
pi-man-uk wrote:Thanks. .. trying the -t 999999999 approach right now. Seems to work thanks.

am I right in thinking that is 11 days?


11.57 days by my calculations

T.
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555
Forum Moderator
Forum Moderator
Posts: 4752
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by TarjeiB » Tue May 28, 2013 11:23 am
ppumkin wrote:Do you think it will be a good idea to pipe raspivide to ffmpeg and output jpg files. Then ffmpeg writes as fast as it can to the ram /tmp while mjpg reads and deltes them as fast as possible.

something like

Code: Select all
raspivid -o -  \
ffmpeg -i - \
    -f image2(?) \
    -c:v mjpeg \
    stream%d.jpg


I tried your method and hardly got 1.5fps - that is with 320x240 or 1080p on lan. So there is something peculiar there. possibly raspistill is slow. I just thought of this at work so will try it at home later.


raspivid may work but I think this will eat CPU like a monster as the stream has to be converted from h.264 to jpg.
In any case, the limitation isn't raspistill, but the process of having a program writing to a file, having another program poll for that file, read it and then delete it, and then having raspistill write a new file. It's just a really bad solution in place of missing stdin/stdout functions.

If someone smart wanted to look at the mjpg-streamer input_file.so filter and make it read directly from stdin, we should be golden :) raspistill and mjpg_streamer in themselves both seem really fast and use little to no CPU.
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by LeoWhite » Tue May 28, 2013 7:48 pm
jamesh wrote:-t -1 didn't make it run forever, that would just mean the max possible which is the 23 days or whatever.

I'll add a forever option in the next release.

Although it's a one line change in the code to make it run forever, for people who are desperate for it. Perhaps actually making some efforts yourselves might be appropriate...


I've very quickly added a -notimeout option if you've not already made the above change yourself! patch below. Its only had a few minutes testing, but it certainly running for more than the default 5 seconds. Obviously I should leave my RPi streaming for the next month to see if it gets past the 23 day mark :D


Code: Select all
diff --git a/host_applications/linux/apps/raspicam/RaspiVid.c b/host_applications/linux/apps/raspicam/RaspiVid.c
index 580ceef..bd6cfa0 100755
--- a/host_applications/linux/apps/raspicam/RaspiVid.c
+++ b/host_applications/linux/apps/raspicam/RaspiVid.c
@@ -103,6 +103,7 @@ int mmal_status_to_int(MMAL_STATUS_T status);
 typedef struct
 {
    int timeout;                        /// Time taken before frame is grabbed and app then shuts down. Units are milliseconds
+   int notimeout;                      /// !0 to disable the timeout
    int width;                          /// Requested width of image
    int height;                         /// requested height of image
    int bitrate;                        /// Requested bitrate
@@ -146,6 +147,7 @@ static void display_valid_parameters(char *app_name);
 #define CommandDemoMode     7
 #define CommandFramerate    8
 #define CommandPreviewEnc   9
+#define CommandNoTimeout   10
 
 static COMMAND_LIST cmdline_commands[] =
 {
@@ -159,6 +161,7 @@ static COMMAND_LIST cmdline_commands[] =
    { CommandDemoMode,"-demo",       "d",  "Run a demo mode (cycle through range of camera options, no capture)", 1},
    { CommandFramerate,"-framerate", "fps","Specify the frames per second to record", 1},
    { CommandPreviewEnc,"-penc",     "e",  "Display preview image *after* encoding (shows compression artifacts)", 0},
+   { CommandNoTimeout,"-notimeout", "nt",  "Disable timeout when capturing video", 0},
 };
 
 static int cmdline_commands_size = sizeof(cmdline_commands) / sizeof(cmdline_commands[0]);
@@ -181,6 +184,7 @@ static void default_status(RASPIVID_STATE *state)
 
    // Now set anything non-zero
    state->timeout = 5000;     // 5s delay before take image
+   state->notimeout = 0;      // Default to obeying the timeout
    state->width = 1920;       // Default to 1080p
    state->height = 1080;
    state->bitrate = 17000000; // This is a decent default bitrate for 1080p
@@ -210,7 +214,7 @@ static void dump_status(RASPIVID_STATE *state)
    }
 
    fprintf(stderr, "Width %d, Height %d, filename %s\n", state->width, state->height, state->filename);
-   fprintf(stderr, "bitrate %d, framerate %d, time delay %d\n", state->bitrate, state->framerate, state->timeout);
+   fprintf(stderr, "bitrate %d, framerate %d, time delay %d, no timeout %d\n", state->bitrate, state->framerate, state->timeout, state->notimeout);
 
    raspipreview_dump_parameters(&state->preview_parameters);
    raspicamcontrol_dump_parameters(&state->camera_parameters);
@@ -363,6 +367,10 @@ static int parse_cmdline(int argc, const char **argv, RASPIVID_STATE *state)
          state->immutableInput = 0;
          break;
 
+      case CommandNoTimeout:
+         state->notimeout = 1;
+         break;
+         
       default:
       {
          // Try parsing for any image specific parameters
@@ -1037,12 +1045,11 @@ int main(int argc, const char **argv)
          {
             // Run for the user specific time..
             int num_iterations = state.timeout / state.demoInterval;
-            int i;
 
             if (state.verbose)
                fprintf(stderr, "Running in demo mode\n");
 
-            for (i=0;i<num_iterations;i++)
+            while (state.notimeout || num_iterations--)
             {
                raspicamcontrol_cycle_test(state.camera_component);
                vcos_sleep(state.demoInterval);
@@ -1084,18 +1091,27 @@ int main(int argc, const char **argv)
                // out of storage space)
                // Going to check every ABORT_INTERVAL milliseconds
 
-               for (wait = 0; wait < state.timeout; wait+= ABORT_INTERVAL)
+               for (wait = 0; wait < state.timeout;)
                {
                   vcos_sleep(ABORT_INTERVAL);
                   if (callback_data.abort)
                      break;
+                     
+                  // Only increase the wait time if we are obeying the timeout
+                  if(!state.notimeout) {
+                     wait+= ABORT_INTERVAL;
+                  }
                }
 
                if (state.verbose)
                   fprintf(stderr, "Finished capture\n");
             }
-            else
+            else {
+               // If we are not obeying the timeout then loop forever here
+               do {
                vcos_sleep(state.timeout);
+              } while(!state.notimeout);
+            }
          }
       }
       else
--
1.7.9.5


Leo
Posts: 123
Joined: Mon Apr 08, 2013 9:20 am
by texy » Tue May 28, 2013 7:52 pm
I believe it's 11.57 days ;-)
Texy
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555
Forum Moderator
Forum Moderator
Posts: 4752
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by LeoWhite » Tue May 28, 2013 7:59 pm
texy wrote:I believe it's 11.57 days ;-)


Mmhh I just got 24.85 days as the max :)

Leo
Posts: 123
Joined: Mon Apr 08, 2013 9:20 am
by texy » Tue May 28, 2013 8:08 pm
LeoWhite wrote:
texy wrote:I believe it's 11.57 days ;-)


Mmhh I just got 24.85 days as the max :)

Leo


I thought it was 999999999ms, which is 11.57 days :?:

T.
Various male/female 40- and 26-way GPIO header for sale here ( IDEAL FOR YOUR PiZero ):
https://www.raspberrypi.org/forums/viewtopic.php?f=93&t=147682#p971555
Forum Moderator
Forum Moderator
Posts: 4752
Joined: Sat Mar 03, 2012 10:59 am
Location: Berkshire, England
by LeoWhite » Tue May 28, 2013 8:20 pm
texy wrote:
LeoWhite wrote:
texy wrote:I believe it's 11.57 days ;-)


Mmhh I just got 24.85 days as the max :)

Leo


I thought it was 999999999ms, which is 11.57 days :?:


The timeout is stored as a 32bit integer value, so the maximum possible value should be 2147483647ms. Its just much easier to put in a bunch of 9s than it is to remember that value :)

Not that I've gone and actually tried that myself but having just looked through the code it should work (There does appear to be an issue where the timeout is read in as an unsigned value, which may explain why the -1 thing was working)

Leo
Posts: 123
Joined: Mon Apr 08, 2013 9:20 am
by TarjeiB » Thu May 30, 2013 10:31 am
Digging a bit more here, the input_file.so seems to be using inotify to check if there are new files.
This will never be fast, so without a new plugin we're stuck at horrid framerates.

Unfortunately I'm not at all a coder - I looked through the input_file.c and tried to find examples of how to make C read from stdin, but it's not possible for me to figure out by just reading the code by a long shot.

The most updated mjpg-streamer seems to be the experimental version at github:
https://github.com/codewithpassion/mjpg ... perimental

That is the one I've compiled on the Pi, if someone wants to look at the plugins and see if it's possible to make a input_stdin one it'd be a huge step forward :)
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by AntoineDUBOSTBULARD » Thu May 30, 2013 11:48 am
Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB
Posts: 8
Joined: Wed Apr 10, 2013 7:17 am
by TarjeiB » Thu May 30, 2013 9:26 pm
AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by sharix » Thu May 30, 2013 10:29 pm
There should be an option to only adjust the lighting levels, awb, etc before the first shot instead of before every shot. someone get to it :D
Posts: 199
Joined: Thu Feb 16, 2012 11:29 am
Location: Slovenia
by AntoineDUBOSTBULARD » Fri May 31, 2013 4:58 am
TarjeiB wrote:
AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first thought that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?


TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"

Changing the resolution do not change anything

Thanks

ADB
Posts: 8
Joined: Wed Apr 10, 2013 7:17 am
by TarjeiB » Fri May 31, 2013 1:17 pm
AntoineDUBOSTBULARD wrote:TarjeiB, can you tell me the exact commands your are using for achieving such frame rates ?
Mine is :
$ mkdir /tmp/stream
$ raspistill -w 1920 -h 1080 -q 25 -o /tmp/stream/pic.jpg -tl 40 -t 9999999 -th 0:0:0 | ./mjpg_streamer -i "./input_file.so -f /tmp/stream" -o "./output_http.so -w ./www"
ADB


Code: Select all
$ raspistill -t 999999 -tl 50 -o /tmp/mjpg/test.jpg -n -w 960 -h 720&
$ mjpg_streamer -i 'input_file.so -f /tmp/mjpg -d 0'&


Goes quite fast! Make sure /tmp is a ramdisk. Not sure the pipe is a good idea either, I run them as separate commands.
Posts: 148
Joined: Thu Jul 12, 2012 3:33 pm
by sergkit » Fri Jun 07, 2013 10:13 am
Does anybody rich normal fps with streaming RaspberryCam? I have get only 0.2 fps
Code: Select all
raspistill -t 999999 -tl 50 -o /tmp/test.jpg -n -w 960 -h 720

Code: Select all
./mjpg_streamer -i './input_file.so -f /tmp/ -d 0' -o "./output_http.so -w ./www -p 8080"
Posts: 1
Joined: Fri Jun 07, 2013 10:05 am
by jacksonliam » Fri Jun 07, 2013 4:42 pm
TarjeiB wrote:
AntoineDUBOSTBULARD wrote:Dear all,

According to me low frame rate is frist due to raspistill.
I profiled the code and found that even if the -tl parameter is set to 40ms, we go in the callback every 0.5s.
I first found that the fwrite take a long time but profiling show that write to file takes less than 10ms.
The vcos_sleep seems to be good also.
So does anyone have an idea of why the callback is in the best case achieved 2 times per sedonds ?

Regards

ADB

I get ~7 images per second at 960x720 with -tl 1 writing to a ramdisk. But are we sure this wouldn't go much faster if the output was -o - (stdout)?

In fact after some further testing, I get all the way to 15 fps, but at that point the picture starts getting darker and at ~20+ fps the picture is black. I assume it's because raspistill checks light and adjusts accordingly for every timelapse snap, and at lower than -tl 50 it can't adjust fast enough?


How did you achieve this >~7fps? I've been playing with the code with aim to write an mjpeg streamer input plugn and in a simple capture->encode loop the fastest i seem to get is about 12fps. Resolution makes very little difference here, and it requires around a 100ms delay between captures else the picture is too dark or just black.


For anyone who is interested, here's my work on an mjpeg streamer plugin for raspicam https://github.com/jacksonliam/mjpg-streamer
User avatar
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm
by TonAu » Sun Jun 09, 2013 11:34 am
I downloaded the experimental mjpgstreamer from github and it works fine for a couple of minutes.

Had to apt-get cmake and libjpeg8-dev

Also, the makefile does not find a directory 'build' in the raspicam plugin and then stops with an error.
That directory is not in the repository. I just did 'mkdir build' there and after that cmake went on and the make finished succesfully.

The webserver starts with 1 fps approximately. But after 800 frames or so the picture slowly turns dark and ends up black.
Then I have to restart mjpg-streamer.
Any clue what that can be?
update:
- a delay of 300 seems to be working better: now for hours already, with 5-6% cpu load. (I specified x and y to 640 480, should make no diff)
Posts: 26
Joined: Sat Jul 21, 2012 11:23 am
by jacksonliam » Mon Jun 10, 2013 12:06 pm
Glad your trying it out. I fixed the build folder missing thing. Didn't know you can't have empty github folders.

I think I have libjpeg62-dev. I've not seen the darker images thing and I've run this for ~24hrs on the default 1000 delay. Wouldn't have expected a shorter delay to help... I wonder if this is related:
viewtopic.php?f=43&t=45783
Like raspistill, I keep the camera open the entire time, though it does seem to do all the light level stuff on each shot, I don't think its possible to have it only do it once so you can get faster stills at the same light level - I've had a good look though threads on here and the source and can't find anything that says so.

which firmware are you on? I dunno if a rpi-update would help?
Code: Select all
/opt/vc/bin/vcgencmd version
Jun  6 2013 18:30:04
Copyright (c) 2012 Broadcom
version e6f0abde032163069972925f175c4a3a5617d46d (clean) (release)
User avatar
Posts: 181
Joined: Tue Feb 07, 2012 10:09 pm