User avatar
solar3000
Posts: 1051
Joined: Sat May 18, 2013 12:14 am

Re: RPi Cam Web Interface

Thu Apr 30, 2015 6:02 am

Why are all the videos and images saved as root:root??
Antikythera

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 10:33 am

solar3000 wrote:Why are all the videos and images saved as root:root??
The data is being captured by the raspimjpeg process which is running as root so that is setting the ownership of the files. The web site processes own the media folder and has permissions to operate on (delete) those files.

kevw
Posts: 22
Joined: Wed Aug 27, 2014 1:49 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 11:35 am

Just wanted to say thank you for the time and effort being put into this project, it's really appreciated.

Kev

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Apr 30, 2015 1:59 pm

Can you help me understand how the purge-by-age function works? I have the setup shown below with PurgeVideo_Hours and PurgeImage_Hours set at 48, however I still have video up to 4 days old showing on the preview page. Does PurgeSpage_ModeEX take priority and nothing gets deleted until I use up 1 GB of space? Is there a way I can purge files older than X hours, AND also purge anything beyond 1 GB of space? Thanks!

Image

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Apr 30, 2015 2:07 pm

ALso, I just ssh'd into my box and found four (4) instances of 'motion' running, each taking 20% CPU. I think that's not good! I think this may have been triggered by changing the UTC offset forward and back one hour which changed the time-of-day from DAWN to DAY and back, which triggered the time-of-day motion scheduling commands aybe starting more instances of motion without killing the old ones first (?)

User avatar
solar3000
Posts: 1051
Joined: Sat May 18, 2013 12:14 am

Re: RPi Cam Web Interface

Thu Apr 30, 2015 2:17 pm

btidey wrote:
solar3000 wrote:Why are all the videos and images saved as root:root??
The data is being captured by the raspimjpeg process which is running as root so that is setting the ownership of the files. The web site processes own the media folder and has permissions to operate on (delete) those files.
why doesn't it start as apache:apache or nobody:nobody?
Antikythera

ethanol100
Posts: 585
Joined: Wed Oct 02, 2013 12:28 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 2:25 pm

From looking at the fuction purgeFiles() in "schedule.php" first the time is checked and older files are deleted and then the space is checked.

I tried to run this on "jessie" and the login shell of www-data is changed from /bin/sh to /usr/sbin/nologin. Now the "su www-data -c 'php /var/www/schedule.php > /dev/null &'" can not be executed in "/etc/rc.local" and the "schedule process" is not running. Not sure how to run php with that shell. Changing it back to /bin/sh would fix this, but I'm not sure about security...
Is your process scheldule.php running?
solar3000 wrote:why doesn't it start as apache:apache or nobody:nobody?
The user running raspimjpeg needs to be in the video group.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 4:01 pm

jbeale wrote:ALso, I just ssh'd into my box and found four (4) instances of 'motion' running, each taking 20% CPU. I think that's not good! I think this may have been triggered by changing the UTC offset forward and back one hour which changed the time-of-day from DAWN to DAY and back, which triggered the time-of-day motion scheduling commands aybe starting more instances of motion without killing the old ones first (?)
That is definitely not good. If that is happening regularly then it could be an extra cause for crashes. Apart from the extra resource being used it will lead to lots of very confusing stop start triggers.

Manually starting motion detect from the web interface should start one instance of motion as would sending md 1 from the scheduler. Sending an md 0 either by pushing motion stop or by sending md 0 from scheduler should kill ALL instances.

Looking at your scheduler set up there are md 1 at start of dawn and day so I think that could explain 2 getting going and I guess if you had manually started one then you'd be up to 3. Then add on timezone tweaking ... I use md 0 md 1 as a pair which ensures that only 1 is ever going but really we should be preventing it occuring in the low level md 1 handling. I'll look a that.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 4:17 pm

ethanol100 wrote:From looking at the fuction purgeFiles() in "schedule.php" first the time is checked and older files are deleted and then the space is checked.
That is certainly how it supposed to work and seems OK on mine it is regularly purging files based on the initial time criteria.

Somebody else reported that is wasn't working for them so I produced a special version of scheduler that just had a bit of extra logging in to show the time comparisons for each file during the purge check. It' in a fairly recent post 2 or 3 days ago.

When they tried it then purge started working even though it was just the same code. So I said to take the logging out but haven't heard where that ended up.

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Apr 30, 2015 4:19 pm

btidey wrote: That is definitely not good. If that is happening regularly then it could be an extra cause for crashes. Apart from the extra resource being used it will lead to lots of very confusing stop start triggers.

Manually starting motion detect from the web interface should start one instance of motion as would sending md 1 from the scheduler. Sending an md 0 either by pushing motion stop or by sending md 0 from scheduler should kill ALL instances.

Looking at your scheduler set up there are md 1 at start of dawn and day so I think that could explain 2 getting going and I guess if you had manually started one then you'd be up to 3. Then add on timezone tweaking ... I use md 0 md 1 as a pair which ensures that only 1 is ever going but really we should be preventing it occuring in the low level md 1 handling. I'll look a that.
Good to know. I thought the command line "md 1" without a prefixed "md 0" for both "dawn" and "day" periods was actually the default config when I first selected the "Sun-based" mode, but maybe I deleted the "md 0;" part without remembering I did it.

Also, the file purge started working as soon as I exceeded the 1 GB limit in /var/www/media but the requested 48-hour limit was and is not causing any deletions for me.

User avatar
solar3000
Posts: 1051
Joined: Sat May 18, 2013 12:14 am

Re: RPi Cam Web Interface

Thu Apr 30, 2015 5:47 pm

I was just concerned that this program runs as root and that it creates files as root.
Would there be any security concerns?
I was thinking of opening a port to this web interface program.
Antikythera

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Apr 30, 2015 5:54 pm

solar3000 wrote:I was just concerned that this program runs as root and that it creates files as root.
Would there be any security concerns?
I was thinking of opening a port to this web interface program.
It is a great system for local LAN. I haven't studied it in detail but with raspimjpeg running as root, communicating with a web interface, and the whole system under active development, it is not something I'd want visible through a firewall, just speaking for myself.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 6:22 pm

jbeale wrote: Good to know. I thought the command line "md 1" without a prefixed "md 0" for both "dawn" and "day" periods was actually the default config when I first selected the "Sun-based" mode, but maybe I deleted the "md 0;" part without remembering I did it.

Also, the file purge started working as soon as I exceeded the 1 GB limit in /var/www/media but the requested 48-hour limit was and is not causing any deletions for me.
Latest raspimjpeg now checks if motion is running and ignores an md 1 if it had previously started it. You should see this in the log.

I don't see the problem so to find out what is happening with the purge of older files on your system then I think we have to use the extra logging version of scheduler. I attach the latest version with logging. This needs to be copied to /var/www/schedule.php and then the scheduler stopped and started from the web interface.

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Thu Apr 30, 2015 6:33 pm

@btidey: I don't see an attachment on your post today, but is this the same debug-scheduler file as attached to this other post on 27 April?
viewtopic.php?f=43&t=63276&start=2000#p747281

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 6:39 pm

solar3000 wrote:I was just concerned that this program runs as root and that it creates files as root.
Would there be any security concerns?
I was thinking of opening a port to this web interface program.
You obviously need to add usr/passwords on the web site as explained in the wiki. That then gives blanket basic protection / access to the site. Yes, it would be nice to have different authorisations for general users and supervisors, but that requires quite a lot of changes to be meaningful.

There are some big concerns if somebody manages to get authorised web access which they shouldn't have. For example the motion command interface which can be edited to issue arbitrary trigger commands and the scheduler management command which could do likewise. The motion one will go away when we replace motion with inbuilt vector based detection. I am planning to replace the arbitrary management one with macros so at least one can control what can and can't be executed. Also with web access one can issue raspimjpeg pipe commands but that doesn't directly do much more than can be done from the web interface.

I don't think there is anything major stopping the current raspimjpeg being run under different authorisation but would require changes to at least the installation, joining the video group, boot up processes, etc. If any body wants to play with that and submit any changes that would be most welcome.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 6:42 pm

jbeale wrote:@btidey: I don't see an attachment on your post today, but is this the same debug-scheduler file as attached to this other post on 27 April?
viewtopic.php?f=43&t=63276&start=2000#p747281
I don't know why it does that. It was there in the preview. Trying again.

The older one is OK for testing. This one is slightly later as it has stuff in for indexes > 9999.
scheduleLogging.zip
(7.58 KiB) Downloaded 39 times

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Thu Apr 30, 2015 10:16 pm

btidey wrote: I am planning to replace the arbitrary management one with macros so at least one can control what can and can't be executed.
I have now made this change.

IMPORTANT
If you have used the scheduler management command facility to regularly execute some system commands this will no longer work until you change them into macros.

The change puts a macros folder under the web folder /var/www/macros. This now holds the limited set of commands that you wish to be able to execute, typically in the form of a shell script which can contain system commands.

The entry in the scheduler management command is now the name of the macro file. An example macro test.sh is there and would then be invoked by putting test.sh in the management command entry.

You have to put the macros there yourself and give them execution rights (chmod). This is deliberate so that a web user does not have any control over what macros are available. The example test.sh there also does not have execution rights and so will not run until given them by chmod.

The macros are run by the raspimjpeg process using a new command (sy macro) This means then can be put in motion trigger and period changes as well. This will allow things like email macros to be used even when motion is replaced and the motion trigger commands are not available.

User avatar
jbeale
Posts: 3494
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: RPi Cam Web Interface

Fri May 01, 2015 3:21 am

I confirm that the special "purge logging" code works to delete the image and video files more than 48 hours old, as I had intended with my config settings. I'm going to leave that as my normal schedule.php file, with the individual file date check save -to-log line commented out.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 8:03 am

jbeale wrote:I confirm that the special "purge logging" code works to delete the image and video files more than 48 hours old, as I had intended with my config settings. I'm going to leave that as my normal schedule.php file, with the individual file date check save -to-log line commented out.
Thanks for testing that.

It doesn't make sense to me from a pure coding perspective because with the log lines commented out it is the same as the original.

One theory I have is whether there could be a difference in the operation between the scheduler started up automatically after boot, and one which is stopped and restarted from the web interface. Mine tends to run after a web page restart as I am fiddling around and time based purging is working Ok with that. In testing the logging version you would probably have done that also. It would be interesting to see if correct operation survives a reboot.

I'm not sure at the moment how that could affect time based purging and not size based.

nthnm
Posts: 15
Joined: Wed Apr 01, 2015 9:35 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 8:36 am

I've been trying to get both 1920x1080 video and the maximum resolution timelapse at the same time. When I add "md 1; tl 1" to my motion start command in the scheduler, I get the "Restart camera or whole Pi" error shortly after motion is detected. Is anyone able to make this work, or should I lower my expectations a little!? I've tried turning off boxing to help reduce load on the processor and I/O on the SD, but it still fails.

KarolGT
Posts: 12
Joined: Tue Jan 27, 2015 3:31 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 9:52 am

Hi

I have very simple script which is:

1) checking if raspimjpeg is running, if yes:

2) echo "ca 1" > /var/www/FIFO; sleep 120; echo "ca 0" > /var/www/FIFO; sleep 0;

and with this I have 2 minutes files recorded in the directory,
it worked fine, till I upgraded yesterday to the latest version.

I see now I have more than 1 instance of raspimjpeg.

How can I now enable full-time video recording?
I just want to have it worked as previously - 2 minutes video

thank you

nthnm
Posts: 15
Joined: Wed Apr 01, 2015 9:35 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 10:49 am

KarolGT wrote:Hi

I have very simple script which is:

1) checking if raspimjpeg is running, if yes:

2) echo "ca 1" > /var/www/FIFO; sleep 120; echo "ca 0" > /var/www/FIFO; sleep 0;

and with this I have 2 minutes files recorded in the directory,
it worked fine, till I upgraded yesterday to the latest version.

I see now I have more than 1 instance of raspimjpeg.

How can I now enable full-time video recording?
I just want to have it worked as previously - 2 minutes video

thank you
Can you just clarify what result you're getting? More than two minutes? Less? Multiple videos?

Also how is your script being triggered? Are you triggering it manually, or is via cron, or something else?

KarolGT
Posts: 12
Joined: Tue Jan 27, 2015 3:31 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 11:16 am

probably I found the solution.

echo "ca 0" > /var/www/FIFO; sleep 0;

doesn't work from root (as my script was started by root from rc.local)

working from pi user

I will let you know.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 11:27 am

nthnm wrote:I've been trying to get both 1920x1080 video and the maximum resolution timelapse at the same time. When I add "md 1; tl 1" to my motion start command in the scheduler, I get the "Restart camera or whole Pi" error shortly after motion is detected. Is anyone able to make this work, or should I lower my expectations a little!? I've tried turning off boxing to help reduce load on the processor and I/O on the SD, but it still fails.
That is definitely not going to work. Time lapse and video capture are mutually exclusive at the lowest camera connection level in raspimjpeg.

The web interface stops you doing this, but as you have discovered it is possible to send commands to do this and the process then curls up and dies. We can put some extra checks in to reject tl commands in active video mode and vice versa to prevent the crash.

Whether it is even possible to do this at all I am not sure about, but the priority at the moment is to get built in motion detection using motion vectors to replace 'motion'.

btidey
Posts: 1622
Joined: Sun Feb 17, 2013 6:51 pm

Re: RPi Cam Web Interface

Fri May 01, 2015 11:33 am

KarolGT wrote:Hi

...
and with this I have 2 minutes files recorded in the directory,
it worked fine, till I upgraded yesterday to the latest version.

I see now I have more than 1 instance of raspimjpeg.
It is correct that you see 2 instances of raspimjpeg running now with latest version. That is because it forks itself into a self-monitoring crash detect and restart mode and the actual process running the camera and commands.

It shouldn't have materially affected the command processing.

Return to “Camera board”