User avatar
jbeale
Posts: 3476
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Aug 17, 2019 7:07 am

I have a few Pi's running PiKrellcam v4.2.1. In addition to motion capture, it takes a still photo every other hour. The still images are triggered by this line in my crontab:

Code: Select all

# take one still photo every other hour 7am-5pm
0 7-17/2 * * * echo "still" > /home/pi/pikrellcam/www/FIFO
PiKrellCam also deletes files when disk space is low. However I seem to recall in the past it would delete the oldest files. Right now, it is deleting the very most recent file, as the below lines from the log file show it immediately deleting the still image just taken:

Code: Select all

17:00:01 : command_process: still
17:00:01 : Still: /home/pi/pikrellcam/media/stills/image_2019-08-16_17.00.01_107.jpg
17:00:02 : Low disk space, deleted: /home/pi/pikrellcam/media/stills/image_2019-08-16_17.00.01_107.jpg
despite the fact that there is quite a bit of space taken by the older still and video archives over the past few months:

Code: Select all

[email protected]:~/pikrellcam/media/archive/2019 $ du -d 1
543972	./08
583252	./06
1540832	./07
2668060	.
I'm sure I've just misconfigured something, but I wonder what? My /home/pi/.pikrellcam/pikrellcam.conf file includes:

Code: Select all

install_dir /home/pi/pikrellcam
media_dir media
archive_dir archive
diskfree_percent 40
check_media_diskfree on
check_archive_diskfree on
Does having the "check_*_diskfree" lines in that order in the .conf file, mean that it first checks the current day's media and deletes that, before even checking the much larger archive/* media folders?

EDIT: Aha! I changed one line in the .conf file to "check_media_diskfree off" and manually recorded some video, and now the log shows it made room by deleting the oldest archive file from some months ago, instead of what was just recorded. I think I have my answer.

Code: Select all

00:17:04 : Manual record stop (size:0.1MB < /tmp:2381.4MB)  vid_time:20.25  vid_fps:24.00  audio_frames:0  audio_rate:0
  convert /run/pikrellcam/manual_2019-08-17_00.16.44_0.jpg -resize 150x84\! /home/pi/pikrellcam/media/thumbs/manual_2019-08-17_00.16.44_0.th.jpg
00:17:05 : Low disk space, deleted: /home/pi/pikrellcam/media/archive/2019/06/01/videos/motion_2019-06-01_18.21.43_289.mp4

billw
Posts: 402
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Aug 17, 2019 6:36 pm

jbeale wrote:
Sat Aug 17, 2019 7:07 am
Does having the "check_*_diskfree" lines in that order in the .conf file, mean that it first checks the current day's media and deletes that, before even checking the much larger archive/* media folders?

EDIT: Aha! I changed one line in the .conf file to "check_media_diskfree off" and manually recorded some video, and now the log shows it made room by deleting the oldest archive file from some months ago, instead of what was just recorded. I think I have my answer.

Code: Select all

00:17:04 : Manual record stop (size:0.1MB < /tmp:2381.4MB)  vid_time:20.25  vid_fps:24.00  audio_frames:0  audio_rate:0
  convert /run/pikrellcam/manual_2019-08-17_00.16.44_0.jpg -resize 150x84\! /home/pi/pikrellcam/media/thumbs/manual_2019-08-17_00.16.44_0.th.jpg
00:17:05 : Low disk space, deleted: /home/pi/pikrellcam/media/archive/2019/06/01/videos/motion_2019-06-01_18.21.43_289.mp4
Strange, could you have done a video archive after that manual record? Because an archive diskfree check should be done only after a video archive.

If check_media_diskfree is on and you take a still, only the still directory is looked at for files to delete. So I should review how this all works and check videos and stills together to delete oldest files from either when a still or video is recorded. I'm not sure if the archive dir should be included in such a global check because I've thought of it as higher priority videos and it can likely be on a different file system.

User avatar
jbeale
Posts: 3476
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Aug 17, 2019 7:05 pm

I had not paid much attention to how all this works, but I think at least on my system that the entire day's videos are simply moved to the archive folder around midnight. I was doing those experiments right around midnight, so the process of archiving may have just happened.
At least on my system, I'm mostly not doing any manual selection/deletion or priority assignment, so the archive ends up being all the videos and stills that were not taken today.

billw
Posts: 402
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Sat Aug 17, 2019 7:51 pm

jbeale wrote:
Sat Aug 17, 2019 7:05 pm
I had not paid much attention to how all this works, but I think at least on my system that the entire day's videos are simply moved to the archive folder around midnight. I was doing those experiments right around midnight, so the process of archiving may have just happened.
At least on my system, I'm mostly not doing any manual selection/deletion or priority assignment, so the archive ends up being all the videos and stills that were not taken today.
So for your usage you could just leave check_media_diskfree off as you did and then let your periodic archiving handle keeping the diskfree maintained. It means that as you record stills and videos during the day your diskfree will drop below your 40% setting and you will have to rely on it not getting dangerously low before the next archiving brings it back to 40%.

And as I was thinking, I'll go ahead and review how this works so that check_media_diskfree can be enabled again and you can have things work reasonably.

kajan001
Posts: 16
Joined: Wed Feb 13, 2019 7:15 am

Re: PiKrellCam: motion vector detect + OSD web interface

Wed Aug 28, 2019 7:31 am

Pi Zero W freezing again.

I am back with frozen pi again. As per earlier posts I have now ruled out power issues, jacked up voltage to core, and have ruled out wifi issues.
I was using Stretch and had the occasional freeze and while I tried to find the cause I have also tried upgrading to Buster.

However, I changed the memory split to 128Mb and that seem to improve things.

I will keep on testing
Last edited by kajan001 on Thu Sep 12, 2019 9:43 am, edited 1 time in total.

kajan001
Posts: 16
Joined: Wed Feb 13, 2019 7:15 am

Re: PiKrellCam: motion vector detect + OSD web interface

Wed Aug 28, 2019 8:10 am

I am currently not getting any conversion from h264 to mp4 on Buster.

I seems a problem with memory allocation:

Code: Select all

07:32:07 :   execl[wait:1]: /home/pi/pikrellcam/scripts-dist/_log-lines 500 /tmp/pikrellcam.log
07:53:24 : Motion record start - /home/pi/pikrellcam/media/videos/motion_2019-09-04_07.53.19_11.h264
07:53:24 :   preview save: copy /run/pikrellcam/mjpeg.jpg -> /run/pikrellcam/motion_2019-09-04_07.53.19_11.jpg
07:53:24 : fork() failed: Cannot allocate memory
07:56:16 : command_process: record off
 external:0
EDIT2: I changed GPU/CPU split from 256 to 128mB. Now the Pi0w converts the .h264 to mp4.
I am thinking that this issue maybe related to the common freezes I have been experiencing on the pi0's. I guess taking 256mb of the 512 the pi0 has may cause issues if the process running is CPU intensive rather than GPU intensive (which I thought it was due to video processing).



@billw - Is pikrellcam using mostly GPU or CPU for the motiondetection and video conversion? Perhaps I am setting the memory split to high (256mb on Pi0)
I remember a block diagram of the program that I saw but I cannot find it anymore, though I think a lot of processing went on in the GPU?

I am also experiencing freezes when at startup if the pi has been off for a while. I am wondering if this can happen when the time is set while pikrellcam has already starte recording, or if it is another cpu overload or memory issue. THe cpu works hard during startup.
When in the process does pikrellcam start recording?

dburg
Posts: 4
Joined: Tue Jul 03, 2018 2:06 am

Re: PiKrellCam: motion vector detect + OSD web interface

Sun Sep 15, 2019 5:15 am

@kajan001, I've also had problems with my PiZero W (running Stretch Lite) freezing. I could not access it through the HTTP port, or through ssh, but it did respond to pings. I eventually discovered that I could connect with "ssh -T", and I found a bunch of processes that were started by pikrellcam were not terminating. Perhaps this technique will be useful to you, too. For example, try "ssh -T [email protected]_cam ps -ef". BTW... I got sidetracked and stopped digging into the root cause.

dburg
Posts: 4
Joined: Tue Jul 03, 2018 2:06 am

Re: PiKrellCam: motion vector detect + OSD web interface

Sun Sep 15, 2019 5:27 am

@billw, Just for entertainment, I'm fooling around with the html.

Just an FYI, I noticed that "motion_enable" appears in pikrellcam.c as one of the "commands[]", as well as in config.c as one of the "config[]" entries. It looks like command_process() is unable to get to the "config[]" entry because it finds the "commands[]" entry first.

I am also curious about the "(cfg->safe && !set_safe)" code in config_set_option(). Are you thinking that it is just too unsafe for some commands to be issued through the FIFO?

Thanks!

billw
Posts: 402
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Mon Sep 16, 2019 12:51 am

kajan001 wrote:
Wed Aug 28, 2019 8:10 am
@billw - Is pikrellcam using mostly GPU or CPU for the motiondetection and video conversion? Perhaps I am setting the memory split to high (256mb on Pi0)
I remember a block diagram of the program that I saw but I cannot find it anymore, though I think a lot of processing went on in the GPU?

I am also experiencing freezes when at startup if the pi has been off for a while. I am wondering if this can happen when the time is set while pikrellcam has already starte recording, or if it is another cpu overload or memory issue. THe cpu works hard during startup.
When in the process does pikrellcam start recording?
When recording video to h264 it mostly uses GPU. When converting h264 to mp4 CPU usage gets very high for slower Pis. You can check CPU usage with a CPU monitor but I don't know how to check GPU usage other than noticing that frames get dropped when the video fps gets too high or the preview jpg update frequency gets too high. When started in rc.local, pikrellcam starts late in the booting and there are delays on top of that - 5 seconds in rc.local before starting pikrellcam and 3 more seconds after pikrellcam starts to allow for camera startup settling.

billw
Posts: 402
Joined: Tue Sep 18, 2012 8:23 pm

Re: PiKrellCam: motion vector detect + OSD web interface

Mon Sep 16, 2019 1:00 am

dburg wrote:
Sun Sep 15, 2019 5:27 am
@billw, Just for entertainment, I'm fooling around with the html.

Just an FYI, I noticed that "motion_enable" appears in pikrellcam.c as one of the "commands[]", as well as in config.c as one of the "config[]" entries. It looks like command_process() is unable to get to the "config[]" entry because it finds the "commands[]" entry first.

I am also curious about the "(cfg->safe && !set_safe)" code in config_set_option(). Are you thinking that it is just too unsafe for some commands to be issued through the FIFO?

Thanks!
I'll look at that motion_enable issue. The commands marked as "safe" are the ones that can configure arbitrary programs to be run with sudo privilege and I considered that not appropriate to configure via the FIFO which is accessible via web pages.

kajan001
Posts: 16
Joined: Wed Feb 13, 2019 7:15 am

Re: PiKrellCam: motion vector detect + OSD web interface

Mon Sep 16, 2019 9:46 am

billw wrote:
Mon Sep 16, 2019 12:51 am
kajan001 wrote:
Wed Aug 28, 2019 8:10 am
@billw - Is pikrellcam using mostly GPU or CPU for the motiondetection and video conversion? Perhaps I am setting the memory split to high (256mb on Pi0)
I remember a block diagram of the program that I saw but I cannot find it anymore, though I think a lot of processing went on in the GPU?

I am also experiencing freezes when at startup if the pi has been off for a while. I am wondering if this can happen when the time is set while pikrellcam has already starte recording, or if it is another cpu overload or memory issue. THe cpu works hard during startup.
When in the process does pikrellcam start recording?
When recording video to h264 it mostly uses GPU. When converting h264 to mp4 CPU usage gets very high for slower Pis. You can check CPU usage with a CPU monitor but I don't know how to check GPU usage other than noticing that frames get dropped when the video fps gets too high or the preview jpg update frequency gets too high. When started in rc.local, pikrellcam starts late in the booting and there are delays on top of that - 5 seconds in rc.local before starting pikrellcam and 3 more seconds after pikrellcam starts to allow for camera startup settling.
Thanks Bill!

I think this confirms that memory split has been one of the issues.
I will continue exploring this with a new set of settings.

Return to “Camera board”