PiKrellCam: motion vector detect + OSD web interface

[woodside]

I was interested to read your comments re security when accessing PiKrellCam via the internet and have to say that I have been doing this on and off for some time using port forwarding on the router.
Are you saying there's a danger in doing this and if so could you please provide more information... I certainly don't want my network going boom!!! Thanks

[boomlikethat]

can't seem to be able to get to my PiKrellCam interfaces from outside my local network.

Do you mean from a larger subnet, or that you want to get in from anywhere on the internet? Normally your router's NAT (address translation) does not make your Pi visible to the world, so first you need to figure out your IP address as seen from the outside, and also if your ISP allows traffic to whatever port you're forwarding to the Pi.

Even before that though, please consider your security plan... It's not something I've tried myself, because without considerable thought and preparation I know a whole network can be "owned" just boom... like that.

Yes, I'm trying to access it from anywhere. I want to be able to monitor the cams if I'm away for any length of time.

[boomlikethat]

I was interested to read your comments re security when accessing PiKrellCam via the internet and have to say that I have been doing this on and off for some time using port forwarding on the router.
Are you saying there's a danger in doing this and if so could you please provide more information... I certainly don't want my network going boom!!! Thanks

Can you give me some guidance on this? I use port 80 for 2 Pi's with PiKrellCam, and have tried to set destination ports to, say, 10101 and 10102. So I should be able to access it using <public IP address>:10101 or :10102

I have a friend who's helping me test it, but it always times out on him. So something isn't right.

Thanks!

[jbeale]

Opening any port to the internet means anything that can come through that port, will do so, given the worldwide swarm of botnets etc continually probing any open port for any weakness. So you have to anticipate every possible attack vector on whatever software is listening on that port. I am not confident that I understand the possible attacks on PiKrellCam and its webserver nginx, so I haven't tried doing this.

[billw]

I was interested to read your comments re security when accessing PiKrellCam via the internet and have to say that I have been doing this on and off for some time using port forwarding on the router.
Are you saying there's a danger in doing this and if so could you please provide more information... I certainly don't want my network going boom!!! Thanks

Can you give me some guidance on this? I use port 80 for 2 Pi's with PiKrellCam, and have tried to set destination ports to, say, 10101 and 10102. So I should be able to access it using <public IP address>:10101 or :10102

I have a friend who's helping me test it, but it always times out on him. So something isn't right.

Thanks!

If you use from outside <public IP address>:10101 then the router must be set up to port forward 10101 to the LAN DHCP
assigned address assigned to your Pi by the router. So, if your router has given the Pi say, 192.168.0.104, then the 10101
port has to be port forwarded to 192.168.0.104 in your router configuration. So get your LAN DHCP assigned IP by running
sudo ifconfig -a. Look in the output for something like: inet addr:192.168.0.104 for your interface.
Then log into your router config and set port forwarding of 10101 to 192.168.0.104 or whatever yours is.

I do this for many Pi's but I can't count on my router giving a fixed DHCP IP, so I go through additional steps to set up a
static IP alias on each Pi and I configure the router to port forward to the alias static IP.

But at least you should be able to get it working with the dynamic IPs first. Then worry about that.

And in the pikrellam install script you have to set up using the port 10101.

I've been doing this for ssh (with iptable firewall rules) and for pikrellcam with nginx password set since the beginning and
haven't had any issues yet. But then I don't keep anything valuable on my Pi's and I don't open up ports on my desktop,
so use your own judgment.

[boomlikethat]

I was interested to read your comments re security when accessing PiKrellCam via the internet and have to say that I have been doing this on and off for some time using port forwarding on the router.
Are you saying there's a danger in doing this and if so could you please provide more information... I certainly don't want my network going boom!!! Thanks

Can you give me some guidance on this? I use port 80 for 2 Pi's with PiKrellCam, and have tried to set destination ports to, say, 10101 and 10102. So I should be able to access it using <public IP address>:10101 or :10102

I have a friend who's helping me test it, but it always times out on him. So something isn't right.

Thanks!

If you use from outside <public IP address>:10101 then the router must be set up to port forward 10101 to the LAN DHCP
assigned address assigned to your Pi by the router. So, if your router has given the Pi say, 192.168.0.104, then the 10101
port has to be port forwarded to 192.168.0.104 in your router configuration. So get your LAN DHCP assigned IP by running
sudo ifconfig -a. Look in the output for something like: inet addr:192.168.0.104 for your interface.
Then log into your router config and set port forwarding of 10101 to 192.168.0.104 or whatever yours is.

I do this for many Pi's but I can't count on my router giving a fixed DHCP IP, so I go through additional steps to set up a
static IP alias on each Pi and I configure the router to port forward to the alias static IP.

But at least you should be able to get it working with the dynamic IPs first. Then worry about that.

And in the pikrellam install script you have to set up using the port 10101.

I've been doing this for ssh (with iptable firewall rules) and for pikrellcam with nginx password set since the beginning and
haven't had any issues yet. But then I don't keep anything valuable on my Pi's and I don't open up ports on my desktop,
so use your own judgment.

Thanks for the help billw and jbeale. I think I got it, but need to test it.

@billw: I have set up hostnames in noip, so dynamic IP (hopefully) won't be an issue.

[bic]

Great project. More stable then others I have tried, kudos. I would like to see, once disc is full, it starts deleting oldest files to record newest.

[jbeale]

@bic : Agree about deleting old files at some % disk full. On my systems, a script in ~/pikrellcam/scripts does that, called pk-archive and I run mine at 15 minutes after the hour from crontab

Code: Select all

15 * * * * sudo -u pi /home/pi/pikrellcam/scripts/pk-archive 2>&1 | /usr/bin/logger -t PK-Archive

but you could also use the scheduler in ~/.pikrellcam/at-commands.conf

Code: Select all

#!/bin/bash

_ARCHIVE_OLDER_DAYS="-1 Days"  # how many days old for media before archiving
# _VOLUME="/dev/sda1"     # volume to check for disk space usage
_VOLUME="/dev/root"     # volume to check for disk space usage
_USED_SPACE_LIMIT=85    # percent disk space used over which files are deleted

echo "archive_still day `date --date="$_ARCHIVE_OLDER_DAYS" +'%F'`"
# Move _ARCHIVE_OLDER_DAYS stills and video to that date's archive folder
echo "archive_still day `date --date="$_ARCHIVE_OLDER_DAYS" +'%F'`"  > /home/pi/pikrellcam/www/FIFO
echo "archive_video day `date --date="$_ARCHIVE_OLDER_DAYS" +'%F'`"  > /home/pi/pikrellcam/www/FIFO

# if used space is over threshold, delete old files and directories to save storage space until threshold reached
# this keeps as much as possible files present
_USED_SPACE=`df  | grep $_VOLUME | awk '{print $5}' |  sed -e "s/\%//g"`
# echo "used space: $_USED_SPACE%"


if [[ ( "$_USED_SPACE" > "$_USED_SPACE_LIMIT" ) ]] ; then
 echo -e "used space over $_USED_SPACE_LIMIT%, deleting old files..."

 _LOOP=0;

 while [[ ( "$_USED_SPACE" > "$_USED_SPACE_LIMIT" )  && ( "$_LOOP" < "100" ) ]] ; do
  _OLDEST=`find /home/pi/pikrellcam/media/archive/ -type f -printf '%T+ %p\n' | sort | head -n 1 | cut -d "+" -f 1 | sed -e "s/\-/\//g"`
  echo "oldest files:$_OLDEST"

  echo "delete oldest folder:/home/pi/pikrellcam/media/archive/$_OLDEST"
  rm -rfv "/home/pi/pikrellcam/media/archive/$_OLDEST"
  sync

  _USED_SPACE=`df  | grep $_VOLUME | awk '{print $5}' |  sed -e "s/\%//g"`
  echo "used space now: $_USED_SPACE%"
  _LOOP=$((_LOOP + 1))
 done

 echo "done"
 df
else
 echo -e "used space below $_USED_SPACE_LIMIT%, nothing to do!"
fi

[wirelessguyny]

Freeing up space is always a good idea.

I'll share some of my own trial-by-fire in case it proves helpful to others here.

I use an external powered USB drive for storage so I'm not dependent on the SD card size limitations. Overall ,this has worked well for me and I think my instructions on how I set that up are in fact buries here in the thread somewhere.

More recently though, I brought my system down cleanly (shutdown - h) but it would not start again. It seems that the temp area of the filesystem is full and somehow will not clear itself out, even though it says it does on each system boot. I don't see any real way for me to get access ot the filesystem other than perhaps mounting it via another working Pi as a secondary drive. I'm basically assuming that SD I have is hosed and I'm just to going to start a fresh install but I'd perhaps suggest that others also look in to what pikrellcam might be creating in temp space and clearing that out with some sort of script during the power down cycle as well.

All the best,
~ WG.

[Gavinmc42]

Just had a SDcard fill up on the weekend and pikrellcam started doing strange stuff.
Copied all the media files to a network drive with Filezilla.
I have Midnight Commander installed so I ssh into the Pi to have a look around then rm the media files.
Pikrellcam still did not work properly after reboot, something got corrupted.
I had to run the ./install... again, had to reboot a few times.
All done remotely over the network

Midnight Commander (mc) goes on all my boxes just for situations like this.
Don't really need it, could have fixed it all cli but it is nice to have.

[Gavinmc42]

Need help to fix my pikrellcam archive network drive settings.

Have one Pi with a USB 1TB drive, 10.0.0.55, no problems archiving to the USB drive.
Have another Pi, 10.0.0.54, currently archiving to it's SD card.
How do I set up all the Pi's so the second Pi archives to the first one's USB drive?

Basic stuff that is driving me nuts, google not helping
And this post has 33 pages and I can't seem to find an answer I understand.

PS Adafruit 1m camera extension cables work great.
Should have got a few 2m ones to try.

[jbeale]

There are many ways to do this. What I do is have the Pi save files on the SD card short-term, then at some schedule (hourly, daily...) I use 'rsync' to copy the local Pi files to the remote server (could be another Pi or any other machine). Then I have separate 'cron' jobs on each machine that delete the oldest files to maintain a certain % disk usage or certain number of days. This is standard Unix/Linux file stuff, nothing specific to RPi. For example usage of 'rsync' try http://www.thegeekstuff.com/2010/09/rsy ... d-examples or some other pages google will show.

The only trick here is that you need to set up the Pi to be able to ssh into the remote server without using a password, but that is easy to set up as explained here: http://www.thegeekstuff.com/2008/11/3-s ... h-copy-id/

[boomlikethat]

Hey all,

I just purchased a Pi NoIR v2. Are are any issues I should be aware of when using it with PiKrellCam?

Thanks!

[jbeale]

I've got a NoIR v2.1 running with PiKrellCam. You do want the current version of the S/W so it supports the higher resolutions of the v2 camera, otherwise it just works. Well, with the additional note that for some reason, the v2 camera will not go to either max ISO or max shutter speed in dark conditions all by itself, in the same way the v1 camera did. If you don't realize that, you may conclude v2 is not as good at night as the v1 camera. You can force it in manual mode though and after that, I found the sensitivity is roughly equivalent to v1.

[boomlikethat]

I've got a NoIR v2.1 running with PiKrellCam. You do want the current version of the S/W so it supports the higher resolutions of the v2 camera, otherwise it just works. Well, with the additional note that for some reason, the v2 camera will not go to either max ISO or max shutter speed in dark conditions all by itself, in the same way the v1 camera did. If you don't realize that, you may conclude v2 is not as good at night as the v1 camera. You can force it in manual mode though and after that, I found the sensitivity is roughly equivalent to v1.

Well, I have what I think is the latest version (3.0.7). If not let me know, but I think I know where to bump up the ISO for nighttime. Thanks!

[Fox Mulder]

Til a few days i used the program motion with my Raspberry and it worked all good. But with the limited cpu power i could only use very low framerates for detection and videos. Now i found pikrellcam and am quite happy about the much lower cpu usage, higher framerate and many new functions compared to motion.
Now i have a setup which works quite good for me but there are a few things i stumbled upon when trying to set things up for me. Maybe someone can help me a bit with these things.

Is it possible to override the script "scripts-dist/_thumb" in the "scripts" directory?
I tried to copy _thumb and changed the content but still the version in scripts-dist is used. I want to create thumbs with higher resolutions and therefor have to change this script.

The script "_thumb" is only used for motion detected videos and not manually recorded videos. Is there another script to create the thumbnails for these videos where i can make modifications?

In media archive for videos i can change between thumbs and list view. But when i'm in the archive for still pictures the link to change the view has no effect and i only see the list, no matter what list mode is active. I also see that there are no thumbnails created for still images like for videos in the thumbnail directory. Is there a way to enable the thumbs view for still pictures?

Ciao,
Rainer

[billw]

Is it possible to override the script "scripts-dist/_thumb" in the "scripts" directory?

The thumb names can't be configurable so the script is not a user script. But I can add a thumb width variable to pikrellcam.conf
that would apply to motion and manual video thumbs For now you could edit scripts_dist/_thumbs for motion videos but manual
video thumb widths can't be changed until I add the variable. If you edit the script, it will be replaced at the next upgrade but then
look for a thumb width variable to change.

There are no thumbs for stills, at least not yet.

[Fox Mulder]

Thanks for the clarification. Everything works now how i want so it's no problem at all. But when upgrading i have to be careful with my local modifications.

At the moment i removed the scaling of the thumbnails and just copy them from the mjpeg in the _thumb script because i use the thumbnail for motion preview with osd overlays in an telegram messenger script where i wanted a higher resolution. I also modified media-archive.php to add a width to the thumbnails "img src" tags because now they are a bit too large for the website. This solution works very good for me and i don't have to create a new still picture at the beginning of each new motion video.

Ciao,
Rainer

[Fox Mulder]

Hi,

i'm using an external script1 to control PiKrellCam from my mobile phone. Now i have another script2 which also could control if motion detection should be enabled or disabled. Because script1 and script2 are completely independent from each other i would need to know the current state if motion detection is enabled or disabled. Correspondingly the script does behave one or another way.

So the question is if i could somehow read the state of the motion status like i could change it with the FIFO (motion_enable) command?

This would really simplify the work with external scripts to control PiKrellCam.

Ciao,
Rainer

[billw]

So the question is if i could somehow read the state of the motion status like i could change it with the FIFO (motion_enable) command?

For bash scripts:

Code: Select all

line=`grep motion_enable /run/pikrellcam/state`
motion_state=${line#motion_enable}

The motion_state variable will then be "on" or "off". When motion_enable is FIFO changed, the new
state should show up in the /run/pikrellam/state file within around 100-200 msec.

[Fox Mulder]

For bash scripts:

Code: Select all

line=`grep motion_enable /run/pikrellcam/state`
motion_state=${line#motion_enable}

The motion_state variable will then be "on" or "off". When motion_enable is FIFO changed, the new
state should show up in the /run/pikrellam/state file within around 100-200 msec.

That's exactly what i needed.

[jbeale]

I want to record longer after an event detection at night, than during the day. So, in /home/pi/.pikrellcam/at-commands.conf I have:

Code: Select all

daily   sunset+60  "@motion_event_gap 20"
daily   sunset+60  "@motion_post_capture 19"
daily   sunrise  "@motion_event_gap 6"
daily   sunrise  "@motion_post_capture 5"

However, this does not seem to work, based on the PKC log output below. Is there a way to change these values between night and daytime?

Code: Select all

[...]
sunrise/sunset times: valid  dawn/dusk times: valid
  dawn:    6:02
  sunrise: 6:29
  sunset:  20:06
  dusk:    20:32

  [...]
21:05:00 : Bad command: [motion_event_gap] [20]
21:05:00 : Bad command: [motion_post_capture] [19]

Note: this was on PKC v3.0.5 and I see there is a newer version 3.0.7 so I just updated and I'll see if that makes a difference.

[billw]

An oversight, you'll need version 3.1.0.
And no '_' after motion, so:

Code: Select all

daily   sunset+60  "@motion event_gap 20"
daily   sunset+60  "@motion post_capture 19"
daily   sunrise  "@motion event_gap 6"
daily   sunrise  "@motion post_capture 5"

[billw]

PKC 3.1.0 has some upgrades.

1) A Multicast interface for communication and control between all PKCs and
desktops running on your LAN.

2) A /run/pikrellcam/motion-events file that can be read to get
motion events by x,y position or motion region as they happen during a
video recording.

These are covered in the help page and there are some scripts you can
look at in the scripts-dist directory to find out more.

Some things you can now do:

Run a pkc-motion command from a desktop terminal to turn off/on motion_enable
on any or all PKCs running on your LAN. If you have work to do outside in
areas watched by some cameras, turn motion off for all of them at once
with a single terminal command from any LAN computer: pkc-motion all off
or turn off for specific ones: pkc-motion rpi2,rpi3 off

I have a wifi connection to one of my Pis where using my microwave can
occasionally knock it out. Two out of three times so far I've been able to
reboot the Pi using pkc-reboot even though the connection was down because
multicasting does not depend on an interface being routed. YMMV on this one.

Have PKCs run an on_motion_begin script that sends an alarm to one or many
desktops (they have to be running a listening program - see pkc-alarm
in scripts-dist) so an audio alarm can be played immediately as motion
is detected. The detected motion that sends the alarm can be the overall
detect event or it can be motion detected in a specific motion region.
So you can put a motion region around a critical area (like part of your
driveway) or object of interest and get an alarm when motion is detected
right there. The magnitude and count triggering the audio alarm can be
different from what you have configured for overall motion detection.
So motion detection can be sensitive for videos including small animals,
but the audio alarm sending can be less sensitive to sound only for
larger objects.

[jbeale]

Thanks for this great upgrade, PKC keeps getting better! I look forward to trying out the new alarm function. I fixed my at_commands syntax and it works as intended now.
Next question: can I also change the Pi Camera image parameters like contrast and brightness which are set in the pikrellcam.conf from the at_commands.conf file and if so, what is the syntax? For example I tried

Code: Select all

daily 14:49  "@brightness 50"

and also without the "@" sign, but that didn't work.