I created a python script to monitor my IP cameras from the raspberry pi 2 model B.
The program is called "Watch Notify Upload" (short wnu), and does the following:
Wnu lets you monitor a folder on the RPI into which a security IP camera (ipcam) uploads photos upon motion detection. Since most ipcams keep posting photos with given upload intervals, e.g. every one second, for about a minute after motion was detected, you typically end up with lots of images that are virtually identical and show nothing interesting. Wnu uses the compare function of ImageMagic to weed out photos that are too similar. It also sends a PUSH notification to an Android phone and it copies all of the remaining "interesting" images into a .tar file which is then uploaded to your Dropbox. Adjustments for the push notifications, dropbox, folders, etc. need to be made in this script for your own installation. Once screened, your photos will be moved into the subfolder "screened" within each ipcam folder. Wnu monitors an ipcam folder every 7 seconds, if new JPG arrives in this folder, it keeps monitoring and waits for 31 seconds of quiet time in which no further photos are received, before starting the photo analysis and upload process. The monitoring times can be adjusted in the script below. The PUSH notification is sent after the initial detection of new photos, but it may take 1.5 minutes or longer, before the screened photos are uploaded onto the dropbox. This is done to keep CPU usage to a minimum.
Requirements:
The software needs to be installed for wnu to work. I am giving the version numbers of software that currently works with this Python3 script.
On the RPI we install:
- ImageMagick 6.7.7-10 2014-04-09
Code: Select all
sudo apt-get update
sudo apt-get install imagemagick- Dropbox-Uploader by Andrea Fabrizi, DropShell v0.2 see: https://www.andreafabrizi.it/?dropbox_uploader
On our android phone we need:
- RemoteAlert app by Soynerdito to receive push notifications.
See http://soynerdito.blogspot.com/2014/05/ ... -from.html. The long registration code that follows "http://remote-alert.herokuapp.com/post/" (here: a32b831c-7623-4c43-99cf-b614ff54e902) in the method send_curl_notification(message, t) needs to be replaced with the app registration code for each new phone installation.
Security cameras used:
This should work for most configurable IP cams. I used two different Apexis ipcams (AMP-J011, AMP-J601) and one Wanscam (JW0004) ipcam. The ipcams uploads JPG images via FTP onto a dedicated folder on the RPI (e.g. /home/pi/ipcam) and then into a dedicated subfolder therein (e.g. /home/pi/ipcam/door or /home/pi/ipcam/patio). All of this happens within the LAN. Hence, no firewalls are breached for outside communication. No incoming communication initiated from outside the LAN is being used here.
Usage of wnu.py:
On the linux command line navigate to the folder in which you copied wny.py. For example, type "cd /home/pi/my pythons". And then type: "python3 wnu.py -w /home/pi/ipcam/Camera_path". Each instance of wnu will provide surveillance for one ipcam, indicated by the folder path into which it uploads its JPG images via ftp. To monitor multiple cameras, simply run a new instance of wnu.py in another linux shell for each ipcam. No worries, CPU remains very low and is typically around 1 - 3 % or less for 3 cameras tested simultaneously.
Enjoy!
The py script follows:
save it as wnu.py
Code: Select all
# Watch-Notify-Upload (wnu.py)
# Version 1.03
# Written in Python 3.2.3 for linux2 (Debian) on Raspberry Pi (RPI)
# © 2015 by Markus K.
#
import os
import argparse
import time
import subprocess
import shlex
import tarfile
from time import localtime, strftime
tf = "%Y-%m-%d %H:%M:%S"
def get_ipcam_dir(): # Read file path for ipcam photos from command line
parser = argparse.ArgumentParser(description='Camera Surveillance by Papa')
parser.add_argument('-w','--watch_path', help='Enter path for ipcam JPG folder.',required=True)
args = parser.parse_args()
jpgdir = args.watch_path
if os.path.isdir(jpgdir) == False:
print("Error: %s" % args.watch_path, "is not a valid directory path.")
exit()
else:
screenfolder = os.path.normpath(jpgdir + '/screened/')
if os.path.isdir(screenfolder) == False:
os.mkdir(screenfolder)
return jpgdir, screenfolder
# I used the CURL notfication from linux, because I could not figure out how to send push notifications with the urllib
# functions of python3. I constantly received unexpexted data type errors and gave up. I plan to replace this with
# google cloud notifications or similar in the future.
def send_curl_notification(message, t): # t is in seconds.
if time.time() - t > 600: # only send notification if last message was sent more than 10 min. ago, so it does not bother too much
h = 'Content-Type: application/json'
d = "'{" + '"message":"%s"' % message + "}'"
cmd = ['curl -X POST -H "%s" http://remote-alert.herokuapp.com/post/a32b831c-7623-4c43-99cf-b614ff54e902 -d %s' % (h,d)]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
output, error = p.communicate()
print(strftime(tf, localtime()), 'Sending notification to phone.')
# print(str(output, encoding='utf-8')) # remove comment if you want to see
# print(str(error, encoding='utf-8')) # the CURL dialogue
t = time.time()
return t
def rename_jpgs(orig_names): # get rid of problematic characters in filename, typically brackets, adjust as needed.
new_names = []
invalidchars = ["(",")"]
for n in orig_names:
if any(i in n for i in invalidchars):
a = n.replace('(','')
b = a.replace(')','')
os.rename(n, b)
new_names.append(b)
else:
new_names.append(n)
return new_names
def new_files_received(t1, t2, raw, ti): # t1 is number of seconds to observe ipcam folder for initial changes
# t2 seconds for furhter changes (additions of ipcam JPGS)
# raw folder path of the unprocessed JPG images
# ti is the last time in seconds a PUSH notification was sent out
#
cam = ' ' + os.path.basename(os.path.normpath(raw)) # Sets a name for your ipcam from the folder name.
print(strftime(tf, localtime()), 'Surveillance active.')
starting_file_number = 0
while starting_file_number <= 1: # at least 2 photos need to be in cam folder for jpg-to-jpg comparisons
time.sleep(t1)
starting_file_list = [fn for fn in os.listdir() if any([fn.endswith('.jpg')])]
starting_file_number = len(starting_file_list)
print(strftime(tf, localtime()), 'Alarm: New photo(s) received!')
t = send_curl_notification(strftime(tf, localtime()) + cam + ' alarm!', ti)
keep_coming = True
while keep_coming == True:
time.sleep(t2)
new_files = [fn for fn in os.listdir() if any([fn.endswith('.jpg')])]
if len(set(new_files).difference(starting_file_list)) > 0:
print(strftime(tf, localtime()), 'New photos keep coming.')
starting_file_list = new_files
keep_coming = True
else:
keep_coming = False
new_file_list = rename_jpgs(new_files)
return sorted(new_file_list), t
def list_jpg_pairs(files):
jpg_pairs = []
number_of_jpgs = len(files)
for i, elem in enumerate(files):
thiselem = elem
nextelem = files[(i+1)%number_of_jpgs]
if i < (number_of_jpgs - 1):
jpg_pairs.append(thiselem + ' ' + nextelem)
return jpg_pairs
def measure_jpg_diffs(pair):
cmd = ['compare -metric MAE ' + pair + ' null:']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
output, error = p.communicate() # needs to read error, because ImageMagick writes results there
diffs = str(error, encoding='utf-8')
d = shlex.split(diffs)
a = float(d[0])
return a
def delete_unwanted_move_wanted_jpgs(pairs, difflist, screenedjpgdir):
print(strftime(tf, localtime()), 'Cleaning up unwanted JPGs.')
firstjpgs = shlex.split(pairs[0])
newfile = os.path.normpath(screenedjpgdir + '/' + firstjpgs[0])
os.rename(firstjpgs[0], newfile)
for d, p in zip(difflist, pairs):
s = shlex.split(p)
if d < 1000: # This sets the threshold for identifying near identical images. Determined empirically.
os.remove(s[1])
else:
newfile = os.path.normpath(screenedjpgdir + '/' +s[1])
os.rename(s[1], newfile)
def surveillance(rawjpgdir, screenedjpgdir, t):
os.chdir(rawjpgdir)
difflist = []
files, ti = new_files_received(7, 31, rawjpgdir, t) # see above for timer settings
print(strftime(tf, localtime()), len(files), 'new photos recieved.')
pairs = list_jpg_pairs(files)
print(strftime(tf, localtime()), 'Measuring photo-to-photo variation.')
for pr in pairs:
dif = measure_jpg_diffs(pr)
difflist.append(dif)
delete_unwanted_move_wanted_jpgs(pairs, difflist, screenedjpgdir)
return ti
def tar_and_upload(raw, screened): # Adjust the path in cmd for your installation of Dropbox-Uploader as needed (see below)
os.chdir(screened)
filenames = [fn for fn in os.listdir() if any([fn.endswith('.jpg')])]
cam = os.path.basename(os.path.normpath(raw))
archivename = cam + '_photos.tar'
archivepath = os.path.normpath(screened + '/' + archivename)
tar = tarfile.open(archivename, "w")
for name in filenames:
tar.add(name)
tar.close()
cmd = ["/home/pi/Dropbox-Uploader/dropbox_uploader.sh upload " + archivepath + " /ipcam/" + archivename]
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
output, error = p.communicate() # see if it got uploaded
print(strftime(tf, localtime()), str(output, encoding='utf-8'), str(error, encoding='utf-8'))
os.remove(archivename)
# Main Program Starts here
if __name__ == "__main__":
try:
rawjpgdir, screenedjpgdir = get_ipcam_dir()
t = 0
while True:
t = surveillance(rawjpgdir, screenedjpgdir, t)
tar_and_upload(rawjpgdir, screenedjpgdir)
finally:
pass
# program has ended