rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

reading domestic electricity use

Thu Nov 01, 2012 11:20 am

Hi

I bought an efergy electricity meter http://www.efergy.com/ which has a clip on induction coil that goes on the main live wire at the fusebox. The clip on has a mini jack connection to a box which sends the reading via rf to an inhome display.

I want to get the electricity use from the induction coil into my Rpi. What's the best way to do it?

1 - is there a min-jack -> usb connector so I can 'read' the real time voltage of the clip on (which I assume is in proportion to the current in the mains wire)?

2 - can I read via GPIP pins? ie connect the voltage at the mini-jack to an A-D converter, the pins of which are read by the GPIO?

3 - any other?

bearing in mind I don't have an oscilloscope

Thanks!

User avatar
scruss
Posts: 3411
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: reading domestic electricity use

Thu Nov 01, 2012 11:51 am

The efergy uses a fairly standard current clamp or CT. It needs an A/D converter, which the Raspberry Pi lacks. There are many Arduino projects to read CT data, one of which being OpenEnergyMonitor.

CTs aren't that accurate, and the limited resolution of microcontroller A/D converters doesn't help.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

User avatar
penguintutor
Posts: 390
Joined: Tue May 08, 2012 9:11 am
Location: UK
Contact: Website

Re: reading domestic electricity use

Thu Nov 01, 2012 3:10 pm

Does the Display have a serial or usb connector on it - if so you may be able to connect that to your Pi instead?

The one I have came free from my energy supplier when I switched to a online statement tariff.

Whilst branded by the energy company is made by Current Cost.
They provide details of the protocol to communicate via serial over USB.
http://www.currentcost.com/cc128/xml.htm

I've created a Python script that can read the current status which I've run on the Pi to test connectivity works. I got it working directly, but my first attempt at converting it to a background daemon didn't work. I've not got the time to work through this any further at the moment as I have some other projects that are more useful. I can provide some basic python code if anyone is interested.

If you're display doesn't have any way of connecting it to a computer then some other people have hobby projects looking at the hardware for the receiver - if you feel up to the challenge.

http://alastair.d-silva.org/hacking-efe ... ower-meter
Read some of the comments and links from that page.

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Thu Nov 01, 2012 8:15 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
Thank you so much for providing this! I have no electronics skills, so was hoping to use Google and here to work something out to read the blinks from our meter, as I never found the coil + monitor to be very reliable!
I'm going to order the parts I need and hope it goes well!

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Fri Nov 02, 2012 11:57 am

scruss wrote:The efergy uses a fairly standard current clamp or CT. It needs an A/D converter, which the Raspberry Pi lacks. There are many Arduino projects to read CT data, one of which being OpenEnergyMonitor.

CTs aren't that accurate, and the limited resolution of microcontroller A/D converters doesn't help.
I think I want to try it this way - can you recommend an AD converter? I haven't done this since some temperature stuff 15 years ago! I'll look at the arduino site

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Fri Nov 02, 2012 12:25 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
really nice, thanks! have ordered my photoresistor off amazon and will try out next week!

User avatar
scruss
Posts: 3411
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: reading domestic electricity use

Sun Nov 04, 2012 1:09 pm

rooster wrote:I think I want to try it this way - can you recommend an AD converter?
The Arduino has several 10-bit ADC channels, so it'll be good enough to track energy usage trends. It won't be accurate enough to check your electricity bills to the penny, though.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
Pronouns: he/him

Hoagie
Posts: 119
Joined: Sun Sep 23, 2012 7:19 pm

Re: reading domestic electricity use

Fri Nov 09, 2012 12:29 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power

I just took a look at the Circuid diagram and it appears to me that you have something like this...
+3.3V
|
10k Resistor
|
Photoresistor
|
+ Raspberry Pi input.
|
Capacitor
|
Gnd

Which seems a bit odd in a two ways.
I'd expect the Raspberry Pi input. to be between the resistor and photoresistor,
I'd expect the capacitor to be across the +3.3 and Gnd to filter it.


I have a similar (but not Raspberry Pi driven) system working with a PC serial port and use this layout

DRT pin (for + supply)
|
100k Resistor
|
+ Input to DCD pin
|
Photoresistor
|
RTS pin (for - supply)

I use C for the software and feed the data to RRDTool to produce graphs. like this http://www.hoagieshouse.com/Electric_Meter/today.png

techpaul
Posts: 1512
Joined: Sat Jul 14, 2012 6:40 pm
Location: Reading, UK
Contact: Website

Re: reading domestic electricity use

Fri Nov 09, 2012 5:35 pm

Hoagie wrote:
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power

I just took a look at the Circuid diagram and it appears to me that you have something like this...
+3.3V
|
10k Resistor
|
Photoresistor
|
+ Raspberry Pi input.
|
Capacitor
|
Gnd

Which seems a bit odd in a two ways.
I'd expect the Raspberry Pi input. to be between the resistor and photoresistor,
That way you may not ever get the voltage going low enough so would always appear as on.
I'd expect the capacitor to be across the +3.3 and Gnd to filter it.
That would filter the 3V3 NOT the signal which could still pick up other light sources or noise.
I have a similar (but not Raspberry Pi driven) system working with a PC serial port and use this layout

DRT pin (for + supply)
|
100k Resistor
|
+ Input to DCD pin
|
Photoresistor
|
RTS pin (for - supply)
Which is great for a RS232 serial port not for a Pi (excessive voltages) and could pick up electrical noise easier.
Just another techie on the net - For GPIO boards see http:///www.facebook.com/pcservicesreading
or http://www.pcserviceselectronics.co.uk/pi/

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Tue Nov 13, 2012 12:32 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
finally implemented this when my photoresistors arrived. works fantastically, thanks!

see what you mean about cutting out the backlight, I have to use different threshholds for night and day. working on a lighttighter solution

R

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Sat Nov 17, 2012 11:42 am

Can someone post a pic of how they've set this up against their meter? I'm just waiting on my Pi > breadboard wires but would like to know how some of you have set up, particularly so not to cause any problems when the meter reader comes round. Cheers

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Tue Nov 20, 2012 11:35 am

Ogoshi wrote:Can someone post a pic of how they've set this up against their meter? I'm just waiting on my Pi > breadboard wires but would like to know how some of you have set up, particularly so not to cause any problems when the meter reader comes round. Cheers
I haven't got a photo, but I just cut half way through a piece of card, slipped the photoresistor flush with one side and the wires coming through the slit then just sellotaped the card to the meter. i will put something behind to cut out daylight a little more but it seems to work for me.

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Mon Nov 26, 2012 1:11 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
Hi Yfory

This solution to reading the LED flashes on the meter worked fine...but I am having an issue with it not working that well when grabbing an image from a webcam at the same time. I put the priority of the monitor job up very high but it looks like the Pi is pausing the monitor for 8 seconds every time it accesses the webcam - maybe contacting the sd card? any idea?

Thanks

rooster
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm

Re: reading domestic electricity use

Mon Nov 26, 2012 1:16 pm

Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
incidentally, I though I might be able to use a similar setup to look at the electricity supply from a clamp meter which seems to have an AC voltage up to ~few volts. I wanted to rectify the circuit using a diode (in series with the 10 k resistor) , but this would need 0.7V across it to start charging the capacitor which would mean I couldn't look at low voltages. anything obvious I am missing to use a similar (and understandable!) method as this for looking at AC mV?

R

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Thu Nov 29, 2012 11:28 pm

Hi Yfory,

I've got put this together and I can see it is writing values to the database, however I'm having some trouble with the web side of things! Hoping you, or somebody here can help, hopefully it's something not too trivial since it seems to be working for others!

I've installed lighttpd and the various php setup bits, however when I go to the URL I get a server 500 error. I've had a look in various configuration files/Google and cannot find anything specific. I've also created a php test file which causes the page to show all the php info (phpinfo()) which does work so I guess the php is working to some extent...

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Fri Nov 30, 2012 11:16 pm

Ogoshi wrote:I've installed lighttpd and the various php setup bits, however when I go to the URL I get a server 500 error. I've had a look in various configuration files/Google and cannot find anything specific. I've also created a php test file which causes the page to show all the php info (phpinfo()) which does work so I guess the php is working to some extent...
Yfory wrote: Could be several reasons; simplest is to check file permissions. On my set up I have to set the owner/group to www-data using this command for example, chmod -R www-data:www-data *

If that doesn't work, are you trying to access the .php file via the web or something else?

Once a solution is found, be sure to add it as an enhancement to https://github.com/yfory/power/issues so others can benefit.
Thanks, that didn't seem to make a difference to it! I've been looking over it for a while now but cannot work it out! All I have found is that the issue seems to be somewhere between rows 5 and 11 of the index.php file as when I comment these out it loads the page ok...just with no data on the chart!

I have also done a new install of debian on a formatted card and still get the same issue so it may be something I have missed - will have another look through your instructions tomorrow.

Unfortunately I cannot get into the lighttpd logs folder in /var/log/lighttpd to see if anything more useful has been written there either.

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Sat Dec 01, 2012 1:10 pm

Yfory wrote:
Ogoshi wrote:Thanks, that didn't seem to make a difference to it! I've been looking over it for a while now but cannot work it out! All I have found is that the issue seems to be somewhere between rows 5 and 11 of the index.php file as when I comment these out it loads the page ok...just with no data on the chart!
I assume you have done the first two commands in the instructions, otherwise Python would be complaining too...

Code: Select all

sudo apt-get install sqlite3 libsqlite3-dev
Could you please try this command as root and see if it resolves the problem?
apt-get install php5-sqlite

Why are you unable to access your log folder?
I've found the problem!! I eventually managed to get into the lighttpd log folder by temporarily changing the folder permissions, had a look through that and it was saying it could not open the database which is in /var/db as instructed. So, had a look at the permissions of the power.db file and it appears that it was set to 640, and also assigned to the root group, therefore, I presume as the server is running as www-data it did not have the 'public' permission to read it. I've now changed the database file to 644 and it works great :D

Thank you very much for this, and apologies for hassling! I'll add a note to the issues on github in case anybody else has the same trouble!

Now to see what I can do further with this :?

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Sun Dec 16, 2012 11:36 pm

Cheers for that little script, I've added some other bits to it and added it to the cron jobs so that it sends me an e-mail each day with a simple summary of kWh used and the cost in GBP. I've got it set to do this at 08:00 am daily, however any time after 00:00 would of course work.

It's set up to use Gmail SMTP servers, however you could change this if you know the details of your own provider.

It may not be the most perfect python code, but I'm learning it, it works for me and I'm happy with it! Therefore use at your own risk!

Code: Select all

#!/usr/bin/python

import sys
import smtplib
import socket
from email.mime.text import MIMEText
import datetime
import sqlite3 as sqlite, os

# Change to your own account information
to = 'recipient@domain.com'
gmail_user = 'sender@gmail.com'
gmail_password = 'senderpasswordhere'
try:
  smtpserver = smtplib.SMTP('smtp.gmail.com', 587)
  smtpserver.ehlo()
  smtpserver.starttls()
  smtpserver.ehlo
  smtpserver.login(gmail_user, gmail_password)
except:
  sys.exit()

#Connect to the power database
dbconn=None
pulsedb = "/var/db/power.db"
pulses = 0
try:
  dbconn=sqlite.connect(pulsedb)
  dbdata=dbconn.cursor()
  dbdata.execute("select count(*) from pulse where substr(datetime(stamp,'unixepoch','localtime'),0,11) like date('now','-1 day')")
  pulses=dbdata.fetchone()[0]
  dbconn.close()
except:
  sys.exit()

#Calculate yesterdays usage
kw = round(float(float(pulses)/1000),3)
consumption = str(kw) + 'kWh consumed (19.670ppkWh): ' + str(round((kw*0.19670),2)) + ' GBP'

#E-Mail yesterdays usage
yesterday=datetime.date.today()-datetime.timedelta(days=1)
try:
  msg = MIMEText(consumption)
  msg['Subject'] = 'Electricity Consumption on %s' % yesterday.strftime('%b %d %Y')
  msg['From'] = gmail_user
  msg['To'] = to
  smtpserver.sendmail(gmail_user, [to], msg.as_string())
  smtpserver.quit()
except:
  sys.exit()

sys.exit()

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Sun Dec 16, 2012 11:44 pm

One other script I have set up on the cron scheduler is to remove duplicate entries within the database as I was finding around one third of the entries were duplicates. These may exist due to my setup, but nevertheless may be helpful to somebody.
I have this set to run at one minute past every hour as I have another process which picks up the data hourly, however you could run this daily, as long as it's before using the database for any reporting.

Again, I'm happy that this does what it's meant to, but use at your own risk - this will be deleting data from your database table, and you could lose data for which I will not be held responsible!! Make a backup first!

Code: Select all

#!/usr/bin/python

import sys
import sqlite3 as sqlite

# Remove duplicate entries from the database where it has written the same epoch time twice or more

# Connect to the power database
pulsedb = "/var/db/power.db"
try:
  con = sqlite.connect(pulsedb)
  cur = con.cursor()
  cur.execute("delete from pulse where rowid in (select min(rowid) from pulse group by stamp having count(stamp)>1)")
  con.commit()
  con.close()
except:
  sys.exit()

sys.exit()

Hoagie
Posts: 119
Joined: Sun Sep 23, 2012 7:19 pm

Re: reading domestic electricity use

Tue Dec 18, 2012 4:20 pm

Right, I must admit I payed no attention whatsoever to S/N on my setup, though mine is in a place that's always dark.

Kimkos
Posts: 1
Joined: Fri Aug 02, 2013 12:44 pm

Re: reading domestic electricity use

Fri Aug 02, 2013 12:59 pm

I noticed that the git from yfory is down so the code isn`t available anymore. Is it possible to put the code on github again or share the code here so I could use it for my own and see how it's made and practice with it so I could learn from it?

Greetings,
Kimkos

Zodiac69
Posts: 5
Joined: Fri Apr 26, 2013 7:38 pm

Re: reading domestic electricity use

Mon Sep 23, 2013 9:51 am

Ditto here.

Tried to send him a PM without success.

PsycoMEn
Posts: 25
Joined: Mon Aug 06, 2012 11:11 am

Re: reading domestic electricity use

Wed Oct 23, 2013 1:10 pm

I'm also very interested.

Br,
Psycomen

Ogoshi
Posts: 46
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England

Re: reading domestic electricity use

Sun Oct 27, 2013 10:22 pm

Hi,

I've just loaded up the old SD card I was using for this project and it looks like I have the necessary files still. I've attached the required files as a zip file to this.

There were some amendments I made to the original to make some small improvements on the analysis side of things. I'll detail them below, but feel free to ask questions and I'll try and answer.

Just follow yfory's original readme which is in the zip, obviously skipping bits such as copying from the git etc.

Essentially you need to move the database file to the specified directory and the power-monitor file to the init.d folder. Make sure the latter references the monitor.py file in the correct directory, it's currently set up to find it in /home/pi/power.

That should get you up and running.

On top of what yfory done I created a py file which deletes duplicates, as I found quite often there would be 2 entries in the database with exactly the same epoch time, which was then inflating the usage analysis. I had a cron job setup to execute this duplicates check every hour on the hour.

Also, I found the php setup yfory created wasn't working for me as everytime the page was opened it executed a query directly against the database which subsequently caused problems in the end. Therefore I created a python script to create a basic csv file every hour (just after the duplicates job ran), and a html file complete with highcharts instead, which queried the csv file.
To use this, set the csv sh file on the crontab (check the output from this is your www directory), and copy the content of the www folder to where your web server is setup, normally /var/www/.

All credit goes to yfory for monitor.py, power.db and power-monitor. Hopefully he doesn't mind me re-posting his work...

Don't plan on using your Pi for much other than this - the process pretty much has the CPU sitting permanently on max, or near as, all the time.
power.zip
Files required for logging electricity consumption
(50.93 KiB) Downloaded 306 times

Machasm
Posts: 11
Joined: Mon Jan 28, 2013 3:54 pm

Re: reading domestic electricity use

Fri Nov 01, 2013 5:35 pm

Thank you Ogishi for reporting the files that wee on the GitHub.
I am fairly new to Linux but have all the necessary components to build this.
There are a few things that I don't understand however.
1) In yfory original instructions he mentions copying a 'config.php' file to a directory. I can't find this config.php file anywhere. Is this because you have replaced it with your python script?
Help here would be appreciated.
In fact would it be possible to have a noobs guide to setting this whole thing up with these newly created files as I am finding it difficult to piece the information from the original poster with the updated info from yourself.
Thanks in advance.
Mac.

Return to “Automation, sensing and robotics”