reading domestic electricity use


39 posts   Page 1 of 2   1, 2
by rooster » Thu Nov 01, 2012 11:20 am
Hi

I bought an efergy electricity meter http://www.efergy.com/ which has a clip on induction coil that goes on the main live wire at the fusebox. The clip on has a mini jack connection to a box which sends the reading via rf to an inhome display.

I want to get the electricity use from the induction coil into my Rpi. What's the best way to do it?

1 - is there a min-jack -> usb connector so I can 'read' the real time voltage of the clip on (which I assume is in proportion to the current in the mains wire)?

2 - can I read via GPIP pins? ie connect the voltage at the mini-jack to an A-D converter, the pins of which are read by the GPIO?

3 - any other?

bearing in mind I don't have an oscilloscope

Thanks!
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by scruss » Thu Nov 01, 2012 11:51 am
The efergy uses a fairly standard current clamp or CT. It needs an A/D converter, which the Raspberry Pi lacks. There are many Arduino projects to read CT data, one of which being OpenEnergyMonitor.

CTs aren't that accurate, and the limited resolution of microcontroller A/D converters doesn't help.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
User avatar
Posts: 354
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
by Yfory » Thu Nov 01, 2012 2:50 pm
I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power
Posts: 96
Joined: Thu Apr 19, 2012 10:29 am
by penguintutor » Thu Nov 01, 2012 3:10 pm
Does the Display have a serial or usb connector on it - if so you may be able to connect that to your Pi instead?

The one I have came free from my energy supplier when I switched to a online statement tariff.

Whilst branded by the energy company is made by Current Cost.
They provide details of the protocol to communicate via serial over USB.
http://www.currentcost.com/cc128/xml.htm

I've created a Python script that can read the current status which I've run on the Pi to test connectivity works. I got it working directly, but my first attempt at converting it to a background daemon didn't work. I've not got the time to work through this any further at the moment as I have some other projects that are more useful. I can provide some basic python code if anyone is interested.

If you're display doesn't have any way of connecting it to a computer then some other people have hobby projects looking at the hardware for the receiver - if you feel up to the challenge.

http://alastair.d-silva.org/hacking-efergy-wireless-power-meter
Read some of the comments and links from that page.
User avatar
Posts: 338
Joined: Tue May 08, 2012 9:11 am
Location: UK
by Ogoshi » Thu Nov 01, 2012 8:15 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power


Thank you so much for providing this! I have no electronics skills, so was hoping to use Google and here to work something out to read the blinks from our meter, as I never found the coil + monitor to be very reliable!
I'm going to order the parts I need and hope it goes well!
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by rooster » Fri Nov 02, 2012 11:57 am
scruss wrote:The efergy uses a fairly standard current clamp or CT. It needs an A/D converter, which the Raspberry Pi lacks. There are many Arduino projects to read CT data, one of which being OpenEnergyMonitor.

CTs aren't that accurate, and the limited resolution of microcontroller A/D converters doesn't help.


I think I want to try it this way - can you recommend an AD converter? I haven't done this since some temperature stuff 15 years ago! I'll look at the arduino site
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by rooster » Fri Nov 02, 2012 12:25 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power


really nice, thanks! have ordered my photoresistor off amazon and will try out next week!
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by scruss » Sun Nov 04, 2012 1:09 pm
rooster wrote:I think I want to try it this way - can you recommend an AD converter?


The Arduino has several 10-bit ADC channels, so it'll be good enough to track energy usage trends. It won't be accurate enough to check your electricity bills to the penny, though.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.
User avatar
Posts: 354
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
by Hoagie » Fri Nov 09, 2012 12:29 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power



I just took a look at the Circuid diagram and it appears to me that you have something like this...
+3.3V
|
10k Resistor
|
Photoresistor
|
+ Raspberry Pi input.
|
Capacitor
|
Gnd

Which seems a bit odd in a two ways.
I'd expect the Raspberry Pi input. to be between the resistor and photoresistor,
I'd expect the capacitor to be across the +3.3 and Gnd to filter it.


I have a similar (but not Raspberry Pi driven) system working with a PC serial port and use this layout

DRT pin (for + supply)
|
100k Resistor
|
+ Input to DCD pin
|
Photoresistor
|
RTS pin (for - supply)

I use C for the software and feed the data to RRDTool to produce graphs. like this http://www.hoagieshouse.com/Electric_Meter/today.png
Posts: 94
Joined: Sun Sep 23, 2012 7:19 pm
by techpaul » Fri Nov 09, 2012 5:35 pm
Hoagie wrote:
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power



I just took a look at the Circuid diagram and it appears to me that you have something like this...
+3.3V
|
10k Resistor
|
Photoresistor
|
+ Raspberry Pi input.
|
Capacitor
|
Gnd

Which seems a bit odd in a two ways.
I'd expect the Raspberry Pi input. to be between the resistor and photoresistor,
That way you may not ever get the voltage going low enough so would always appear as on.
I'd expect the capacitor to be across the +3.3 and Gnd to filter it.
That would filter the 3V3 NOT the signal which could still pick up other light sources or noise.
I have a similar (but not Raspberry Pi driven) system working with a PC serial port and use this layout

DRT pin (for + supply)
|
100k Resistor
|
+ Input to DCD pin
|
Photoresistor
|
RTS pin (for - supply)
Which is great for a RS232 serial port not for a Pi (excessive voltages) and could pick up electrical noise easier.
Just another techie on the net - For GPIO boards see http:///www.facebook.com/pcservicesreading
or http://www.pcserviceselectronics.co.uk/pi/
Posts: 1510
Joined: Sat Jul 14, 2012 6:40 pm
Location: Reading, UK
by rooster » Tue Nov 13, 2012 12:32 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power


finally implemented this when my photoresistors arrived. works fantastically, thanks!

see what you mean about cutting out the backlight, I have to use different threshholds for night and day. working on a lighttighter solution

R
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by Ogoshi » Sat Nov 17, 2012 11:42 am
Can someone post a pic of how they've set this up against their meter? I'm just waiting on my Pi > breadboard wires but would like to know how some of you have set up, particularly so not to cause any problems when the meter reader comes round. Cheers
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by rooster » Tue Nov 20, 2012 11:35 am
Ogoshi wrote:Can someone post a pic of how they've set this up against their meter? I'm just waiting on my Pi > breadboard wires but would like to know how some of you have set up, particularly so not to cause any problems when the meter reader comes round. Cheers


I haven't got a photo, but I just cut half way through a piece of card, slipped the photoresistor flush with one side and the wires coming through the slit then just sellotaped the card to the meter. i will put something behind to cut out daylight a little more but it seems to work for me.
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by rooster » Mon Nov 26, 2012 1:11 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power


Hi Yfory

This solution to reading the LED flashes on the meter worked fine...but I am having an issue with it not working that well when grabbing an image from a webcam at the same time. I put the priority of the monitor job up very high but it looks like the Pi is pausing the monitor for 8 seconds every time it accesses the webcam - maybe contacting the sd card? any idea?

Thanks
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by rooster » Mon Nov 26, 2012 1:16 pm
Yfory wrote:I didn't want to spend money on gadgets like this, and I'm fortunate enough to have an electricity meter that encorporates a flashing LED. I built a simple circuit and some Python code to read the blinks and store them in a database, from which I can create my own graphs. You can grab the source code (and example circuit) from github: http://github.com/yfory/power


incidentally, I though I might be able to use a similar setup to look at the electricity supply from a clamp meter which seems to have an AC voltage up to ~few volts. I wanted to rectify the circuit using a diode (in series with the 10 k resistor) , but this would need 0.7V across it to start charging the capacitor which would mean I couldn't look at low voltages. anything obvious I am missing to use a similar (and understandable!) method as this for looking at AC mV?

R
Posts: 100
Joined: Thu Aug 02, 2012 8:09 pm
by Ogoshi » Thu Nov 29, 2012 11:28 pm
Hi Yfory,

I've got put this together and I can see it is writing values to the database, however I'm having some trouble with the web side of things! Hoping you, or somebody here can help, hopefully it's something not too trivial since it seems to be working for others!

I've installed lighttpd and the various php setup bits, however when I go to the URL I get a server 500 error. I've had a look in various configuration files/Google and cannot find anything specific. I've also created a php test file which causes the page to show all the php info (phpinfo()) which does work so I guess the php is working to some extent...
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by Yfory » Fri Nov 30, 2012 12:04 pm
rooster wrote:This solution to reading the LED flashes on the meter worked fine...but I am having an issue with it not working that well when grabbing an image from a webcam at the same time. I put the priority of the monitor job up very high but it looks like the Pi is pausing the monitor for 8 seconds every time it accesses the webcam - maybe contacting the sd card? any idea?


You may want to look in to coding a C/C++ solution as that would put less load on your Pi than the supplied Python solution. Also, it may be an issue with not enough power being available, so I would suggest trying a different power supply for your Pi or using a powered USB hub for the webcam.

rooster wrote:incidentally, I though I might be able to use a similar setup to look at the electricity supply from a clamp meter which seems to have an AC voltage up to ~few volts. I wanted to rectify the circuit using a diode (in series with the 10 k resistor) , but this would need 0.7V across it to start charging the capacitor which would mean I couldn't look at low voltages. anything obvious I am missing to use a similar (and understandable!) method as this for looking at AC mV?


I haven't played with clamp meters but I would personally pull it apart and wire in to whatever is producing the digital display output. There's probably far easier & less hackity ways to do it. ;)

Ogoshi wrote:I've installed lighttpd and the various php setup bits, however when I go to the URL I get a server 500 error. I've had a look in various configuration files/Google and cannot find anything specific. I've also created a php test file which causes the page to show all the php info (phpinfo()) which does work so I guess the php is working to some extent...


Could be several reasons; simplest is to check file permissions. On my set up I have to set the owner/group to www-data using this command for example, chmod -R www-data:www-data *

If that doesn't work, are you trying to access the .php file via the web or something else?

Once a solution is found, be sure to add it as an enhancement to https://github.com/yfory/power/issues so others can benefit.
Posts: 96
Joined: Thu Apr 19, 2012 10:29 am
by Ogoshi » Fri Nov 30, 2012 11:16 pm
Ogoshi wrote:I've installed lighttpd and the various php setup bits, however when I go to the URL I get a server 500 error. I've had a look in various configuration files/Google and cannot find anything specific. I've also created a php test file which causes the page to show all the php info (phpinfo()) which does work so I guess the php is working to some extent...


Yfory wrote:Could be several reasons; simplest is to check file permissions. On my set up I have to set the owner/group to www-data using this command for example, chmod -R www-data:www-data *

If that doesn't work, are you trying to access the .php file via the web or something else?

Once a solution is found, be sure to add it as an enhancement to https://github.com/yfory/power/issues so others can benefit.


Thanks, that didn't seem to make a difference to it! I've been looking over it for a while now but cannot work it out! All I have found is that the issue seems to be somewhere between rows 5 and 11 of the index.php file as when I comment these out it loads the page ok...just with no data on the chart!

I have also done a new install of debian on a formatted card and still get the same issue so it may be something I have missed - will have another look through your instructions tomorrow.

Unfortunately I cannot get into the lighttpd logs folder in /var/log/lighttpd to see if anything more useful has been written there either.
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by Yfory » Sat Dec 01, 2012 11:54 am
Ogoshi wrote:Thanks, that didn't seem to make a difference to it! I've been looking over it for a while now but cannot work it out! All I have found is that the issue seems to be somewhere between rows 5 and 11 of the index.php file as when I comment these out it loads the page ok...just with no data on the chart!


I assume you have done the first two commands in the instructions, otherwise Python would be complaining too...
Code: Select all
sudo apt-get install sqlite3 libsqlite3-dev


Could you please try this command as root and see if it resolves the problem?
apt-get install php5-sqlite

Why are you unable to access your log folder?
Posts: 96
Joined: Thu Apr 19, 2012 10:29 am
by Ogoshi » Sat Dec 01, 2012 1:10 pm
Yfory wrote:
Ogoshi wrote:Thanks, that didn't seem to make a difference to it! I've been looking over it for a while now but cannot work it out! All I have found is that the issue seems to be somewhere between rows 5 and 11 of the index.php file as when I comment these out it loads the page ok...just with no data on the chart!


I assume you have done the first two commands in the instructions, otherwise Python would be complaining too...
Code: Select all
sudo apt-get install sqlite3 libsqlite3-dev


Could you please try this command as root and see if it resolves the problem?
apt-get install php5-sqlite

Why are you unable to access your log folder?


I've found the problem!! I eventually managed to get into the lighttpd log folder by temporarily changing the folder permissions, had a look through that and it was saying it could not open the database which is in /var/db as instructed. So, had a look at the permissions of the power.db file and it appears that it was set to 640, and also assigned to the root group, therefore, I presume as the server is running as www-data it did not have the 'public' permission to read it. I've now changed the database file to 644 and it works great :D

Thank you very much for this, and apologies for hassling! I'll add a note to the issues on github in case anybody else has the same trouble!

Now to see what I can do further with this :?
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by Yfory » Sat Dec 01, 2012 1:26 pm
Ogoshi wrote:Thank you very much for this, and apologies for hassling! I'll add a note to the issues on github in case anybody else has the same trouble!


No hassle at all! I'm glad you found the problem and thank you for sharing the solution on github, I am sure it will help others. :)

Here's a simple Python script I use to calculate how much in total the cost has been:
Code: Select all
import sqlite3 as sqlite, datetime, os

dbconn=None
pulsedb="/var/db/power.db"
pulses=0

dbconn=sqlite.connect(pulsedb)
dbdata=dbconn.cursor()
dbdata.execute("select count(stamp) from pulse")
pulses=dbdata.fetchone()[0]
dbconn.close()

kw=round(float(float(pulses)/800),3)

print "==[ Total Recorded ]============"
print "Number of pulses:",pulses
print "1kWh=800 pulses:",kw

print "Ebico (17.35ppkWh):",round((kw*0.1735),2),"GBP"
Posts: 96
Joined: Thu Apr 19, 2012 10:29 am
by Ogoshi » Sun Dec 16, 2012 11:36 pm
Cheers for that little script, I've added some other bits to it and added it to the cron jobs so that it sends me an e-mail each day with a simple summary of kWh used and the cost in GBP. I've got it set to do this at 08:00 am daily, however any time after 00:00 would of course work.

It's set up to use Gmail SMTP servers, however you could change this if you know the details of your own provider.

It may not be the most perfect python code, but I'm learning it, it works for me and I'm happy with it! Therefore use at your own risk!

Code: Select all
#!/usr/bin/python

import sys
import smtplib
import socket
from email.mime.text import MIMEText
import datetime
import sqlite3 as sqlite, os

# Change to your own account information
to = 'recipient@domain.com'
gmail_user = 'sender@gmail.com'
gmail_password = 'senderpasswordhere'
try:
  smtpserver = smtplib.SMTP('smtp.gmail.com', 587)
  smtpserver.ehlo()
  smtpserver.starttls()
  smtpserver.ehlo
  smtpserver.login(gmail_user, gmail_password)
except:
  sys.exit()

#Connect to the power database
dbconn=None
pulsedb = "/var/db/power.db"
pulses = 0
try:
  dbconn=sqlite.connect(pulsedb)
  dbdata=dbconn.cursor()
  dbdata.execute("select count(*) from pulse where substr(datetime(stamp,'unixepoch','localtime'),0,11) like date('now','-1 day')")
  pulses=dbdata.fetchone()[0]
  dbconn.close()
except:
  sys.exit()

#Calculate yesterdays usage
kw = round(float(float(pulses)/1000),3)
consumption = str(kw) + 'kWh consumed (19.670ppkWh): ' + str(round((kw*0.19670),2)) + ' GBP'

#E-Mail yesterdays usage
yesterday=datetime.date.today()-datetime.timedelta(days=1)
try:
  msg = MIMEText(consumption)
  msg['Subject'] = 'Electricity Consumption on %s' % yesterday.strftime('%b %d %Y')
  msg['From'] = gmail_user
  msg['To'] = to
  smtpserver.sendmail(gmail_user, [to], msg.as_string())
  smtpserver.quit()
except:
  sys.exit()

sys.exit()
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by Ogoshi » Sun Dec 16, 2012 11:44 pm
One other script I have set up on the cron scheduler is to remove duplicate entries within the database as I was finding around one third of the entries were duplicates. These may exist due to my setup, but nevertheless may be helpful to somebody.
I have this set to run at one minute past every hour as I have another process which picks up the data hourly, however you could run this daily, as long as it's before using the database for any reporting.

Again, I'm happy that this does what it's meant to, but use at your own risk - this will be deleting data from your database table, and you could lose data for which I will not be held responsible!! Make a backup first!

Code: Select all
#!/usr/bin/python

import sys
import sqlite3 as sqlite

# Remove duplicate entries from the database where it has written the same epoch time twice or more

# Connect to the power database
pulsedb = "/var/db/power.db"
try:
  con = sqlite.connect(pulsedb)
  cur = con.cursor()
  cur.execute("delete from pulse where rowid in (select min(rowid) from pulse group by stamp having count(stamp)>1)")
  con.commit()
  con.close()
except:
  sys.exit()

sys.exit()
Posts: 44
Joined: Wed Dec 28, 2011 5:13 pm
Location: Southend-on-Sea, Essex, England
by Hoagie » Tue Dec 18, 2012 4:20 pm
Right, I must admit I payed no attention whatsoever to S/N on my setup, though mine is in a place that's always dark.
Posts: 94
Joined: Sun Sep 23, 2012 7:19 pm
by Kimkos » Fri Aug 02, 2013 12:59 pm
I noticed that the git from yfory is down so the code isn`t available anymore. Is it possible to put the code on github again or share the code here so I could use it for my own and see how it's made and practice with it so I could learn from it?

Greetings,
Kimkos
Posts: 1
Joined: Fri Aug 02, 2013 12:44 pm