CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Jul 31, 2018 2:29 pm

Hi Lee

sorry to hear about your fire. Fire's are never nice.

You say you have

MS4448PAE....master,slave,
PT-100 charge controller
BMK
Magweb.

I infer from this you have
TWO MS4448PAE
ONE PT-100
ONE BMK
ONE Magweb

But you do not have an AGS Auto Generator Start device.

This is not a problem, I'm just making sure.

I am currently fixing up my Magnum reader to be more indifferent to the configuration it is working with. I working with Yam in BC who has a very large system. I hope to be finished in august.

My reader will be able to generate either a CSV file or a JSON file (for techies who want to do more). When it's closer to ready I'll post it.

I have no experience with the Magweb

Meanwhile if you're interested let me know your email and I'll send you instructions for building the required Magnum network to Pi interface. It's cheap and, i believe, easy to make. There is also a bundle of test software that's not ready for public use.

The only way I have found to send an email in this forum is to post in the thread and then delete as soon as you are contacted.

PS Yes, I'm much further south, only 45 degrees north.

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Jul 31, 2018 7:00 pm

@lee
Do you have a ME-ARC remote control?

EdwardHomestead
Posts: 2
Joined: Sun Jul 29, 2018 10:46 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Fri Aug 03, 2018 4:37 pm

I have the RTR,

No, I do not have the ARC.
Last edited by EdwardHomestead on Fri Aug 03, 2018 6:52 pm, edited 1 time in total.

Willeert
Posts: 7
Joined: Tue Jun 14, 2016 2:06 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Fri Aug 03, 2018 5:04 pm

I have not made much progress towards getting an Arduino to communicate with the Magnum system. Please, when you have time, would you also send me the information on the equipment needed to have the Magnum equipment communicate with a Pi. I bought a bunch of 485 parts in an effort to capture the Magnum packets as a first step but so far have had no success.

Thank you

Will

[email protected]

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Fri Aug 03, 2018 7:51 pm

I have prepared a short document on how to build an RS485 interface to Magnum energy. The document has not been field tested and my have errors. https://goo.gl/u4vaXZ

Willeert
Posts: 7
Joined: Tue Jun 14, 2016 2:06 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Aug 04, 2018 2:05 am

Charles - Thank You!

Will

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Wed Sep 05, 2018 3:04 pm

Interested in this thread, certainly glad I found it and excited that its still somewhat active.

I am on the front end of wanting to do something very similar. I'm thinking that I want to build a network interface to front end both a Magnum inverter and an Outback charge controller. I have an ARC, AGS, BMK and Magweb in the loop. Want to send the data captured by all of the devices (including the Outback) as well as receive update inputs from an upstream system that I will write to both serve as a web controller (emulate the ARC), as well as a data visualization portal.

Charles, im wondering if you have gotten your code to a point yet where you are ready to share the source?

Thanks in advance,
Scott...

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Wed Sep 05, 2018 8:20 pm

Thanks for your interest.r.

I too run a hybrid system with a MorningStar charge controller. Concerning your plans - I have no experience emulating an ARC remote. I'm not sure if you can easily add software to the system as an ARC. But you can try. I just can't add any useful information.

I have a web site providing an information dashboard that is integrated into a custom logger I built. All of this was written for my internal version of this code. I wrote it 5 years ago for my use andhave not rewritten the logger and website to work with the new one I've just developed. It's underway but a long way form being done.As far as the code is concerned it is part of the install image available if you read https://goo.gl/u4vaXZ and install the software.

Read to programmer section too.

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 08, 2018 5:38 pm

This has been resolved.

I've hit some snags.

although I have all this code running s expected on my old veteran Pi 2 B, It is not running on Pi 3 B systems. 2 testers reported the problem and I have reproduced it with a clean machine. I don't know why yet.

The symptom is there is no data at the USB port. I must have done something to make it work on the old Pi that has slipped from memory. I'll stat testing.

The problem seems to be OS version related. I cab switch OS between two Pi (a 2 B and a 3 B). The problem is with the OS, not the CPU
Last edited by CharlesGodwin on Tue Sep 11, 2018 11:57 am, edited 1 time in total.

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 1:04 am

Thank you! Was just in the process of installing on my Raspi 3

Please let me know what you find.

Scott...

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 2:04 am

Problems have been resolved. Should be ok to use

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 11:54 am

Thanks!

I downloaded last night, wondering if what I have has your fixes?

Scott...

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 11:59 am

The changes were posted at 17:02 EDT Monday, Sept 10.

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 12:18 pm

Thank you, that should do it. I downloaded last night @ 21:06 CST.

Anxious to give this a try. Charles, this is a pretty amazing piece of work you have put together. I really appreciate the amount of effort you have put in to make the install instructions clear! I need to get this on my system to start reading packets, but so far so good.

Seems like there was a ME simulator that was written some time back that reads data packets from storeage, and sends them down the USB channel (on a separate device) so you could do testing without being near the ME inverter. Did you write this simulator?

Scott...

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 11, 2018 1:12 pm

I don't know of the emulator as you describe it. But...

I have an ME "emulator" built into my server. When you install the software you are asked to run a test named testrs585. It generates a file with a HEX dump of 100 packets and my server can be started to read that file and serve up that data instead of reading a ME network. The limitation is it is a revolving buffer of data so you get the same packet after the data has been read but it is useful for testing parts of the system and analyzing other users systems. The server defaults to consuming 50 packets for each data request.

The test sample can be increased by running testrs485 with the parameter packets=nnnnn i.e.

Code: Select all

./tests485 packets=1000
To run the server using the generated data use

Code: Select all

./startserver mockdata=true mockdatafile=<fully qualified name of file generated by testrs485>

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 12:30 am

I have the server up, and its logging packets. That part is awesome!

The bad part is that I think that only one device on the network can read the packets cause now my normal magweb interface is not showing any statistics for my system.

http://data.magnumenergy.com/MW5278/

I assume that once the server reads the packet the magweb device doesnt get the packets.

Scott...

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 1:17 am

I got the magweb working again. Something must have been out of sync in the setup. I unplugged all the connections then got the magweb working again. Next I start the server and the logger and everything was great. Sorry for the false alarm.

Next step: Get the logger running on a device that is offsite and push the JSON to a db and build a web interface!

Will keep you in the loop.

Scott...

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 3:42 am

Charles,

I'd like to make some changes to the java for my local setup. Did you happen to use a build tool like Maven or were you running build scripts manually? Would you be willing to share your pom.xml or the shell script you used to build the jar? Would probably save me a lot of time.

Scott...

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 11:09 am

sagreen83 wrote:
Sat Sep 15, 2018 1:17 am
I got the magweb working again. Something must have been out of sync in the setup. I unplugged all the connections then got the magweb working again. Next I start the server and the logger and everything was great. Sorry for the false alarm.

Next step: Get the logger running on a device that is offsite and push the JSON to a db and build a web interface!

Will keep you in the loop.

Scott...
@Scott. glad its working. I'd really like a copy of the txt file generated when you run the testrs485 script as I'm trying to fine tune the packet processing and every site seems to have different packet "features" so it's a learning process. Please let me know what equipment you have in the network too.

Thanks
Charles

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 11:13 am

sagreen83 wrote:
Sat Sep 15, 2018 3:42 am
Charles,

I'd like to make some changes to the java for my local setup. Did you happen to use a build tool like Maven or were you running build scripts manually? Would you be willing to share your pom.xml or the shell script you used to build the jar? Would probably save me a lot of time.

Scott...
I use the Eclipse IDE http://www.eclipse.org/ on a Win 10 PC and copy the results to a Pi using a Samba mounted drive.

I can't provide the exported project at this time as I have other distractions preventing me from making it a shareable project. However, if you know Eclipse its easy to import the src directory into an Eclipse project. If you don't know Eclipse, then there's a learning curve. What ever you do you will need to include the 4 jar files used for run time. the script mesetup can show you where to find the originals.

Building the jar file is straight forward as there are no special manifest items. It's just a jar file. I use an ant task.

I plan to migrate to Visual Studio Code platform but that's months away.

I'm curious. what do you need to change? I may be able to help or simplify your efforts. I have had a version of this system running for years with a logger that calculates statistics and feeds a website dashboard. Both are primitive but cover the principles.
Last edited by CharlesGodwin on Sun Sep 16, 2018 11:45 am, edited 1 time in total.

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Sat Sep 15, 2018 9:00 pm

I've just updated the reader using information provided by contributing testers (Thanks, Lee and David) and I think this is pretty close to being version 1.0. But I'd like feedback and generated data from other users. ;)
To update your site download magnum.zip and mesetup to your install directory and run

Code: Select all

bash mesetup

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Sun Sep 16, 2018 4:26 pm

We needed to change the date format to ISO 8601 Notation. Was required to get mondodb to import it as a date. We were able to do that and rebuild the .jar file. I have the system now logging locally on the PI and sending the JSON to an offsite mondodb. Starting to code the UI in node.js.

The things that I plan to change are:

1) Get the system to import logs from my Outback FlexMax 80 charge controller. Interested in the process you use to get your charge controllers data into the server.

2) Make the system bi-directional. I really want to be able to change settings on my devices remotely. I know that there are significant changes required to do this.

3) Ultimately I'd like this all to run on an ESP32 (Arduino Variant). I like the raspberry pi for one off projects, but the ESP32 is a bit more imbedded and compact. Has an awesome deep sleep mode that allows you to run the system on a lipo battery.

My configuration is

Magnum MS4448PAE
MagWeb interface
BMK
AGS (w/Honda EU7000is generator)
ME-ARC
Outback FlexMax 80 charge controller
12 VmaxTank 155slr Batteries (420ah)
6 300 watt 72 cell solar panels.

Scott...

p.s. I still owe you that file. I'll get it for you.

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Sun Sep 16, 2018 8:29 pm

sagreen83 wrote:
Sun Sep 16, 2018 4:26 pm
Edited for brevity
We needed to change the date format to ISO 8601 Notation. Was required to get mondodb to import it as a date. We were able to do that and rebuild the .jar file. I have the system now logging locally on the PI and sending the JSON to an offsite mondodb. Starting to code the UI in node.js.

I revised the program so the date format is a settable option. Refer to page 6 of my interface document. this will allow others to fine tune to suit their needs too. I for, one am no convinced I need 100th of a second precision on minute by minute logs.

The things that I plan to change are:

1) Get the system to import logs from my Outback FlexMax 80 charge controller. Interested in the process you use to get your charge controllers data into the server.
See below

2) Make the system bi-directional. I really want to be able to change settings on my devices remotely. I know that there are significant changes required to do this.
You're on your own on this one. :D I have enough trouble reading their packets. But, using relay boards, I have a plan to remote start and stop my generator, but that doesn't need Magnum interaction.

3) Ultimately I'd like this all to run on an ESP32 (Arduino Variant). I like the raspberry pi for one off projects, but the ESP32 is a bit more imbedded and compact. Has an awesome deep sleep mode that allows you to run the system on a lipo battery.
remainder deleted....
Plugins
Adding extra "device" data is straight forward but not yet documented. You need to create a java class that implements ca.godwin.magnum.DataReader interface. This class is dynamically loaded at run-time using the plugins= option. Your can have multiple classes by coma delimiting their class names in the option. This class is responsible for returning an ArrayList of objects that implement ca.godwin.magnum.DeviceRecord. How you collect the data in these objects is implementation specific. For example my MorningStar controller has a ModBus interface so I communicate using their published interface, extract the data I want, populate the object so the two methods provide data and I'm done. I also have a class that returns a weather record with current temperature, pressure and humidity. The values are appended after the magnum fields in the order that the plugins are defined and order of each DeviceRecord object in the ArrayList.

NOTE that any extra libraries used by your plugin must be added to the classpath defined in the startserver script. I had to add pi4j directory to support my temperature device and add a .jar for modbus support. I recommend you create your own, separate, script to prevent a future override of the standard script.

sagreen83
Posts: 15
Joined: Wed Sep 05, 2018 2:54 pm

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 18, 2018 12:30 am

CharlesGodwin wrote:
Sat Sep 15, 2018 11:09 am
@Scott. glad its working. I'd really like a copy of the txt file generated when you run the testrs485 script as I'm trying to fine tune the packet processing and every site seems to have different packet "features" so it's a learning process. Please let me know what equipment you have in the network too.

Thanks
Charles
testRS485-20180917_1920.zip
(1.99 KiB) Downloaded 14 times
@Charles

See attached file created from testrs485

Looks like there are some unknown's in the file. such as:


Bytes read:21:UNKNOWN=>0005140A6419280601EB9B820005000000000000D0
Bytes read:22:UNKNOWN=>400001F8000577000100331E312A73000002025800FE

Again, my configuration is

Magnum MS4448PAE
MagWeb interface
BMK
AGS (w/Honda EU7000is generator)
ME-ARC
Outback FlexMax 80 charge controller
12 VmaxTank 155slr Batteries (420ah)
6 300 watt 72 cell solar panels.

Thanks,
Scott...
Last edited by sagreen83 on Tue Sep 18, 2018 12:33 am, edited 1 time in total.

CharlesGodwin
Posts: 51
Joined: Wed Aug 22, 2012 9:57 pm
Location: Ottawa Canada

Re: Using a Pi to Log Data in a Magnum Energy System

Tue Sep 18, 2018 12:32 am

Are you using the latest version. I made some changes in the last 2 days

Return to “Automation, sensing and robotics”