Binding For Brutlech GEM Whole-home Energy Monitor?

Add-ons Bindings
, ,
1

Continuing the discussion from Calculating kWh from instantaneous W:

I used a breakout from the OP so I didn’t further pollute their thread.

There are a couple of routes that Brultech GEM Energy Monitor integration could go, but I think the best would be to adapt the existing btmon.py script to support publication of the data to a new target - specifically, a set of MQTT topics, one per Energy Monitoring channel, Pulse-Counters, and Temperature input they expose

BTMon is already setup to pull data from a variety of sources (different HEM’s, mostly) and publish that data to a variety of targets (SmartEnergyGroups.com, PVOutput.org, RRD etc)

This is how I get my stuff to SEG, for graphing and visual analysis, and also out to PVOutput (for Solar).

Anyhow, if I go this route, then a set of MQTT topics could also be published from it, using something like publish.multiple from Paho

It would likely be very little change to that code and then openHAB could be setup to just subscribe to those new Topics, using it’s MQTT Binding, and other standard MQTT tools could also see/alert from those streams (like mqttwarn)

The downside of this is the series of middle-men. I already have the MQTT Server, and BTMon, running on RPi’s so it’s not a hassle to add it there

The alternatives include:

  • A native Binding for openHAB
    This would be fairly simple, and it’s what I wrote for MiOS/Vera a few yrs back. Rules for new Bindings require it be validated against the compat layer of OH2, so it would need a bunch of people to chip in to help validate it there.
  • Use the HTTP Binding, and GEM configured for “URL Format” output, to pull apart the result.
    This can be done today, but it’ll be a whole bunch of REGEX matching to extract all the values. It’s default mode has a binary output, but’s a simpler to parse GEM’s URL Format (which is how I did it for Vera)

@sipvoip Thoughts?

2

I guess this this will be a good excuse to get on the openHAB bandwagon and help test. Still need to buy some hardware (newer Pi or Odroid) and get my GEM working again properly after its WIZnet died.

@strangely

3

Richard,
Do you have a preference on which way you want to run it (from the options above)?

If you’re lucky I might configure you a RPi2 with all the bits by the next time I see you…

4

My preference would be a binding, I see some value in the btmon scripts, and personally I don’t mind that route, but getting rid of middle ware if possible can be a good thing. How much effort is involved in writing a binding without btmon.py?

5

The btmon.py changes should take < 2hrs tops (Python changes will be probably less than 20 lines)

I’ll do that first since it won’t require me to disrupt my existing deployment.

In Java, and openHAB, it’ll be a bunch longer since I have to both config the GEM and parse the input… All of which will require me to take my GEM out of its existing config… Then there’s the OH1 AND OH2 certification before it can be vetted for acceptance to OH1.

Anyhow will try the MQTT route first, since it’ll give me an idea of how useful it’ll be. From there we can map any other course needed.

For Richard’s Pi, I’ll preconfigured it with Mosquitto so he could use it out of the gate

6

Thanks for your time on this, I just ordered two GEM units and will start down the MQTT path. Let me know when you get time to add MQTT support.

7

Just ordered two GEM units and 48 CTs! Can’t wait!

8

Wow, that’s a lot of circuits! Did you get the Serial, Wifi or Ethernet interfaces for the GEM units?

My GEM is on a Wifi interface. The ECM-1240 is on Serial and direct attached to a dedicated RPi, on Wifi, that’s running btmon.py and co-ordinating the whole show.

My Garage temperature sensor is attached to the GEM itself… quite interesting how much the internal temp rises when you drive a regular car into the Garage.

9

And just to wind you up, here’s a shot of the Dashboard I built in SEG. It includes the AC, which dominates the values:

AllLoads.png780×938 215 KB

and here’s the same chart, with the AC circuits suppressed so you can see the other channels, including the cycling fridges etc:

NoAC.png759×931 235 KB

@teichsta: You may want to put the Forum/Community Image server (aka AWS S3 Bucket) behind a xxx.openhab.org CNAME’d address. File uploads are currently showing up in forum posts as hyperlinks directly to the native S3 bucket address (community-openhab-org.s3-eu-central-1.amazonaws.com/original/1X/), and links will break if you ever need to do a AWS region failover, or if you wanted to move to another service-provider.

1 Like
10

Ya. new house, lots of separate circuits. I bought ethernet interfaces for both.

11

Nice, does it work with two GEMs? Let me know when you have MQTT working with OpenHab.

12

thanks for the hint @guessed, i will have a look how we could handle that …

13

In my setup, I run two btmon.py processes. Each one runs one config, one for my GEM, uploading to SEG and PvOutput, and one for my ECM-1240 which uploads just to SEG. These processes are very lightweight, and are setup to startup on reboot of the Pi they’re running on.

I did a little prelim rough out work on the btmon.py MQTT extensions this morning, mostly parameters and defaults. This week is fairly busy, so what I don’t get done in the week I’ll push to this weekend.

Currently I’m planning to have a base topic for all the ECM Channels, then I’ll just use the channel name atop that. Once the basics are functioning, and data is correctly arriving in MQTT, I’ll add a name map to let folks configure with a more ‘natural’ topic name.

14

Two steps closer:

2015/09/09 22:43:11 SOCKET: waiting for 14 seconds
2015/09/09 22:43:25 opening socket connection to 192.168.xxx.xxx:2000
2015/09/09 22:43:25 sending request 1 to device 
2015/09/09 22:43:25 SOCKET: sending ^^^APISPK
2015/09/09 22:43:25 waiting for data from device 
2015/09/09 22:43:25 reading 1 of 1 packets
2015/09/09 22:43:25 buffering packet ts:1441863805 sn:010xxxxx
2015/09/09 22:43:25 closing socket connection
2015/09/09 22:43:25 buffer info:
2015/09/09 22:43:25   010xxxxx:   3 of 120 (1441863805)
2015/09/09 22:43:25 processing with MQTTProcessor
2015/09/09 22:43:25 2 buffered packets sn:010xxxxx
2015/09/09 22:43:25 1 calculated packets sn:010xxxxx
2015/09/09 22:43:25 MQTT: len=32, msgs=[{"topic": "/house/energy/mains", "retain": false, "qos": 0, "payload": 0.001}, {"topic": "/house/energy/subpanel", "retain": false, "qos": 0, "payload": 0.001}, {"topic": "/house/energy/010xxxxx_ch3", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch4", "retain": false, "qos": 0, "payload": 0.925125}, {"topic": "/house/energy/010xxxxx_ch5", "retain": false, "qos": 0, "payload": 0.008}, {"topic": "/house/energy/010xxxxx_ch6", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch7", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch8", "retain": false, "qos": 0, "payload": 1.4417083333333331}, {"topic": "/house/energy/010xxxxx_ch9", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch10", "retain": false, "qos": 0, "payload": 0.00020833333333333335}, {"topic": "/house/energy/010xxxxx_ch11", "retain": false, "qos": 0, "payload": 0.004645833333333333}, {"topic": "/house/energy/010xxxxx_ch12", "retain": false, "qos": 0, "payload": 0.022208333333333333}, {"topic": "/house/energy/010xxxxx_ch13", "retain": false, "qos": 0, "payload": 0.0570625}, {"topic": "/house/energy/010xxxxx_ch14", "retain": false, "qos": 0, "payload": -4.1666666666666665e-05}, {"topic": "/house/energy/010xxxxx_ch15", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch16", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch17", "retain": false, "qos": 0, "payload": 0.0021041666666666665}, {"topic": "/house/energy/010xxxxx_ch18", "retain": false, "qos": 0, "payload": 0.003}, {"topic": "/house/energy/010xxxxx_ch19", "retain": false, "qos": 0, "payload": 0.02002083333333333}, {"topic": "/house/energy/010xxxxx_ch20", "retain": false, "qos": 0, "payload": 0.0100625}, {"topic": "/house/energy/010xxxxx_ch21", "retain": false, "qos": 0, "payload": 0.11152083333333333}, {"topic": "/house/energy/010xxxxx_ch22", "retain": false, "qos": 0, "payload": 0.01922916666666667}, {"topic": "/house/energy/010xxxxx_ch23", "retain": false, "qos": 0, "payload": 0.0011041666666666667}, {"topic": "/house/energy/010xxxxx_ch24", "retain": false, "qos": 0, "payload": 0.003}, {"topic": "/house/energy/010xxxxx_ch25", "retain": false, "qos": 0, "payload": 0.00022916666666666666}, {"topic": "/house/energy/010xxxxx_ch26", "retain": false, "qos": 0, "payload": 0.10397916666666668}, {"topic": "/house/energy/010xxxxx_ch27", "retain": false, "qos": 0, "payload": 0.09620833333333333}, {"topic": "/house/energy/010xxxxx_ch28", "retain": false, "qos": 0, "payload": 0.0009375}, {"topic": "/house/energy/010xxxxx_ch29", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch30", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch31", "retain": false, "qos": 0, "payload": -0.0}, {"topic": "/house/energy/010xxxxx_ch32", "retain": false, "qos": 0, "payload": -0.0}]

Next steps are to get the MQTT Publish working correctly, and then authentication schemes (in the following order)

  • Without authentication
  • With username/password auth
  • With cert-auth

Then I’ll go back to getting the device aliases working in a more flexible manner (010xxxxx_ch1 is simply aliased as mains in the output) as well as getting the other channels included (W & Wh, Pulse Counters & Temperature channels)

15

Nice!

16

@sipvoip,
I have this running on my Mac, pulling the data from the [same] GEM unit and it’s correctly pushing it all up to openHAB over MQTT.

It handles sending everything the GEM can emit, notably:

  • 32x Energy channels,
  • 8x Temperature channels
  • 4x Pulse counters
  • Volts

For the Energy channels, it also publishes separate MQTT topics for Power, Total Energy, and Energy used since the last sample.

At the auth-layer it supports no-auth, password-auth and it has TLS support (all courtesy of the underlying paho-mqtt library). I can’t test TLS at the moment, but I’m sure someone will get around to it eventually (my MQTT Broker is internal-only to my Network)

I’ll let it run overnight, and then reach out to the original author (Matthew Wall) to see if he wants to include it formally, or how he wants it published)

The only thing missing, from my original list, is the ability to override, the [parameterized] MQTT Topic hierarchy on a per-Channel basis. Probably not a big deal for most people… and a writeup/wiki page to doc it all.

For reference, my Items have this form:

Number  HouseVolts             "House Volts [%f]" <energy> (GPersist) {mqtt="<[home:/house/energy/volts:state:default]"}
Number  HousePowerSolar        "Solar Power [%f]" <energy> (GPersist) {mqtt="<[home:/house/energy/solar_w:state:default]"}
Number  HouseEnergySolar       "Solar Energy [%f]" <energy> (GPersist) {mqtt="<[home:/house/energy/solar_wh:state:default]"}
Number  HouseDEnergySolar      "Solar DEnergy [%f]" <energy> (GPersist) {mqtt="<[home:/house/energy/solar_dwh:state:default]"}
Number  HouseTemperatureGarage "Garage [%.1f °F]" <temperature> (GTemperature,GTemperatureChart,GMonitorTemperature) {mqtt="<[home:/house/energy/garage_temperature:state:default]"}

and the data in openHAB ends up like:

2015-09-12 19:35:52 - HouseTemperatureGarage state updated to 79.5
2015-09-12 19:35:52 - HousePowerSolar state updated to -0.438
2015-09-12 19:35:52 - HouseEnergySolar state updated to 6854735.733
2015-09-12 19:35:53 - HouseDEnergySolar state updated to -0.002

I have a lot more circuits, this is just a sample. In this case, the values are sampled/sent every 15 seconds.

17

@sipvoip,
It’s been running solidly now for a few days (on the Pi). I had PM’d Matthew on the Brultech forum, but haven’t heard back yet.

I just posted there to see if anyone has a better way to contact him. I’d like to get it vetted by him first, if at all practical.

18

and here’s that discussion on Brultech’s forum:

19

@sipvoip,
Matthew got back to me at the end of last week, and it’s now under review. I’ve placed it here until the review/formal integration is complete:

  • [REDACTED]

and you can see the diffs from the 3.1.1 baseline version here:

  • [REDACTED]

EDIT: The code is now formally part of the mainline:

20

Just got back from a business trip and want to try this, do you have any more info on how your using it with openhab? Can you share more of your items file, I see how to pull the volts and temp, but how do you pull the individual sensors?