New Binding: Wireless M-Bus / Techem heat cost allocators

Hi everybody,
I just wanted to let you know that I am currently working on a new openHAB2/Eclipse Smart Home binding to receive the messages sent by Techem heat cost allocators vario 4 (Heizkostenverteiler) via Wireless Metering Bus (WMBus).
You can find it here:

There has been some interest and even some effort on this earlier:

Those links all refer to openHAB 1, also the pull request was never incorporated. My effort is a new approach without using the code from the old PR.
What I am using though, is the library jMBus by the openMUC project of the Fraunhofer Institute for Solar Energy Systems (ISE) in Freiburg, it is included in the binding, with a small patch, which I submitted to them recently and hope that it will be included upstream. It in turn uses the RXTX library, which is therefore also included in the binding, together with the native parts for Windows and Linux. It seems, one needs to install librxtx-java via packet manager as well.

I use an Amber Wireless AMB8425-M USB stick as receiver, which is supported by the library. It is available for example at http://www.mysmartshop.de/amber-wireless-m-bus-usb-stick-amb8465-m.html. It should be possible to use a CUL (which I donā€™t have yet) as well, thatā€™s what the FHEM people use, to whom most of the credits on decoding the messages go to here: Wireless M-Bus fĆ¼r CUL, WMBUS - TECHEM, war: Techem HKV (ok) -> war WasserzƤhler (ok) -> war WƤrmemengenzƤhler (ok), Testversion: 32_TechemWZ, Rauchmelder von Techem, WMBUS ā€“ FHEMWiki, TechemHKV ā€“ FHEMWiki, FHEM reference

There was some earlier and less deep reverse engineering approach, which is still quite impressive and interesting:

There is also a privacy issue here: All devices transmit their data openly very regularly (1-10 minutes), and they have quite some range, so you can easily read the temperature of all your neighbours rooms. You donā€™t, however know, which device is in which room of which neighbour and it is questionable, how much information you are able to deduce from the readings, but still it would be much better if the AES encryption defined in the standard would be used. This, however would stop us from reading our own meters as wellā€¦

The binding is not yet working completely, only the ā€œfirst stepā€: You can manually add a new bridge-type thing, representing the receiver stick, fill in the device name (like /dev/ttyUSB0) and henceforth see the messages received in the console. Also, for every message, a new DiscoveryResult will be created in the Inbox, representing the different heat cost allocators. openHAB updates results of the same allocator instead of creating a new one. This way I have around 70 results, 6 of which are ours and the others most probably from the neighbours in the flats around us.
You can also add those results as things and they show up in the control section, but the data from the messages is not filled in yet, I am still working on this.

A received message (in decoded form) looks like this:
Sat Nov 19 15:02:02 CET 2016;-62;68;TCH;92313926;105;RESERVED;160;11;2016-06-30;208;2016-11-19;7;16.46;16.48;000600060100000000000000000000000f00161a271d2f0a01000b

receivingDateTime;RSSI;controlField;vendor;deviceID;version;deviceType;controlInformation;status;lastDate;lastValue;currentDate;currentValue;temperature1;temperature2;almanac

The vendors can be looked up here: http://dlms.com/organization/flagmanufacturesids/index.html. The deviceID is the one also shown in the display and listed on the heating bill. The lastDate is the end date of the last metering period, lastValue is the corresponding meter value (which is also shown on the display), currentDate is ā€¦ well the current date and currentValue the current metering value, (also shown on the display). temperature1 and 2 are the room temperature and that of the radiator, the difference is used to calculate the metering value internally. Lastly, the almanac seems to carry bi-wekly information on the last heating period, so that you could deduce, how much heat was used in the separate bi-weeks. There is a patent on the compression for this, meaning that the complete algorithm is open: EP About this file - European Patent Register / EPO - European Publication Server, I didnā€™t bother to try to understand the mathematics of this and implement it, because as soon as you add the things to openHAB, it will have an even higher resolution. But implementing this might be interesting for academic reasons or to evaluate the last metering period before introduction to openHAB.

The most valuable information is probably the metering value (which is only set daily) and the temperature (which is sent live). This way, you have exact measurements of all of your rooms free of charge, IF you live in a rented flat, have those devices and they are set to transmit. They can be set to be quiet, which is cheaper for the landlord, I guess, in this case they do not show a radio tower in the display.

So, feel free to try it out or have a look into the code, I am open to questions, suggestions and remarks. It is totally in the scope of this binding, to later also read other WMBus devices like the EnergyCam or other Techem devices like water or heat meters or smoke detectors, which are currently rolled out to many german homes because of a legal requirement and use other receivers like the CUL or Amazon.de or Wireless M-Bus USB Stick iM871A | Smart Meter fĆ¼r HomeMatic or USB Meter Reader - Stick - wM-Bus - WebShop Kernbauer Energy Solutions GmbH, but currently I have neither of those devices, so I canā€™t do it, it would therefore be a second or third step and I would need help from others.

Until the binding is mature enough, I plan to leave it in the custom Java package namespace and as a single binding repo. But as soon as it works reliably, I would like to move the package and do a pull request against the official repo.

So, have fun and let me know, what you think and if you are interested! :slight_smile:

Cheers,
Hanno

3 Likes

Hi,
I just wanted to let you know, that the binding is now operational (though beta). It works for me in our flat, drawing beautiful graphs:

Sender
EmpfƤnger
Control
Diagramm

It is available as source and compiled, ready-to-drop-in extension at github.
(thanks to @hmerk and @SuperOok at this thread)

Is anyone interested? :slight_smile:
Cheers,
Hanno

1 Like

Hi,
very good job. Iā€™m using the same dongle (Amber).
I want to decode the message received from FAST FORWARD AG EnergyCam.
Can you give the TechemHKVMessage source, so I can study it and create the same for my device?

Thanks

Hey,
glad somebodyā€™s interested :wink:

My fork of jMBus with the small alterations and the TechemHKVMessage is at https://github.com/pokerazor/jmbus

I will be very interested, if you can get it to work with the EnergyCam!

Cheers

Hi all,

is there any activity going on for a OH2 M-Bus binding? If not - would it be the correct 1st step to create an issue to add M-Bus support?

thanks & kind regards,
Patrik

Hi Patrik,
My OH2/ESH WMBus/Techem binding should work as a base to implement generic M-Bus. Just try, if it already shows the M-Bus messages. I would be very happy to work together with you to implement a more generic solution!
We can discuss this next week. :slight_smile:

Hi @friesenkiwi,

thank you for your reply :slight_smile:
I should receive the required HW (opto & cabling) this week; then Iā€™ll give it a try. Unfortunately my knowledge of java and the used build system is very limited. However - Iā€™ll try my best to help to evolve your binding to a generic M-BUS one.

As soon as I have some results (or questions :wink: ) Iā€™ll get back to you again.

with kind regards,
Patrik

Hi @friesenkiwi,

I installed the HW & installed the binary version of your binding; but Iā€™m not able to start it yet - probably a missing prereq on my system:

Error executing command: Error executing command on bundles:
        Error starting bundle 228: Could not resolve module: de.unidue.stud.sehawagn.openhab.binding.wmbus [228]
  Unresolved requirement: Require-Capability: osgi.native; native.paths.8:List<String>="lib/rxtx/Windows/win32/rxtxSer
ial.dll"; native.paths.7:List<String>="lib/rxtx/Mac/mac64/librxtxSerial.dylib"; native.paths.9:List<String>="lib/rxtx/
Windows/win64/rxtxSerial.dll"; native.paths.0:List<String>="lib/rxtx/Linux/arm/librxtxSerial-2.2pre1-armhf.so"; native
.paths.2:List<String>="lib/rxtx/Linux/linux64/librxtxSerial.so"; native.paths.1:List<String>="lib/rxtx/Linux/linux32/l
ibrxtxSerial.so"; native.paths.4:List<String>="lib/rxtx/Linux/sh4/librxtxSerial.so"; native.paths.3:List<String>="lib/
rxtx/Linux/linux64/librxtxSerial.so"; native.paths.6:List<String>="lib/rxtx/Mac/mac64/librxtxSerial.dylib"; native.pat
hs.5:List<String>="lib/rxtx/Linux/mips/librxtxSerial.so"; filter:="(|(&(osgi.native.osname~=linux)(osgi.native.process
or~=arm))(&(osgi.native.osname~=linux)(osgi.native.processor~=x86))(&(osgi.native.osname~=linux)(osgi.native.processor
~=x86_64))(&(osgi.native.osname~=linux)(osgi.native.processor~=amd64))(&(osgi.native.osname~=linux)(osgi.native.proces
sor~=sh4))(&(osgi.native.osname~=linux)(osgi.native.processor~=mips))(&(osgi.native.osname~=MacOSX)(osgi.native.proces
sor~=x86_64))(&(osgi.native.osname~=Mac OS X)(osgi.native.processor~=x86_64))(&(|(osgi.native.osname~=Windows)(osgi.na
tive.osname~=Windows 7)(osgi.native.osname~=Windows7)(osgi.native.osname~=Windows Vista)(osgi.native.osname~=Windows X
P)(osgi.native.osname~=Windows NT))(osgi.native.processor~=x86))(&(|(osgi.native.osname~=Windows)(osgi.native.osname~=
Windows 7)(osgi.native.osname~=Windows7)(osgi.native.osname~=Windows Vista)(osgi.native.osname~=Windows XP)(osgi.nativ
e.osname~=Windows NT)(osgi.native.osname~=Windows Server2008 R2))(|(osgi.native.processor~=amd64)(osgi.native.processo
r~=x86_64))))"

Iā€™ll try to figure out what is missing - do you have any hints? Would it be possible to make the binding self-contained if it is a prereq?

Thanks for your work!

with kind regards,
Patrik

Hi @friesenkiwi,

Iā€™ve installed RXTX (.jar & .dll) into my jre; but still see the issue. To be safe I installed V$ 2008 & V$ 2010 redist packages as well. But still no luck. I do not know if the binding can not find the .dll, or if a dependency is missing. Dependency-Walker reports some missing .dllā€™s - but I do not know whatā€™s missing nor if that is the issue. Iā€™ll continue to dig (OS is M$-Windows 10 x86):

with kind regards,
Patrik

Hi Patrick,
I had only tested this binding under Linux, so I did not run into the problem before.
I just made a little fix in the MANIFEST.MF, which should now allow to resolve the package, because the rxtx library is included in the jar, but was not annotated for Windows 10.
Can you try https://github.com/pokerazor/openhab-binding-wmbus/releases/tag/v0.1.1 ?
Cheers,
Hanno

Hi Hanno (@friesenkiwi),

thank you for the new binding. Iā€™ve updated it - but still see the following error:

267 | Installed |  80 | 2.0.0.201706190807     | WMBus Binding
openhab> start 267
Error executing command: Error executing command on bundles:
        Error starting bundle 267: Could not resolve module: de.unidue.stud.sehawagn.openhab.binding.wmbus [267]
  Unresolved requirement: Require-Capability: osgi.native; native.paths.8:List<String>="lib/rxtx/Windows/win32/rxtxSer
ial.dll"; native.paths.7:List<String>="lib/rxtx/Mac/mac64/librxtxSerial.dylib"; native.paths.9:List<String>="lib/rxtx/
Windows/win64/rxtxSerial.dll"; native.paths.0:List<String>="lib/rxtx/Linux/arm/librxtxSerial-2.2pre1-armhf.so"; native
.paths.2:List<String>="lib/rxtx/Linux/linux64/librxtxSerial.so"; native.paths.1:List<String>="lib/rxtx/Linux/linux32/l
ibrxtxSerial.so"; native.paths.4:List<String>="lib/rxtx/Linux/sh4/librxtxSerial.so"; native.paths.3:List<String>="lib/
rxtx/Linux/linux64/librxtxSerial.so"; native.paths.6:List<String>="lib/rxtx/Mac/mac64/librxtxSerial.dylib"; native.pat
hs.5:List<String>="lib/rxtx/Linux/mips/librxtxSerial.so"; filter:="(|(&(osgi.native.osname~=linux)(osgi.native.process
or~=arm))(&(osgi.native.osname~=linux)(osgi.native.processor~=x86))(&(osgi.native.osname~=linux)(osgi.native.processor
~=x86_64))(&(osgi.native.osname~=linux)(osgi.native.processor~=amd64))(&(osgi.native.osname~=linux)(osgi.native.proces
sor~=sh4))(&(osgi.native.osname~=linux)(osgi.native.processor~=mips))(&(osgi.native.osname~=MacOSX)(osgi.native.proces
sor~=x86_64))(&(osgi.native.osname~=Mac OS X)(osgi.native.processor~=x86_64))(&(|(osgi.native.osname~=Windows)(osgi.na
tive.osname~=Windows 7)(osgi.native.osname~=Windows7)(osgi.native.osname~=Windows Vista)(osgi.native.osname~=Windows X
P)(osgi.native.osname~=Windows NT))(osgi.native.processor~=x86))(&(|(osgi.native.osname~=Windows 10)(osgi.native.osnam
e~=Win10)(osgi.native.osname~=Win32)(osgi.native.osname~=Windows)(osgi.native.osname~=Windows 7)(osgi.native.osname~=W
indows7)(osgi.native.osname~=Windows Vista)(osgi.native.osname~=Windows XP)(osgi.native.osname~=Windows NT)(osgi.nativ
e.osname~=Windows Server2008 R2))(|(osgi.native.processor~=amd64)(osgi.native.processor~=x86_64))))"

Please let me know if I can help with additional information & if additional steps are required.
Many thanks for your work & help :slight_smile: ,
Patrik

Well, I have to thank YOU that you are willing to test and help develop the binding!

Unfortunately it seems, the combinations given in MANIFEST.MF (https://github.com/pokerazor/openhab-binding-wmbus/blob/v0.1.1/src/de.unidue.stud.sehawagn.openhab.binding.wmbus/META-INF/MANIFEST.MF) still doesnā€™t match your system configuration.

Can you do the following:

  1. Download https://osgi.org/download/properties.jar and place it into the OpenHAB2 addons folder
  2. Start the Bundle
  3. It should output some lines, of which os.name, os.arch, org.osgi.framework.os.name, org.osgi.framework.os.version and org.osgi.framework.processor should be the important/interesting
  4. Post the lines here or, even better, directly try to edit the MANIFEST.MF inside the binding jar accordingly (you can just open it with an archiver or rename it to .zip).

The RXTX dependency is a little bit clumsy and I would like to get rid of it sooner or later, but I am not sure, whether or how the jMBus library works out of the box with the nrserial delivered by OpenHAB. If anybody can give me a hint, I am more than happy to remove that dependency.

Sooo, it took me a while and a bit of hair, fighting the way through dependency and build hell, but now the included RXTX is gone and instead openHABā€™s nrjavaserial is used. It seems to be working now, at least for me and so I hope it will also work for you on Windows
Please try:

Hi Hanno (@friesenkiwi),

thanks - this version of the binding starts & is able to connect to the serial port :slight_smile: - nice work!
Unfortunately it is not able to see any devices yet - but this is not completely unexpected. Iā€™ve read some stuff to prepare myself to be of use and learned that optical interfaces (like I use) only talk after they have received a special wake-up sequence.

The test tool I use to verify the communication has a flag to enable "ZVEI optical mode). I also found something mentioned in another forum: http://www.sonsivri.to/forum/index.php?topic=60412.0

Iā€™ll try if I can capture the serial communication of the test tool to find what sequence it uses. I assume for those type a polling configuration would be required that 1st sends this command & then queries the sensor(s). If I understand correctly you listen directly to the M-Bus traffic - thus I fear that would be a bigger change.

with kind regards,
Patrik

@friesenkiwi

attached the serial log from the test tool (Iā€™ve changed the extension to .xml to be able to upload):

m-bus.log.xml (11.6 KB)

I hope it is of use.

with kind regards,
Patrik

@friesenkiwi

ā€¦ and in hex format:m-bus-HEX.log.xml (11.5 KB)

Glad it is running and connecting! :slight_smile:

Huh, this sounds like it may become a bit of work and fiddelingā€¦
I have to admit I am not very familiar with programming so close to hardware, bytes and serial interfaces :wink:
Also, because I donā€™t have the hardware, I am not sure, how much I can help.
Judging from https://www.openmuc.org/m-bus/user-guide/#_wired_m_bus_2 the general handling of wired M-Bus might be a bit different to WMBus with the Amber Stick like done until now.
I suspect, one could start with adding a new de.unidue.stud.sehawagn.openhab.binding.wmbus.handler.MBusBridgeHandler and the related Listener etc. classes.

What is your setup? What hardware do you use?

Hanno (@friesenkiwi),
I use this: http://www.relay.de/en/produkte/m-bus-accessories/optokopf.html optical head to connect to the IR interface of my NeoVac Supercal 539 (https://www.hk-gebaeudetechnik.ch/typo3temp/processed/1/b/csm_pr-ews_1405_20_NeoVac_WZ_Supercal749_8b042a0e63.jpg) which has an optical interface and acts as master (several devices are connected to it.

ā€œNormalā€ serial M-Mus test software can not read the device; even if the requests look the same. A Wakeup sequence is required 1st; but I guess after the interface is alive it acts as/uses M-BUS message formats (as SW that does not support the optical wakeup sends the same bytes to query the device as the one that does). But no broadcasting from the device - the SW needs to query/poll the value(s). Do not know if that is possible with your architecture?

I can try to figure out more about this additional communication element involved in optical <=> M-BUS & maybe the library you use already as build in support for it (jmbus) - Iā€™ll try to read through the doc. But the other more important question IMHO can you support to poll/request info @ a certain interval? As those devices (for power save seasons) only use their optical interface on specific request.

I hope the above makes sense. Iā€™ll read me into the jmbus docu & get back if I find something that might help. Do you think a poll/query mechanism (if not yet present) is within scope of this binding?

with kind regards,
Patrik

OK, sounds interesting.
Yes, a query/poll mechanism is definitely within the scope of this Binding! The WMBus/TechemHKV part was actually really quite easy, because you only had to wait and listen passively, but I see no reason, why it shouldnā€™t be possible to add active communication. Everything is built extendably and I would think it is the right place to add it. jmbus certainly supports this mode of operation.
I updated the sources on github to the latest changes, so probably it would be a good idea for you to set up the build of the binding and my fork of jmbus before continuing. It shouldnā€™t be too complicated. The binding is built through Maven inside an OpenHAB-SDK and jmbus via Gradle in a separate Vanilla Eclipse.
Happy to help!

well :slight_smile: ā€¦ Iā€™ll try my best - but it will take some time for me to read the respective docu & Iā€™m sure Iā€™ll need some help later on; as Iā€™ve no experience in that aerea. Iā€™ll get back as soon as Iā€™ve questions, or a working setup.

with kind regards,
Patrik