Reducing Item updates (and thus log entries) caused by a Modbus TCP energy meter

Setup, Configuration and Use Items & Sitemaps
1

A modbus TCP energy meter is constantly sending the power and energy data.
So the power values are jittering around 1 Watt:

09:17:22.906 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ez_pool_Wirkleistung_L123' changed from 261 W to 260 W
09:17:23.913 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ez_pool_Wirkleistung_L123' changed from 260 W to 261 W
09:17:25.928 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ez_pool_Wirkleistung_L123' changed from 261 W to 260 W
09:17:26.936 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ez_pool_Wirkleistung_L123' changed from 260 W to 261 W

This spams the log and causes unnecessary entries in the persistence data base.
Is it possible to set a threshold value somewhere at item level?
The item should only be updated when the difference to the previous value is |± 10W|

The Modbus Definition is:

// Zaehler Pool
Bridge modbus:tcp:zaehler_4 [ host="192.168.178.176", port=502, id=1, rtuEncoded=false, timeBetweenTransactionsMillis=1000 ]
{
	Bridge poller Wirkleistung [ start=8999, length=2, refresh=1000, type="holding" ] {
		Thing data L123 [ readStart="8999", readValueType="float32" ]
	}
	Bridge poller Wirkenergie_import [ start=7999, length=2, refresh=60000, type="holding" ] {
		Thing data L123_Total [ readStart="7999", readValueType="uint32" ]
	}
}

The Item definition:

// Zaehler Pool
Number:Power ez_pool_Wirkleistung_L123 (gEnergiezaehler) {
	channel="modbus:data:zaehler_4:Wirkleistung:L123:number"
//	[ profile="modbus:gainOffset", gain="0.001", pre-gain-offset="0" ]
}

Number:Energy ez_pool_Wirkenergie_import_L123_Total (gEnergiezaehler) {
	channel="modbus:data:zaehler_4:Wirkenergie_import:L123_Total:number"
//	[ profile="modbus:gainOffset", gain="0.001", pre-gain-offset="0" ]
}
2

You could set a longer refresh value. Doesn’t completely solve the jitter, but you get a lot less transactions in the logs.

3

Hi Mikael,
maybe this was a bad example, there is the same energy meter for a car charging station too. So the starting point detection should be within a couple of seconds. Therefore the accuracy doesn’t need to be Watt precise.
So lets say I want to detect a charging start within 10 seconds I’d need a polling interval of 5 seconds.
And still I have the problem that the database is updated often.
I could get around this by usind a separate rule for the actual value to be persisted but this is also overhead.
Maybe there is an option to do this already at item level.

4

If you don’t need the events.log for that item at all, you could add a regex to log4j2.xml:

<!-- Event log appender -->
...		
<!-- Regex Filter added below -->
<RegexFilter regex=".*ez_pool_Wirkleistung_L123.*" onMatch="DENY" onMismatch="ACCEPT"/>
...
1 Like
5

How about setting the particular item(s) to persist less often?

6

I normally prefer strategy = everyChange
That way no data is “lost” and if there are only 5 changes a days only 5 values are logged.
So if there would be a possibility to have item changes only if a certain absolute value threshold has reached would be the goal.

Right now I could make a combination:

  1. As sihui said block the real item from the log
  2. define the filtered item
  3. write a rule for the filtered item and use the filtered item to be persisted

However, I also see that the CPU load gets high if permanently rules are fired. Right now there are 5 energy meters in the system.

7

You can set persistence per item. That way all your other items are persisted as they are now, and this/these particular items less often.
The way I see it is you lose data if you only persist with a threshold or you persist less often

One possible way would be to disable automatic persistence for the item(s) and run a rule in every change comparing to the previous value. And if the change is over a certain threshold then do something.
Then again the overhead of just persisting every second might be smaller than running the rule every second? If you’re using the default persistence, the values do get averaged over time anyway.

8

Instead of going into rules, consider using profiles which can keep frequent polling on binding side, but reduce amount of updates emitted into system.

Sadly there are very few profiles in core which would allow that so you either have to install them from third party or write them yourself.

9

First, concerns about the number of entries in persistence do not matter for MapDB nor for rrd4j. In both cases, the number of entries into the database is fixed regardless of how often the Item changes.

Combining some of the other suggestions made here already, you could use the SCRIPT transformation with the GraalVM JS language for the transformation. In this case (probably others too) the help library also gets injected into the transformation meaning everything you have access to in rules will be available in the transformation. So you can pull the current state of the Item and only if the new value is different enough return the new value.

Or you could just round the value to the nearest tenth or integer or whatever. The Item will still get updated on every reading but it won’t change nearly as much.

10

I guess there is a simpler solution for every use case of each energy counter.
I have 3 modbus energy counters for different purposes and 2 wallboxes, one connected via modbus, the other via binding. Went through all these persistence and logging trouble and it took quite a long time to optimize which registers to poll at what interval. I guess there is still room for optimizing but as CPU load is quite low and the whole system is very responsive there is no need at the moment.

For wallboxes there are no charging sessions below 6A. So, instead of polling the “W” registers you could poll the “A” registers, depending on how precise they are.
Stop persisting the current W, instead persist the TotalEnergyCounter every 30 or 60 seconds, depending on what statistics you have in mind.

There might be many more solutions by thinking “different”.

11

Hi Lukasz,
yes a profile would be the best solution like the modbus gainOffset profile.

12

This is actual for a customer of mine and he wants to see that data (Watt + Energy) and get a report with start, end and amount of energy.
Also he expects somehow an immediate feedback once his car is charging so I need to have a decent sample period.
When only changes is used in the persistence strategy there is already a high compression without loosing any data.
Requirement: No jitter.

For the databases I use mariadb (historic reason and as there are a lot of tools being able to process the data).

13

Quite the same here. Only difference is that I am my own customer.

Does the charging station have a LAN interface? What about manually persisting the item when you discover a relevant change in value?

Just to show you the way. Each of my two wallboxes has a RFID reader to authenticate a charging session. Both are connected to openHAB via LAN. So if an authentication is detected the status change of the wallbox is persisted. This is the start time of the charging process. When the status of the wallbox changes again this is the end of the charging session and the status change is persisted again.
That way I have start and end of every charging session ever authenticated. If you manually persist the energy counter value at start and end the difference of (or delta between) both values is the amount of energy.
If the energy circuit behind the energy counter is only the charging station the amps should be constant below 1 while no charging session is active. You could use a read transformation which sets values below 1 A to zero so that there is no value change for openHAB.
I configured my modbus poller to poll every 30 seconds which, in my case, is more than enough to create an accurate “bill” for a charging session. In fact it is a little more complicated as I use solar power which results in a couple of status changes during a charging session as well as more than a couple of changes in charging power.

14

@marco_hoefle I’ve implemented about two years ago two profiles for my own use:

I do not publish these, but it you would like to test it I can make a build with only these profiles for 3.3 or 3.4.

15

Using a profile also runs a piece of code somewhere so the load is similar.

16

I’m having the same issue. My meter is providing the power in 0.001W steps and therefore I have a lot of item updates every couple seconds. An absolute or relative delta value would be good to decide if the item gets updated or not. Is a profile like this available?

@ssalonen What would be your proposal? I think the modbus profile can not handle this?

17

OH 4 has support for a threshold filter for persistence configs. Persistence | openHAB This will only persist a value if it is above a threshold delta. If the value is too close to the last persisted value, it doesn’t persist the value.

To keep the Item from updating in the first place, that either needs to be implemented in the binding or in a profile. There is presently not an official profile that implements this that comes with OH out of the box. But it would be pretty simple to implement using a Script profile.

(function(i) {
  const lastValue = cache.private.get('last', () => -999999); // initialize it so if there isn't a lastValue the delta is large
  const delta = Math.abs(lastValue - parseFloat(input));
  if(delta > 1) {
    cache.private.put(parseFloat(input));
    return input; // pick a reasonable threshold
  }
  return null; // if the delta isn't large enough return null, suppressing the event
})(intput)

Edit: I forgot to save the last value to cache.

2 Likes
18

I added in OH4 your script under Transformation. Then I have assigned it in “Thing To Item Transformation” (SCRIPT ECMAScript - Profile Configuration), but I get this error. Do you have any idea?

org.graalvm.polyglot.PolyglotException: ReferenceError: “intput” is not defined

at <js>.:program(<eval>:9) ~[?:?]

at org.graalvm.polyglot.Context.eval(Context.java:399) ~[?:?]

at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.eval(GraalJSScriptEngine.java:458) ~[?:?]

at com.oracle.truffle.js.scriptengine.GraalJSScriptEngine.eval(GraalJSScriptEngine.java:426) ~[?:?]

at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:262) ~[java.scripting:?]
19

Typo. Should be “input”

20

Ok, this is fixed. Now I have:

org.graalvm.polyglot.PolyglotException: TypeError: invokeMember (put) on org.openhab.core.automation.module.script.rulesupport.internal.CacheScriptExtension$ValueCacheImpl@727160 failed due to: Cannot convert '15.90567'(language: Java, type: java.lang.Double) to Java type 'java.lang.String': Invalid or lossy primitive coercion.
	at <js>.put(@openhab-globals.js:2) ~[?:?]

	at <js>.:anonymous(<eval>:5) ~[?:?]

	at <js>.:program(<eval>:1) ~[?:?]

	at org.graalvm.polyglot.Context.eval(Context.java:399) ~[?:?]