I am using the MQTT binding to communicate with a number of Shelly devices (dimmers, switches, rollershutters,…). I also have a power meter from Shelly (3EM) which I read data from through MQTT regarding the current power consumption. This device produces a high number of updates as every change is being sent via MQTT (almost every second times 3 phases).
While everything is working I am somewhat annoyed by the fact that my events.log files have become almost uselsess, because 99,9% of its content are power meter updates.
I am wondering if someone else has solved this or a similar issue. So far I have not had any luck to change this behaviour at the source. Shelly provides a parameter to be set but it does not really reduce the frequency.
I was wondering if I could use a transformation to check the frequency and only pass through some of the updates. But I guess for that I would need some stateful implementation: Now when I call a javascript or something, it will not have any memory of previous updates…
Does anyone know if the MQTT binding does somehow support this (undocumented?) or could think of another trick through transformations or such to throttle the overall amount of Item updates.
That would be a really good idea … but as you’ve already found, we cannot inspect the existing state to decide if there was a change or not.
Off-topic - for the same reasons, the Modbus binding implements this as an internal option, because it is generally a fast-polling service, and can be set to only pass changes through.
A side effect of many updates can be churning away saving persistence records too - this can have real performance impacts, it’s not just a cosmetic nuisance.
I doubt there’ll be any such feature in the MQTT binding anytime soon, you would have to add some internal caching.
That’s your real problem.
But can we deal more gracefully with a rogue device?
Going to guess that’s not quite true, and the logs are changes? Updates-to-same are not usually logged, unless you’ve chosen that.
So assuming it’s a lot of changes like 245.75 volts to 245.74 volts …
You should be able to reduce some churn by rounding to fewer digits in a transformation.
The updates will still take place, but not be logged when no change e.g. 245.7.
To instead treat it as a cosmetic issue, just filter off the offending logs to a separate logfile
There’s a number of posts, like this giant -
But do consider your persistence as well - are you persisting every update unnecessarily, and should configure for every change instead?
Thanks again for your swift and qualified response.
Yes the logs are changes, less digits will unfortunately not help as I am interested in the power usage (in Watts) and these changes are unpredictable (well more or less) and differ a lot between updates.
My best bet seems to be the logging for now, thanks for the link, I will look into that.
Having some experience with log4j from other java projects I am slightly worried about the performance impact of evaluating non-trivial filter patterns, but hopefully my Raspberry will be able to handle that…
I am using rrd4j in this context specifically and I’m only persisting everyMinute.
I configured Shelly 3EM via Generic MQTT thing (not using shelly binding) just exactly because it generated very high traffic. Maybe it could also help you to solve your problem.
I did not map the power channels which changed every second, but just used minute energy counter (WattMinute) changing just once per minute:
