I encountered an issue where the trendline for an item is not visualized even though the analyzer functions as expected. I spent some time in getting behind this issue so I want to share the solution for others encountering this.
tl;dr
trendlines need the data to be persisted in fixed time intervals. The default is every minute.
The trendline is using every 60th value to draw the trend (so the default is every hour).
There is an undocumented parameter “trendSampling” that can be used to change it.
Having a custom persistence strategy for e.g. every 10 minutes this parameter needs to be set to 6 to get the same result (every hour).
Having a persistence strategy on every change might causes data that is not usable for the trendlines and results in weird behavior. In my case a temperature item from my radiator does not have 60 changes a day, so nothing gets shown.
long story
The default setup for OH3 comes with a rrd4j persistence. Without any changes it will persist every known item value every minute.
In my setup I installed the influxdb persistence addon (kept from former OH2 installation). It is set in a text based config file. The item persistence strategy for influx is to persist every change.
Installing the influx addon, causes the default rrd4j persistence to be removed completely.
Anyway getting the data series by calling http://[openhabip:port]/rest/persistence/items/[trendItemName] works fine.
The problem is, that the trendline needs to have a fixed interval to be drawn.
(due to the implementation openhab-webui/bundles/org.openhab.ui/web/src/components/widgets/system/oh-trend.vue at main · openhab/openhab-webui · GitHub)
I use the persisted data with grafana and I do not want to record every minute.
Following this post I found during my research, I decided to install rrd4j addintionaly and explicitly in my addons.cfg file.
Afterwards rrd4j needs to be set in settings->persistence as default and then everything is working again. Data is then persisted in influx and rrd4j afterwards.
Having a look at the implementation I noticed an undocumented parameter “trendSampling” that can be used in widget configurations to set the detail level of the drawn line. It defines how many recoded data points get skipped between each point to draw the trend.
Maybe it is not documented, because the behavior can be ambigous due to different persistence strategies.
trendSampling: 10 on the left and none on the right
results in:
Welcome and thanks for posting! I’ve moved this to the Tutorials & Examples: Solutions category.
Note this is a limitation caused by how RRD4J works and is not a general limitation. RRD4J stands for “Round-Robin Database for Java” and it is a specially constructed database to store data for very long periods of time without growing. All rrd4j files remain a fixed size. To achieve this, it requires:
a reading saved at at least a set frequency; the configuration in openHAB requires every minute
as the data ages the older data gets compressed through decimation; ten values in the database get replaced with one value that is the average of the ten
thus over time data from two years ago might only have one value per day but data from yesterday will have at least one value per minute
Because of this, rrd4j requires there to be values actually stored in the database at a regular interval to work correctly. When using rrd4j using any strategy different from once every minute is likely to lead to failures when the data is queried back out of it. There is an rrd4j config file one can use to change how often the data needs to be stored, but I don’t really see any reason to do that unless you want to save more frequently than every minute. You won’t be saving any space on the disk because the actual file doesn’t grow no matter how long you save values to it.
I just want to make sure this is clear for those using rrd4j.
It also persists on change and it does restoreOnStartup.
It shouldn’t. Did you actually see in the logs that it was being uninstalled? If it really is gone that’s a bug. You can have any number of persistence add-ons installed and working at the same time. A common combo is to use rrd4j for charting and mapdb for restoreOnStartup.
I think most of the UI was written expecting rrd4j so that is probably where that assumption/restriction came from. This is a good find.
If you are using addons.cfg that explains why rrd4j was uninstalled. When using addons.cfg, anything not in addons.cfg gets uninstalled.
If you have more than one persistence engine installed, it wold be a good idea to create a .persist file for each one of them. Persistence engines now have default strategies defined and it isn’t well defined what happens when those strategies conflict. For example, if both rrd4j and mapdb are configured with restoreOnStartup (which I believe both are using by default) then most of your Items will be restored twice and which DB it gets restored from last will depend on timing. So if you are like me and have mapdb for restoreOnStartup, you’ll want to define an rrd4j.persist to not restoreOnStartup.
Or it might just not have been covered yet. This seems like something worth documenting. If you navigate to the bottom of the oh-label-card documentation page there is a link that will take you to where you can edit the page and create a PR to add this.
Some of these pages are automatically generated though so I don’t know if it’s as simple as all that.
Thanks for investigating. This is a component I wrote in maybe less than an hour actually so I perhaps regrettably took some shortcuts. Taking every 60th data point for sampling is not ideal, and I believe it ought to be improved and take into account the period (by default 24 hours) and derive a final set of 24 points to feed the trendline, maybe performing some aggregation (like average) to the sampling groups; the period, number of points and aggregation could all be made configurable.
Thanks for your very detailed response
Your hints gave me some more insights to understand the things better!
The uninstalled rrd4j comes then from the textual cfg file. This already existed before updating.
I guess there are lots of setups out there having a textual cfg file from OH2 with a persistence addon set. Maybe it is worth a hint in the 3.0 breaking changes: Release openHAB 3.0.0 · openhab/openhab-distro · GitHubMigration to openHAB 3 | openHAB
I added a pull request to update the documentation for the trendSampling
I’m lucky to have this great component. It does what it is supposed to do. Just drawing a trend.
I don’t know if more configurations are necessary. But it sounds like a cool improvement!
Trying to make the ui compatible with individual persistences in general sounds like a hard try to me.
Maybe it’s more stable to setup an autonomous rrd4j instance that is not touched by the custom persistences. Otherwise there might always be some unwanted side effects.
I know it’s not that simple, but you probably gut the point…
But it’s not a breaking change. It’s not a change at all. Ever since OH 2.0 this is how text configs, in particular addons.cfg, has worked. Once you start using addons.cfg you are stuck with it because anything you do through the UI or the Karaf Console or the REST API will get overridden by what’s defined in addons.cfg at some point.
