OK, thanks. I’ll do that. I don’t use rrdj4 in any of my widgets. I use influxdb in all of them and I don’t have any issues with these widgets. The problems with rrdj4 occur only when I click the analyzer of any Item.
Is Influxdb configured as your default persistence? As far as I’m aware MainUI only supports generating charts from the configured default. That means either you have configured InfluxDB as your default and it’s being used for all charts, including the ones you are not having problems with, or it’s being used for none of them.
@rlkoshak, My defauls persistence is rrdj4 but I use also InfluxDB to store some of my Items on an hourly basis. I’m not using InfluxDB for my MainUI charts at all. This was my setup also in OH3.4 and I didn’t have any issues with the MainUI charts.
Then I’m confused. what did you mean by
I don’t use rrdj4 in any of my widgets. I use influxdb in all of them and I don’t have any issues with these widgets.
@rlkoshak, sorry if I haven’t been clear enough. I have used InfluxDB only in my own custom widgets, see e.g. my post. These widgets work really nicely and the data is plotted almost immediately.
I’m using rrdj4 only when plotting any Item’s chart by clicking Analyze in the MainUI. In these cases plotting of the chart is very slow.
Well, one difference is rrd4j saves a value once per minute which is a lot more than one value per hour. Have you verified the problem isn’t in the browser processing all of those values?
Is there a difference in how long the calls to the persistence rest API endpoints for each takes.
@rlkoshak yes, but this is only partly true. MainUI Analyzer plots data for the last 24h as a default which means that there are 24x60=1440 data points to be plotted. With InluxDB I plot also yearly values without any problems, i.e. 365x24=8760 points so many more than with rrdj4.
I have used Firefox, Edge and Chrome on different computers and the problem is not related to the browser not to the PC I’m using. I have the same problem also with the OH server (NUC).
Is there a difference in how long the calls to the persistence rest API endpoints for each takes.
I don’t quite understand what you mean here.
As I said in my first post I didn’t have any issues under OH3.4 and most likely not under OH4.0.
As far I remember there are no changes in query logic of persistence services. There were adjustments of persistence config, but this is how data arrives in services, bot how it returns.
I’d first check if there was any change in rrd4j library version as this might be one of destabilizing factors.
Try in karaf/openhab shell la -l|grep rrd or la -s|grep rrd
The rrd4j library is embedded into rrd4j persistence. By checking sources I found that OH <= 3.4.5 used rrd4j in version 3.8.1. OH >= 4.0 uses rrd4j version 3.8.2.
However, looking at changes between these two versions of rrd4j library I see no reasons for change in behavior. Hence rrd4j itself is unlikely to be guilty of performance issues you face.
@splatch OK, thanks. Task manager shows quite high Disk utilization (see attached screen copies) so could this be the culprit?
As far I remember rrd pre-allocate its file upon first launch. This means that it does not grow over time. However - if your system uses virtual memory (so called swap) it might happen that copying of data from memory/to disk and from disk/to memory interferes with processes which try to read from it.
Take a look if your windows installation uses virtual memory and how. Its been a while since I used windows last time, so my knowledge on how it works nowadays might be rusted.
Navigate to Developer Tools → API Explorer → Persistence and query for some data and see how long it takes.
And there have been no relevant changes to rrd4j I know of that would account for this different between OH 3.4 and 4.1, let alone between 4.0 and 4.1. In fact, there was only one PR for rrd4j between 4.0 and 4.1 which fixed a problem with restoreOnStartup which added an additional condition to an if statement, not something that’s going to impact querying performance.
https://github.com/openhab/openhab-addons/pulls?q=is%3Apr+is%3Aclosed+rrd4j
@rlkoshak, OK , now I understand what you mean. I made some tests (see attached screencopies).
In the first case (with rrd4j) there were 1440 data points and with InfluxDB 1467 points. When I tested rrd4j and kept pressing the Execute button the response was immediate but after ~5 trials the response was very slow so it took longer than ~10s.
With InfluxDB the response was always immediate regardless how many times I pressed Execute.
I don’t know what is causing this behaviour.
OK, I accept the fact that there haven’t been relevant changes to rrd4j but something is slowing down rrd4j in my case.
I think this last experiment is almost enough to file an issue. Put rrd4j and org.openhab.core.persistence (I think) into DEBUG or TRACE level logging and run the repeated queries against rrd4j again. See if you can find which parts seem to take longer between runs.
Gather those logs and description of what you did and file an issue on openhab-core or openhab-addons depending on where it appears the slowdown is occurring, which is what you need the logs for.
@rlkoshak, I have now enabled debug level logging by typing in Karaf:
log:set DEBUG org.openhab.persistence.rrd4j
I’m wondering one thing. It seems that rrd4j stores each Item twice, e.g. my Item Nibe_S1255_UlkoT is stored twice
024-01-30 18:42:00.679 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'Nibe_S1255_UlkoT' as value '0.1' with timestamp 1706632918 in rrd4j database (again)
2024-01-30 18:42:00.679 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'Nibe_S1255_UlkoT' as value '0.0' with timestamp 1706632919 in rrd4j database
Same happens with all my Items. Could this be the reason for performance issues? I haven’t filed a bug report yet.
I doubt it. For one, the performance issues you are encountering are on read and you’ve run a query and confirmed that you are not getting bact twice as many values as expected.