Openhab3.3 rrd4j and counter

Scriptwriter · September 5, 2022, 7:52am

i have some counters which im trying to save and graph. the thing where im getting stuck is that its saving the values to the item, but the graph is every time 0

what i have done:
cat persistence/rrd4j.persist


Strategies {
    // for rrd charts, we need a cron strategy
    everyMinute : "0 * * * * ?"
}

Items {

    KWH* : strategy = everyMinute 
    ws_regen : strategy = everyMinute  
}

cat services/rrd4j.cfg

# configure specific rrd properties for given items in this file.
# please refer to the documentation available at
# https://www.openhab.org/addons/persistence/rrd4j/
#
# default_numeric and default_other are internally defined defnames and are used as
# defaults when no other defname applies

#<dsName>.def=[ABSOLUTE|COUNTER|DERIVE|GAUGE],<heartBeat>,[<minValue>|U],[<maxValue>|U],<sampleInterval>
#<dsName>.archives=[AVERAGE|MIN|MAX|LAST|FIRST|TOTAL],<xff>,<samplesPerBox>,<boxCount>
#<dsName>.items=<comma separated list of items for this dsName>

#cntr24h.def=COUNTER,1800,0,U,3600
#cntr24h.archives=TOTAL,0.5,1,168:TOTAL,0.5,24,730:TOTAL,0.5,7,520

ctr24h.def=COUNTER,1800,0,U,60
ctr24h.archives=AVERAGE,0.5,1,480:AVERAGE,0.5,10,144
ctr24h.items=ws_regen,SmartMeter_180,SmartMeter_181,SmartMeter_182

The smartmeteritems will change about 10 times per day or more, the ws_regen only some times if its raining.

SmartMeter_180,SmartMeter_181,SmartMeter_182 are member of group KWH

I read some casesabout graphing counters and also the documentation but im stuck. The thing im trying to avoid is to make items for each smartmeteritem per hour/per day per month because im thinking this should be possible by accumulated graph then if the rrd would have the data correctly in there.

if im trying to ask the rest then im seeing only 0 values in there:

{
  "name": "SmartMeter_180",
  "datapoints": "12",
  "data": [
    {
      "time": 1662321600000,
      "state": "0"
    },
    {
      "time": 1662325200000,
      "state": "0"
    },
    {
      "time": 1662328800000,
      "state": "0"
    },
....

i have rrd4j in debug mode and if im looking for the value, then im seeing its changing:

....
2022-09-05 08:05:00.896 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.51604E8' in rrd4j database (again)
2022-09-05 08:05:00.897 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database
2022-09-05 08:06:00.827 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database (again)
2022-09-05 08:06:00.828 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database
2022-09-05 08:07:00.830 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database (again)
2022-09-05 08:07:00.831 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database
2022-09-05 08:08:00.853 [DEBUG] [d4j.internal.RRD4jPersistenceService] - Stored 'SmartMeter_180' as value '2.516076E8' in rrd4j database (again)
....

Is there anyone which can help me with that to give me a hint whats wrong?

Best thanks and regards

Andreas

opus · September 5, 2022, 10:03am

What kind of items are you using?
What are the decimaö values of the item states? I have never seen a value like “2.51604E8” in the logs(the rrd4j inspector however does show the values like that).

Scriptwriter · September 5, 2022, 11:13am

the item is from type Number:Energy. The Value if im Clicking oon the Item is like that: “69892 kWh” The Value is only in debugmode shown like that.
The itemvalue is generated from official smartmeter-plugin which is filling that item from serial connection via infrared from a smartmeter
in the item i have in state-description a pattern “%.0f kWh”

opus · September 5, 2022, 11:29am

COUNTER! I missed that one in your OP.

Upfront:I have never used a counter!

When using a COUNTER rrd4j will store the difference to last value only. For me that explains the zeros in your .rrd file.
I think (!) the standard charts of MainUI can’t worl with COUNTERs.

In cases were I need consumption data since last measurement I store the last measurement, calculate the difference, postUpdate to a seperate item and store the actual value as a new last value. That way the standard GAUGE is working (for me).

Scriptwriter · September 5, 2022, 11:35am

COUNTERs should make this possible what you are doing with rules out of the box, because its a database. Therefore COUNTER were founded. From Monitoringsystems like nagios im knowing this showld work.
0 is ok if the counter is not growing. but it is growing. so sometimes there should be a 1 in there.

I guess because COUNTER were also described in manual of openhab 3.3 im only just a small bit away from a solution. im hoping someone can give me a hint for that…

opus · September 5, 2022, 11:50am

As for your REST printout:
What timeframe didyou ask for? The default is 24 houres!
The timesteps look like 3600000, which I don’t see in your .cfg.

Scriptwriter · September 5, 2022, 5:28pm

…ip uploaded a screenshot to better see the values as graph. on the rest request i did a full day, yes, but copied only a short part to show.
Also after each change in the rrd4j.cfg i deleted the existing rrd that new one was generated (with other size if im changed something i ds-config)

what i also cann tell you with the default gauge it shows the values, but then i cannot graph the differences in aggregated graph

opus · September 6, 2022, 8:43am

That is understood. However I wonder why the REST output shows that timesteps. Your posted .cfg should make a 10 minute timesteps (i.e. 600 seconds) when calling for 24 houres.

Scriptwriter · September 6, 2022, 8:47am

…the rest shows timestamps also then if im entering policy “everychange” and they are not from a change.
google told me that there is already a bug open that counters are not working
https://github.com/openhab/openhab-addons/issues/11924

so maybe its senceless this time to use COUNTER…

opus · September 6, 2022, 10:43am

For rrd4j this is not the case, because this db ENSURES all values in an archive are stored with the same timespacing.

Note that a changed rrd4j.cfg is only assured to be used (at least for me) when stopping openHAB, deleting the old .rrd files and restart openHAB again.
A solid check if the .rrd file is using the correct .cfg is possible with the RRDInspector.

opus · September 7, 2022, 7:02am

I did some tests with an item using the above posted rrd4j.cfg:

Used rrd4j.cfg:

Counter24hAVERAGE.def=COUNTER,1800,0,U,60
Counter24hAVERAGE.archives=AVERAGE,0.5,1,480:AVERAGE,0.5,10,144
Counter24hAVERAGE.items=Systeminfo_EmpfangeneDatenAVERAGE,Systeminfo_GesendetePaketeAVERAGE

Used Strategy: everyMinute

Using those settings my .rrd files have 6 KB (480 values in archive 1 and 144 in archive 2).

A REST output with the default 24 houre setting returns:

{
  "name": "Systeminfo_EmpfangeneDatenAVERAGE",
  "datapoints": "74",
  "data": [
    {
      "time": 1662487200000,
      "state": "1.4"
    },
    {
      "time": 1662487800000,
      "state": "0.1"
    },
    {
      "time": 1662488400000,
      "state": "0.0"
    },
    {
      "time": 1662489000000,
      "state": "0.0"
    },

Note that the system did not run for the whole 24 houres, hence there are only 74 values in this archive (yet) instead of 144.
Also note the timedifference is 600000 msec (i.e.10 minutes) as set in the .cfg for the second archive).

The MainUI Graph for this item looks like:

The item is linked to the ReceivedData[MB] of SystemInfo.

Looking at those findings I’d say a COUNTER is working as expected and (although differently stated above) the MainUI Chart is working with a COUNTER archive. Only the difference since the last measurment is logged (everyMinute), since we are looking at a whole day, the archive 2 is presented, which shows the average of 10 minutes.

In relation to problem of @Scriptwriter:

-The posted parts of a REST output are NOT from an .rrd file using the above posted .cfg. The shown REST entries are from 2022-09-04 22:00, …23:00 and 2022-09-05 00:00. The shown log entries are from 2022-09-05 08:05:00.896 and later.
The logged values of “2.51604E8”, etc. point ot a different .cfg as well (I’d say a GAUGE).
The lateron posted graph is from a different item.

-the created .cfg is using AVERAGE as a consolidation function, that way the aggregated values in archive 2 will be computed from the average of 10 values. That way an

would not show the desired accumulated values.
This cfg does store data for 24 houres only by the use of 2 archives holding 624 values. Why not use a single archive for the whole 24 houres ( that would by 1440 values), that way no real consolidation will be done (one needs to define a define for archive 1, the “usual” setting of 1 as samplesPerBox will render it unused).

Scriptwriter · September 7, 2022, 7:51am

Hi Juergen,
ok, i just entered exact your config with my itemnames. First step lets collect on my site some values and we will see if i will get some values in there than zero.
If this would work then it could be a second step to get more values in there for making akkumulated graphs. you made me some hope that it is possible to get it also if this bugcase i referenced is open from another person. We will see it in a view hours…

What im not sure is where there could be another different config with Gauge…

Scriptwriter · September 7, 2022, 9:05am

ok, i have entered me a really new item which i can count up myself as i like it. the result:
cat events.log|grep “‘kwhtest’”

2022-09-07 10:44:54.131 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'kwhtest' received command 1
2022-09-07 10:44:54.136 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'kwhtest' changed from NULL to 1
2022-09-07 10:44:58.679 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'kwhtest' received command 2
2022-09-07 10:44:58.684 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'kwhtest' changed from 1 to 2
2022-09-07 10:51:10.710 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'kwhtest' received command 3
2022-09-07 10:51:10.713 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'kwhtest' changed from 2 to 3

REST: /rest/persistence/items/kwhtest?serviceId=rrd4j&starttime=2022-09-07T00%3A00%3A00&endtime=2022-09-07T11%3A00%3A00
{"name":"kwhtest","datapoints":"15","data":[{"time":1662539820000,"state":"1.0"},{"time":1662539880000,"state":"1.0"},{"time":1662539940000,"state":"1.0"},{"time":1662540000000,"state":"1.0"},{"time":1662540060000,"state":"1.0"},{"time":1662540120000,"state":"1.0"},{"time":1662540180000,"state":"1.0"},{"time":1662540240000,"state":"1.0"},{"time":1662540300000,"state":"1.3280701754385964"},{"time":1662540360000,"state":"2.968421052631579"},{"time":1662540420000,"state":"2.968421052631579"},{"time":1662540480000,"state":"2.968421052631579"},{"time":1662540540000,"state":"2.968421052631579"},{"time":1662540600000,"state":"2.968421052631579"},{"time":1662540660000,"state":"2.968421052631579"}]}

aggregated graph:

config:
  label: kwhtest
  chartType: day
slots:
  grid:
    - component: oh-chart-grid
      config: {}
  xAxis:
    - component: oh-time-axis
      config:
        gridIndex: 0
  yAxis:
    - component: oh-value-axis
      config:
        gridIndex: 0
  series:
    - component: oh-aggregate-series
      config:
        name: kwhtest
        gridIndex: 0
        xAxisIndex: 0
        yAxisIndex: 0
        type: line
        item: kwhtest
        dimension1: minute
        aggregationFunction: diff_first
  legend:
    - component: oh-chart-legend
      config:
        show: true
        orient: horizontal

aggregated graph as result of this

awaited result of aggregate per minute:
graph should show 10:44 peak 2
10:45-50 peak 0
10:46 peak 1

Info: i have changed Code of aggregated Chart and screenshot some mins after first post so not confusing you

first fazit of my test:

it is entering values to rrd-counter-item
aggregated graph seems to have the next problem with interpreting timestamps of rrd? (rrd contains other timestamps than the shown in graph from 1970

opus · September 7, 2022, 10:09am

Good! Remember, I was never able to make rrd4j use my changed .cfg without restarting openhab and deleting the old .rrd files. By the use of a totaly new item you migth have overcome that hurdle, but I’m not sure about that.
The REST ( with a custom timeframe) output shows the entries of archive 1 ( exactly 60000 msec between readings).
The posted chart however shows a whole day, that way archive 2 is used! Switching the chart to a timeframe of 8 houres or less would use archive 1!
From where did you get the

[quote=“Scriptwriter, post:13, topic:138657”]
awaited result of aggregate
[/quote] ?
I do not understand where “1970” on your graph is coming from. Mine do show times like “04:00” and at 00:00 the new date as a sole number “7”.
Working more with the MainUI chart ( I tend to stay with the old sitemap charts) I noticed the “aggregsted” chart-type. I have to digg deeper into that one, it sounds promissing, but…

As this was the config you posted in the OP, I sense a missunderstanding from my side!
Was this customized .cfg your desired setup?

EDIT:
Found the problem with the Chart! Instead of creating the chart from the items page using the analyze link you created a totally new chart via the pages menu. That way I get the same (weird) chart. The time reference went to start with 1970-01-01! Which datapoints are used, I can not tell.
If you start via the item page it should work better.
Additionally you should NOT use “Minute of Hour” as first dimension (I can’t say what is presented using that one), use “Hour of Day”. Secondly switch to the aggregation function “sum” and you will be presented with the hourly sum (If I’m not mistaken).
My aggregated chart looks like this:

Leaving this topic for a 2 week bicycle trip!!

Scriptwriter · September 21, 2022, 12:40pm

in the meantime i got out that the most trouble was coming from the instability of writing counteritems in rrd4j on my site.
my solution was, that i remembered that i had also most items in an influxdb-persistence. So i tested to change the standard-persistence to influxdb. rrd4j deactivated. all is working fine and also agregated graphs working without problems so i can now print out graphs with daily, monthly amounts from the same item as i wanted.
Best thanks for your help Juergen.

opus · September 21, 2022, 1:28pm

Have fun with influxDB!
I tried it for a ahort time but went back to rrd4j in order to minimize the effort for my raspberry.