Persistence averageSince becomes null

meingraham · November 13, 2019, 8:19pm

My persistence configuration:

rrd4j.cfg

ctr30m.def=GAUGE,300,0,305,300
ctr30m.archives=AVERAGE,0.5,5,24
ctr30m.items=sensor_luminescence

rrd4j.persist

Strategies {
    everyMinute : "0 * * * * ?"
}

Items {
   sensor_luminescence : strategy = everyChange, everyMinute
}

rule "Darkness Log"
    when
        Time cron "0 0/5 5-21 ? * * *" // Check every 5 minutes between 5am and 9pm
    then
        logInfo ("myLog", "Den Darkness is {}, average is {}"
                    , sensor_luminescence.state
                    , sensor_luminescence.averageSince(now.minusMinutes(60),"rrd4j")
                )
end

The sensor sends a message every 5 minutes; 24 hours a day. During the night, the sensor is logging “darkness” which comes in as 1. The rule begins logging values at 5am.

The issue is that all of the sudden, the average value becomes null and, obviously, that throws off the average. Why is averageSince returning null?

2019-11-12 05:00:00.083 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:05:00.078 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:10:00.060 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:15:00.077 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:20:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:25:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:30:00.075 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:35:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:40:00.073 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:45:00.078 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:50:00.064 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 05:55:00.071 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:00:00.060 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:05:00.078 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:10:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:15:00.061 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:20:00.065 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:25:00.072 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:30:00.097 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:35:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:40:00.073 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:45:00.085 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:50:00.073 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 06:55:00.072 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:00:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:05:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:10:00.078 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:15:00.116 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:20:00.098 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:25:00.079 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:30:00.082 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:35:00.057 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:40:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:45:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:50:00.079 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 07:55:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:00:00.094 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:05:00.072 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:10:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:15:00.082 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:20:00.096 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:25:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:30:00.030 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:35:00.018 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:40:00.043 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:45:00.033 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:50:00.055 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 08:55:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 09:00:00.087 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.033355369329782
2019-11-12 09:05:00.099 [INFO ] [smarthome.model.script.myLog] - Darkness is 2, average is 1.153357196848912
2019-11-12 09:10:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.196660002077707
2019-11-12 09:15:00.059 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.196660329833816
2019-11-12 09:20:00.065 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.196659783574242
2019-11-12 09:25:00.056 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.196661422362070
2019-11-12 09:30:00.084 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.155270704334580
2019-11-12 09:35:00.058 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.043331985227126
2019-11-12 09:40:00.086 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 09:45:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 09:50:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 09:55:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:00:00.082 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:05:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:10:00.081 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:15:00.101 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:20:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:25:00.066 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:30:00.081 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:35:00.071 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:40:00.085 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:45:00.065 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:50:00.084 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 10:55:00.064 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:00:00.089 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:05:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:10:00.094 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:15:00.064 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:20:00.058 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:25:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:30:00.088 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:35:00.079 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:40:00.041 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:45:00.085 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:50:00.071 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 11:55:00.033 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:00:00.045 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:05:00.060 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:10:00.074 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:15:00.077 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:20:00.072 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:25:00.066 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:30:00.101 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:35:00.059 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:40:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:45:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:50:00.081 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 12:55:00.086 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 13:00:00.084 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 13:05:00.073 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.0
2019-11-12 13:10:00.089 [INFO ] [smarthome.model.script.myLog] - Darkness is 2, average is 1.120733618491489
2019-11-12 13:15:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 1, average is 1.166661018709921
2019-11-12 13:20:00.070 [INFO ] [smarthome.model.script.myLog] - Darkness is 2, average is 1.283360406384648
2019-11-12 13:25:00.074 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 1.566721609004988
2019-11-12 13:30:00.093 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 1.900056219348789
2019-11-12 13:35:00.064 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 2.233360165727533
2019-11-12 13:40:00.049 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 2.450014666275566
2019-11-12 13:45:00.081 [INFO ] [smarthome.model.script.myLog] - Darkness is 4, average is 2.850044081643537
2019-11-12 13:50:00.077 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 3.116661870567543
2019-11-12 13:55:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 3.166660463193870
2019-11-12 14:00:00.091 [INFO ] [smarthome.model.script.myLog] - Darkness is 3, average is 3.166658333749979
2019-11-12 14:05:00.074 [INFO ] [smarthome.model.script.myLog] - Darkness is 120, average is 10.96868791914990
2019-11-12 14:10:00.071 [INFO ] [smarthome.model.script.myLog] - Darkness is 100, average is 17.71796164480362
2019-11-12 14:15:00.065 [INFO ] [smarthome.model.script.myLog] - Darkness is 80, average is 24.86859626579736
2019-11-12 14:20:00.064 [INFO ] [smarthome.model.script.myLog] - Darkness is 78, average is 37.41806448888983
2019-11-12 14:25:00.094 [INFO ] [smarthome.model.script.myLog] - Darkness is 81, average is 50.11807046996643
2019-11-12 14:30:00.098 [INFO ] [smarthome.model.script.myLog] - Darkness is 86, average is 65.70778606915140
2019-11-12 14:35:00.075 [INFO ] [smarthome.model.script.myLog] - Darkness is 94, average is 83.26101445358325
2019-11-12 14:40:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 98, average is 85.10725305702640
2019-11-12 14:45:00.067 [INFO ] [smarthome.model.script.myLog] - Darkness is 88, average is 82.10002249943751
2019-11-12 14:50:00.063 [INFO ] [smarthome.model.script.myLog] - Darkness is 95, average is 86.21667626819313
2019-11-12 14:55:00.059 [INFO ] [smarthome.model.script.myLog] - Darkness is 100, average is null
2019-11-12 15:00:00.088 [INFO ] [smarthome.model.script.myLog] - Darkness is 97, average is null
2019-11-12 15:05:00.020 [INFO ] [smarthome.model.script.myLog] - Darkness is 89, average is null
2019-11-12 15:10:00.071 [INFO ] [smarthome.model.script.myLog] - Darkness is 92, average is null
2019-11-12 15:15:00.066 [INFO ] [smarthome.model.script.myLog] - Darkness is 89, average is null
2019-11-12 15:20:00.101 [INFO ] [smarthome.model.script.myLog] - Darkness is 100, average is null
2019-11-12 15:25:00.061 [INFO ] [smarthome.model.script.myLog] - Darkness is 97, average is null
2019-11-12 15:30:00.088 [INFO ] [smarthome.model.script.myLog] - Darkness is 95, average is null
2019-11-12 15:35:00.052 [INFO ] [smarthome.model.script.myLog] - Darkness is 102, average is null
2019-11-12 15:40:00.112 [INFO ] [smarthome.model.script.myLog] - Darkness is 91, average is null
2019-11-12 15:45:00.073 [INFO ] [smarthome.model.script.myLog] - Darkness is 100, average is null
2019-11-12 15:50:00.069 [INFO ] [smarthome.model.script.myLog] - Darkness is 99, average is null
2019-11-12 15:55:00.081 [INFO ] [smarthome.model.script.myLog] - Darkness is 100, average is null
2019-11-12 16:00:00.083 [INFO ] [smarthome.model.script.myLog] - Darkness is 85, average is null
2019-11-12 16:05:00.072 [INFO ] [smarthome.model.script.myLog] - Darkness is 81, average is null
2019-11-12 16:10:00.066 [INFO ] [smarthome.model.script.myLog] - Darkness is 81, average is null
2019-11-12 16:15:00.066 [INFO ] [smarthome.model.script.myLog] - Darkness is 92, average is null
2019-11-12 16:20:00.061 [INFO ] [smarthome.model.script.myLog] - Darkness is 80, average is 80.0
2019-11-12 16:25:00.034 [INFO ] [smarthome.model.script.myLog] - Darkness is 91, average is 82.75012030723656
2019-11-12 16:30:00.075 [INFO ] [smarthome.model.script.myLog] - Darkness is 88, average is 82.76934594582815
2019-11-12 16:35:00.068 [INFO ] [smarthome.model.script.myLog] - Darkness is 88, average is 83.11116542877935

opus · November 14, 2019, 10:59am

Frankly, I have no idea!
I would look into the rrd4j database using the REST API and check the actually persisted values.

This setting, especially the last “300” surprised me. My understanding was the rrd4j needed the setting of 60 seconds here as well. But that understanding could be wrong.

rossko57 · November 14, 2019, 11:12am

I’m a bit suspicious of that, being suspicious of any rrd4j update that isn’t everyMinute.
In this case you don’t need every change anyway, I’d cut that out.

meingraham · November 14, 2019, 3:18pm

@opus @rossko57

Thanks. I’ve made your suggested changes and will monitor and report back.

Mike

P.S. Persistence settings, in general, are very much black magic to me.

meingraham · November 19, 2019, 2:33pm

I removed everyChange and changed the .def last parameter to 60 as suggested and I was still having the average go to null.

I then adjusted the .archives and that seems to have done the trick… although it really doesn’t make sense why. I haven’t seen a null for a couple of days. If it drops again, … “Ahll be back”

gag5mm.def=GAUGE,300,0,120,60
gag5m.archives=AVERAGE,0.5,5,288
gag5m.items=sensor_luminescence

I just adjusted the min/max because I now know more about the data range than I did when I first set this up. I don’t think that should affect my issue.

Mike