Persistence stopping at midnight at the start of each month

Hello,
I’m experiencing a very weird behaviour with persistence service. I have a standard InfluxDB+Grafana setup.
Practically every month, precisely at midnight of the last day of a month, persistence seems to stop for one single of the few items I have in my setup. It re-starts after a couple of days without me doing anything.
Here is what I see in Grafana:


InfluxDB is still running as the other items are tracked correctly. I am a newbie, I’m not sure what to check here. Any suggestion?

This was last month:

Thanks!

What do you seen in events.log at that time?

We need to determine if the problem is caused by persistence or the Item no longer updating. If the Item is no longer updating we need to look at what is producing the updates (binding, device) to see if it is generating events during this time.

I should have specified. The item is online and keeps updating regularly and I can see the readings in BasicUI ok.
Nothing except item change events appear on the events.log.
So this is definitely a persistence problem.

What I also find very weird is that it generally re-starts working fine without me doing anything after exactly 72h at midnight!

Do you have a cron rule that runs at midnight every month? That lasts for 72h?

I don’t have any cron rule. This is my rules file, very simple, I have nothing else that I can think of:

Blockquote
rule “Records when device was last seen”
when
Item GF_Backyard_Temperature received update or
Item GF_Backyard_Humidity received update or
Item GF_Backyard_Barometer received update
then
postUpdate(GF_Backyard_LastUpdate, new DateTimeType())
end

This is my influxdb .persist file:

Blockquote
Strategies {
everyMinute : “0 * * * * ?”
everyHour : “0 0 * * * ?”
everyDay : “0 0 0 * * ?”
every2Minutes : "0 /2 * ? * "
}
Items {
GF_Downstairs
: strategy = everyMinute
GF_Backyard
: strategy = every2Minutes
FF*: strategy = everyMinute
}

It’s the GF_Backyard items that have the issue (there are 3 of them).

Do you see anything in InfluxDB logs?

Put the influxdb add-on into trace logging (maybe logging to a separate file) and see if there is something going on during this down time.

See Log4j2 Sample Config for how to do that.

Hmmm. You are using only cron type strategies. I think @unparagoned might be on to something.

Try this. Change the first zeros on all of your cron expressions to a different number so these don’t all run at the same time. Perhaps you are running out of scheduling threads. It seems unlikely but without further information it’s the only thing that stands out.

NOTE: I assume FF is a Group. If not then the * isn’t doing what you think it’s doing. * means “All members of this Group”, not “All Items whose name starts with FF”.

2 Likes

@rlkoshak, I did what you suggested, changing the cron expressions and rebooted the system. Persistence re-started immediately. I’ll check it out to see if the behaviour re-appears.
Before simply restarting openhabian wasn’t effective. So I believe @unparagoned was actually pointing in the right direction!
I will also consider changing the strategy for some items to everyChange…

For now it seems solved! Many thanks for the fantastic help here!

No expertise here, but have often seen recommendation that boils down to “for graphing use everyMinute”.
Wonder if your graphing util asks for odd or even minutes data at first.

interesting point… I must keep this in mind and see what happens at midnight and/or start of the month.