Timers not cancelled when rule reloaded

I have a rule that runs 7 daily scheduled tasks, when I reload the rule some of the current running timers seem to remain scheduled rather than following the rule and cancelling on a reload. Is there a command to find and cancel the orphaned timers, or a workaround other than rebooting?

Each individual task cancels the timer, and then resets it, one of the rules is below:

rule "HVAC Evening Scheduler"
when Time cron "30 5 0 ? * *"
or System started
or Item TodayEveningTime changed
or Item TodayEveningHVAC changed
or Item Vacation changed from ON to OFF
if (HVACEveningOnTime != null)
 {	HVACEveningOnTime.cancel()
	logInfo("Timers", "Evening HVAC Timer cancelled, current status is "     +HVACEveningOnTime)	}
if (DisableScheduler.state == OFF && TodayEveningHVAC.state == ON)	{
	var DateTime TodaysEveningOnTime = parse(Year.state.toString+"-"+Month.state.toString+"-    "+Day.state.toString+"T"+TodayEveningTime.state.toString+":00")
	if (TodaysEveningOnTime.beforeNow) {	}	else	{
		logInfo("Timers", "HVAC Evening On Scheduled for " +TodaysEveningOnTime)
		HVACEveningOnTime = createTimer(TodaysEveningOnTime) [|
			logInfo("Timers", "HVAC Evening Scheduled on rule ran ")
			sendCommand(HVAC, ON)	

Log Entries from normal operation with one change causes the script to cancel the timer and reschedule it as expected:

2016-05-05 08:53:57.228 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening On Scheduled for 2016-05-05T18:30:00.000+12:00
2016-05-05 08:56:01.549 INFO  o.openhab.model.script.Timers[:53]- Evening HVAC Timer cancelled, current status is org.openhab.model.script.internal.actions.TimerImpl@6ca989
2016-05-05 08:56:01.564 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening On Scheduled for 2016-05-05T18:35:00.000+12:00

After saving the rule again with no changes I see:

2016-05-05 08:57:28.309 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening On Scheduled for 2016-05-05T18:35:00.000+12:00
2016-05-05 08:57:50.631 INFO  o.openhab.model.script.Timers[:53]- Evening HVAC Timer cancelled, current status is org.openhab.model.script.internal.actions.TimerImpl@1a6d7a7
2016-05-05 08:57:50.647 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening On Scheduled for 2016-05-05T18:30:00.000+12:00

BUT after the rule is reloaded there is NO timer cancelled message before the timer is created, suggesting the old timer is still running.
I have seen this reflected in the logs as the same event being triggered twice, and under certain conditions with a stack dump which thankfully seems to have gone now I’ve put them in their own rule file with separate logging.

2016-05-04 18:30:00.052 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening Scheduled on rule ran 
2016-05-04 18:30:00.287 INFO  o.openhab.model.script.Timers[:53]- HVAC Evening Scheduled on rule ran 

It doesn’t seem to happen to all the timers in the rule after I edit the rule file, yesterday it affected 2 out of the 4 left to run.
The Cron Job that schedules the rule to run just after midnight seems to only invoke one instance of each timer.

Interesting. We’ve just had another thread that was asking about what Timers do when a rules file is reloaded. I thought they went away. Apparently I was wrong.

One good question would be whether the a System shutdown rule gets triggered when a rules file reload occurs. If yes then you can use that rule to cancel all of your outstanding Timers.

Beyond that we are going to have to get clever to solve this. Some initial thoughts, none of which are compelling solutions:

  1. Create a System started rule and populate a DateTime Item with the current time. Before you create the Timer create a val that contains a copy of that DateTime Item’s state. Since the Timer’s lambda inherits the state from the rule it will have a copy of that val. In your Timer add a check to see if the System Started DateTime state matches your val. If it does you know there hasn’t been a restart and this is still a valid Timer. If it has changed you know that the rules file was reloaded and therefore the Timer is stale so do nothing.

  2. Try to move away from Timers. Instead of setting a Timer to go off at a certain time, consider whether you can based the HVACEveningTime on some other event (e.g. 90 minutes after sunset) rather than a manually set time.

  3. Add some checks in your Timer’s lambda so that even if the Timer is stale it doesn’t matter, beyond having extra log statements. For example, add a check to see if TodayEveningHVAC.state still matches TodaysEveningOnTime. If it does then only send the ON command if HVAC isn’t already ON. If it isn’t then don’t do anything. You can even put in a reentrantlock if you still see cases where the timing falls just right that it still sends the ON command twice. NOTE: I’m not certain that you can do reentrant locks inside a Timer.

Thanks for the reply, what is the syntax for system shutdown? Designer doesn’t like

rule "Cancel All"
when System shutdown then
{	logInfo("Timers", "System has been shutdown")	}

Your points 1 and 3,

If I did something like

var String Test = TodayWakeTime.state.toString

and when the timer fires

if (Test == TodayWakeTime.state.toString)
{    Do the timer complete stuff   }

would work?

EDIT: Thinking about this again it’ll need to be more accurate than that as I have seen duplicate timers firing within less than a second of each other

Point 2 won’t really work as this is about tailoring the automation to the working day, so its a glorified scheduler.

I’ve just done a further experiment and adding conditions to the timer resulted in an “unkown item” stack dump when it expired as there was a duplicate timer that fired at the same time due to editing the rule file. My guess is the two of them try to do the same “if” command at the same time and get upset when they can’t.
I’ve been trying to find this stack dump for days so being able to reproduce it is a good result!

I was going to say whenever I have a timer running and (usually inadvertently) cause that rules file to reload, when the timer is set to go off I get a stack dump, even if I don’t actually change the rules file in question. (i.e. add a line, delete that line, save file.) But I never use cron rules, so those may somehow be a special case.

System shut down

There is a space there for some reason.

I don’t think so because TodayWakeTime.state isn’t guaranteed to change on a reload of the rules file so this would not be a reliable way to detect a stale Timer.

I will believe you but just want to push back a little bit more. If it is based on the working day then that implies a relatively static schedule which could easily be implemented with cron triggers instead of Timers. If the schedule is something that changes every day and you cannot base the behavior off of some other event (e.g. sunset, building occupancy, etc.) then I agree a Timer is probably what you would need.

The reason I push back is because many users of this forum (and others) get so focused on HOW they think they want to solve their problem they don’t think to step back to consider if there might be a better way to solve it better using a different approach.

I doubt the simultaneous call of the if statements is the cause. The Timers are running in separate threads completely independently of one another.

I suspect the problem might be that because of the reload of the rules files resets the context which renders the Item references in the stale timer stale. So when you try to reference HVAC in the old Timer it cannot find the Item. This could actually be a good thing as the thrown exception is actually preventing the stale Timer from completing its execution. You could probably add a try/catch to catch that exception and if you fall into the catch you know you are in a stale timer.

That further supports my hypothesis that when you reload a rules file the context that the old Timer inherited becomes stale.

Yep, I missed the space - does it have long enough to work after a Sudo Reboot, or does a reboot need something like “sudo service openhab stop” followed by maybe a pause and then “sudo reboot” on Raspbian to work properly? Given the need to limit SD Card writes on a Raspberry Pi having a shutdown rule might be a better way of persisting my items rather than after every change. Do you use “System shut down” in any of your rules?

The 7 day scheduler is something I’ve recently developed that allows me to change things easily at will, its very useful especially as I use it to open curtains as an Alarm Clock among many other things. It’s something I’ve evolved via Cron statements, and with the Timer problems I’ve had, I’d have gone back to cron if it made sense.
I do still use Cron for things that don’t changed such as when the Hot Water has a chance to heat at the end of the day if the Solar hasn’t performed. So far simplifying things by putting the timers into their own rule file with the result switching an item, and then leaving that file alone as much as possible seems to be working. Yesterday I added a switch to cancel all the timers which I can use before editing the rule which seems to prevent the orphans.

Orphaned timers losing their context would make sense as its always the “unkown” error that appears, I assumed from reading on the forum that once the timer had been scheduled it included the rest of the code which it likely doesn’t. However I’ve seen both double events and also stack dumps so maybe it can logInfo and then just switch an item in duplicate, or maybe when too many of them are created the stack dumps appear? Either way now I know the problem I can work with it.

Maybe Catch / Throwable T is the way to go with each timer statement in a rule that I’m working on. I use this in a rule that gets a JSON string from a cloud server every minute after learning about it in an earlier conversation. It works well at keeping my logs clean after the occasional error.

Thanks again for your help, hopefully this is useful information for those following in my footsteps!

I believe systemd calls stop on all running services and waits for then to exit before rebooting or shutting down the machine so your shut down rule should work just fine.

Only persisting your items in a shutdown rule would only work in cases where OH shuts down cleanly. If OH crashes or your pi loses power, no persisted values. If writes are a concern, better to move the writes off of the pi entirely (e.g. USB HD, NFS mounted share, use a non-embedded dB like MySQL running on a separate server, etc.) If you can’t trust your restore on startup every time no matter what, you may as well not have any because you will have to write all the code and checks you need if it fails that you would need to write if you didn’t have any persistence at all.

I do not use a shut down rule. I try to have all my state stored in Items with restore on startup and try to write my rules and timers so shutdowns, reloads, crashes, what have you just work. This means always checking states before sending a command or update to avoid duplication, avoiding long running timers, and taking advantage of using events to drive behavior instead of cron triggers, polling, or timers.

The only special rules I have are system started rules. One sends a message to my door sensors to report their current status in case they changed while OH was down. The other calculates what the current time of day (a series of named time periods I use to drive behavior or check before doing things (e.g. turn on the lights at dusk, don’t send informational alerts at night) in case OH was down during a transition from one time period to the next.

Thanks, it seems the shutdown rule doesn’t seem to work with either a simple “sudo reboot,” or “sudo service openhab shutdown” I tested it using the following rule

rule "system rebooting"
when System started
or System shuts down then
{	logInfo("LogOnFileSystem", "This is a shutdown log entry")
logInfo("LogOnFileSystem", "This is a shutdown log entry from 5 seconds later")

Nothing was logged on shutdown, and designer doesn’t like “System shut down” as an option, only “System shuts down.”

Maybe this part of Openhab only works in very specific circumstances?

I currently use MapDB for persisting everything that needs a memory between reboots, as I only want to know the last state, do you know if the whole database is rewritten everytime one item changes, or is it more advanced than that? The database folder is 126kb, so if its a full rewrite everytime a single item changes, the writes are starting to add up.
Is there another persistence method that stores the last state only so I could maybe persist one on graceful shutdown only?
Elsewhere on the Pi Redis is used for disk caching of power data, is there anything I can do to tie into this that you know of? (A quick search doesn’t find anything in the persistence section)

I’m thinking that I may be better off replacing a lot of my persistence with persist on shutdown and twice a day, with only the very important ones being on everyChange.

Is there a persist on shutdown command that can be put into mapdb.persist or is it something that has to reside in a shutdown rule?

I don’t know for sure but would be astonished if MapDB writes the whole file every time. It should only write the changes.

There is no Redis binding that I know of but if Redis has a standard JDBC connection the JDBC binding should work.

There is no persist on shutdown option in OH. And even if there were I have no reason to believe it would work better than the System shuts down rule. If you want to persist at shutdown you must use a rule. And since you have demonstrated that the error didn’t execute even that wouldn’t work.

Again, if wires are a concern, you should look into how to move those files off of the SD Card. That will be more reliable and less work trying to force a partial solution inside of OH itself.

Thanks for the reply, very informative as always!

I will leave persistance as it is for now and take regular backups. If the SD card dies an early death I’ll symlink the persistence file to a USB stick when I get a new card.

Since creating a cancellation switch, and moving the daily timers into a seperate rule file that I largely leave alone I’ve had NO stack dumps, so I can happily say the workaround is working perfectly. Thanks for your help as ever.

One of the working rules placed in a seperate rules file for those following in my footsteps is below:

import org.openhab.core.library.types.*
import org.openhab.model.script.actions.*
import java.util.concurrent.locks.ReentrantLock //lock range available for this file is 91-99
import org.joda.time.*
var Timer HVACMorningOnTime
var java.util.concurrent.locks.ReentrantLock lock91 = new java.util.concurrent.locks.ReentrantLock()

rule "HVAC Morning On Scheduler"
when Item DisableScheduler changed
or Item TodayWakeTime changed
or Item Vacation changed from ON to OFF then
try    {
if (HVACMorningOnTime != null)
{    HVACMorningOnTime.cancel ()
     HVACMorningOnTime = null
     logInfo("Timers", "HVAC Morning ON Timer cancelled, current status is " +HVACMorningOnTime)
if (DisableScheduler.state == OFF && Vacation.state == OFF)    {
    var DateTime TodaysMorningOnTime =parse(Year.state.toString+"-"+Month.state.toString+"-"+Day.state.toString+"T"+TodayWakeTime.state.toString+":00")
   if (TodaysMorningOnTime.beforeNow) {    }    else    {
       logInfo("Timers", "HVAC Morning On Scheduled for " +TodaysMorningOnTime)
       HVACMorningOnTime = createTimer(TodaysMorningOnTime) [|
          logInfo("Timers", "HVAC Morning Scheduled On Rule Ran")
          sendCommand(HVAC, ON)
}    finally    {lock91.unlock()    }