Rules running at system start

[Andrew_Rowe]

This thread is not asking for help, more of a best practices discussion. I have a rule running on my system that at system start or 5 secs after midnight every day sets an unbound item to the day of the week. Other rules use this unbound item to make decisions, everything seems to work well.

rule "WhatDayofWeek"
when
    System started or
	Time cron "5 0 0 ? * * *"
then
DayOfWeek.sendCommand(now.getDayOfWeek)
end

Restarted the system OpenHAB runs on this morning and notice an error in the log during start up

08:14:18.263 [ERROR] [untime.internal.engine.RuleEngineImpl] - Error during the execution of startup rule 'IsTheWeekend': Could not cast NULL to java.lang.Number; line 133, column 15, length 25

Easy enough to figure out, the item DayOfWeek was NULL when the rule attempted to set a variable to it’s value. Here is the rule in question

rule "IsTheWeekend"
when
    System started or
    Item DayOfWeek changed
then
	var daynum = DayOfWeek.state as Number
	if (daynum > 5){
		IsWeekend.sendCommand(ON)
	}
	else {
		IsWeekend.sendCommand(OFF)		
	}
end

As I considered this error, I realized this was something that occurred because the system was starting and may not be of much concern. The item had not been set to any value when the rule file got loaded. Looking at the log in fact reveal that a mere 26 thousands of a second later the item got set

08:14:18.286 [INFO ] [smarthome.event.ItemCommandEvent     ] - Item 'DayOfWeek' received command 7
08:14:18.296 [INFO ] [smarthome.event.ItemStateChangedEvent] - DayOfWeek changed from NULL to 7

(btw system is PC running Deb i3 4gb 1 tb) These two events are actually successive in the log. The two rules are also one right after the other in the rules file. Obviously the first rule above runs now.getDayOfWeek, the next rule listed runs immediately after this one and throws the error because the system is going to require 26 thousands of a second more time to be able to set the item so at that moment it is NULL
I don’t want to get into a discussion about fixing the architecture of the program or how to monkey with loading rules at start up. It is what it is.

Even thou I considered this error a ‘non’ problem, I still figured I’d see if I could craft a simple way to eliminate it. This has led me to rethink (and question) my strategy. My first thought was to test for NULL and perhaps set the value to a ‘non-nominal’ value… like zero. (1 = Monday, 7 =Sunday) That way the error on start goes away and a rule needing this value could be made to know 0 means the value is not yet set and re-trigger whatever mechanism that sets the value. After a minute that idea started to smell.
My next thought was test for NULL and use return; to bail out of the rule early if the value was not yet set. Then it occurred to me that maybe a better strategy may be to just not run the second rule at start up.

rule "WhatDayofWeek"
when
    System started or
	Time cron "5 0 0 ? * * *"
then
DayOfWeek.sendCommand(now.getDayOfWeek)
end

rule "IsTheWeekend"
when
    //System started or
    Item DayOfWeek changed
then
	var daynum = DayOfWeek.state as Number
	if (daynum > 5){
		IsWeekend.sendCommand(ON)
	}
	else {
		IsWeekend.sendCommand(OFF)		
	}
end

This seems the best solution for setting unbound items in a start up process. After all, with the System started line removed from the second rule, 26 thousands of a second later, the item gets set, theoretically that should trigger the second rule and all is good. Although start up is an edge case, the use of unbound objects as part of rules decision making requires these unbound objects to be set to something at start up.

Also, I have influxDB set up and grafana and a working persistence scheme to restore on start but for some reason, these unbound items (I use a few) don’t seem to get recorded in the DB and don’t get restored on start up. Still working on that (I’m a noob and have lots to learn)

Thoughts?

[5iver (Scott Rushworth)]

I seems to me that you are overcomplicating things, since these values could easily just be obtained directly in the rules, rather than storing the values in Items (and bothering with persisting them).

[Andrew_Rowe]

What if the value is used in some other way Scott? Such as displayed in a custom web page ui I created? thanks I’m sorry but this is a little upsetting, I didn’t really do a good job expressing that this is actually part of a much bigger rule that includes Friday evening and excludes Sunday evening to adjust lighting schedules. Yes I could calculate those as well in a rule but it seemed easier to write one rule I could use anywhere since the value is used in several places.
Granted the example code is extremely simple and rather pointless but you seem to have missed the point. The problem came up in much more complicated experimental code that I didn’t want to post as an example and posted the above instead. The point was trying to find a safe way to initialize values at start up

[5iver (Scott Rushworth)]

@Andrew_Rowe

What if the value is used in some other way Scott? Such as displayed in a custom web page ui I created?

I had originally asked that, by deleted the question because I thought it unlikely that someone would display the day of week or a switch respresenting the weekend.

@Andrew_Rowe

The point was trying to find a safe way to initialize values at start up

Persistence will get the Items initialized. The rules you ended up with will keep them updated, but you could combine the rules into one that updates both Items. I was a bit confused by your post, as you were asking for best practice for something so general. I think this is one of those things that will have a specialized solution for each implementation.

[Andrew_Rowe]

Again sorry Scott because the example rule was a complete over simplification and I was a little upset because I realized I wasn’t making my issue clear enough. It is from when I first started and now that I’ve learned enough javascript to make the website retrieve the value itself it seems silly. There is more to the rule but I left out for clarity (and it is still really messy).

@5iver

Persistence will get the Items initialized

As mentioned, for some reason persistence doesn’t seem to work on these unbound items. Here is my persistance file.

Strategies {
    everyMinute : "0 * * * * ?"
    everyHour   : "0 0 * * * ?"
    everyDay    : "0 0 0 * * ?"
}

Items {
    ghome* :strategy = everyChange
}

The file is named influxdb.persist and I have influxdb as my default in system setting. The unbound items all are in the (gHome) group. I have grafana running and can see charts and values for most items but none of these unbound objects record values (or at less grafana isn’t finding them) I know little about grafana. That could be the problem. I planned to try to use unfluxdb command line tool to run query and check for items but hadn’t got that far.
That wasn’t the big deal thou. Problem was I had used the same rule for an example rule I had posted. I found the system start of my example rule didn’t work, throws an error and isn’t therefore a real great example. I wanted a better method and basically have devise one but was wondering if other folks deal with this differently. Maybe others have persistence working on these unbound object and it isn’t an issue.

[Udo_Hartmann]

Now, is the name of the Group gHome or ghome?

In question of system started Triggers:
Please be aware that openHAB uses 5 threads for rules per default, so if using system started or matching Time cron triggers for more than five rules (regardless if in the same file or not) it’s very likely that some of that rules may not be triggered at all. It’s better to build less rules to do more jobs, in the example above it would make sense at least to calculate both weekend and day of week in one rule. Jep, I got it that these rules are simplified, but as the triggers are the same (when triggering the first rule, the second will also be triggered), there is no point to split them up.

[Andrew_Rowe]

@Udo_Hartmann

Now, is the name of the Group gHome or ghome ?

Udo… I’m not home right now but I think you cracked the persistence problem!!! The group name is indeed gHome. What a silly mistake. Thank you! All the items which are bound to an binding must find there way into infuxdb on their own (?) because I can make charts and graphs ect.

[Andrew_Rowe]

@Udo_Hartmann

In question of system started Triggers:
Please be aware that openHAB uses 5 threads for rules per default, so if using system started or matching Time cron triggers for more than five rules (regardless if in the same file or not) it’s very likely that some of that rules may not be triggered at all. It’s better to build less rules to do more jobs, in the example above it would make sense at least to calculate both weekend and day of week in one rule. Jep, I got it that these rules are simplified, but as the triggers are the same (when triggering the first rule, the second will also be triggered), there is no point to split them up.

In fact maybe I should probably try to consolidate all system start stuff in one rule (or as few as possible).
I’ve also begun initializing these unbound items to things like ON or a number. I feel it is important/easier to not have to test for NULL in every place I need to use these unbound objects.

Anyhow, I think with persistence working, this might not be a problem with these unbound items?
Gosh, it is funny because a few of these unbound items I had created quickly without a group and the other day I reread the docs, check all had groups and looked at that dog gone persistence file!!! Didn’t catch the capitialization mistake. This community is AWESOME!

[Udo_Hartmann]

In fact, it’s very easy to see somebody else’s (simple) mistakes (especially if taking the first look at it), but it’s damn hard to find my own mistakes…

[Andrew_Rowe]

@Udo_Hartmann

In fact, it’s very easy to see somebody else’s (simple) mistakes (especially if taking the first look at it), but it’s damn hard to find my own mistakes

I try to explain that to my boss but he still thinks I’m a moron, thanks again Udo

[Udo_Hartmann]

@Andrew_Rowe

I try to explain that to my boss but he still thinks I’m a moron

[rlkoshak (Rich Koshak)]

@Andrew_Rowe

I don’t want to get into a discussion about fixing the architecture of the program or how to monkey with loading rules at start up. It is what it is.

And yet, sometimes those discussions are where the answer to your original question lie.

@Andrew_Rowe

Even thou I considered this error a ‘non’ problem

Often whether or not something like this is a problem or not depends on the context. So it is good to consider a solution assuming it were a problem.

@Andrew_Rowe

My first thought was to test for NULL and perhaps set the value to a ‘non-nominal’ value… like zero. (1 = Monday, 7 =Sunday) That way the error on start goes away and a rule needing this value could be made to know 0 means the value is not yet set and re-trigger whatever mechanism that sets the value. After a minute that idea started to smell.
My next thought was test for NULL and use return; to bail out of the rule early if the value was not yet set.

Actually these two approaches are really indistinguishable. In both cases there is a special symbol (NULL or 0) to represent when the Item is in an unknown state.

@Andrew_Rowe

Although start up is an edge case, the use of unbound objects as part of rules decision making requires these unbound objects to be set to something at start up.

Really, the problem is OH 2 startup is broken. You are attempting to mitigate something that isn’t working properly in the OH core in the first place. Anything that relies on a System started Rule is going to potentially face problems like these until the OH 2 startup problems get solved, which probably won’t happen until OH 3 really.

I don’t see a whole lot of issues with your overall approach of using two different Rules, but I don’t think it really solves the problem. It just moves the problem. For example, you can have Rules executing which depend on the state of IsWeekend and IsWeekend may not have been restoreOnStartup yet and is therefore NULL. What do you do in those Rules in that case?

@Andrew_Rowe

Thoughts?

Personally, if indeed the error during startup isn’t really a problem, I would simply ignore it. If it is a problem any solution I think will depend on the specific nature of the problem.

As a general Rule, all of your Rules should be prepared to handle any Item it depends upon being NULL or UNDEF. If you do that then most of these OH restart problems will go away. I can’t give you a good approach for how to do that though. In some cases simply exiting the Rule would be appropriate. In others setting a timer and trying again later would be appropriate. In still others some alternative path through the Rule would be appropriate (e.g. assume default values).

And, in still others, that discussion about the architecture or overall approach is appropriate.

@Andrew_Rowe

Also, I have influxDB set up and grafana and a working persistence scheme to restore on start but for some reason, these unbound items (I use a few) don’t seem to get recorded in the DB and don’t get restored on start up. Still working on that (I’m a noob and have lots to learn)

I strongly recommend using MapDB for restoreOnStartup. It is very much faster than any of the external databases which will give you fewer of these startup type problems.

@Andrew_Rowe

I’ve also begun initializing these unbound items to things like ON or a number. I feel it is important/easier to not have to test for NULL in every place I need to use these unbound objects.

Because the root problem is Rules can start executing before restoreOnStartup, the only truly effective way to deal with this problem is to deal with NULL in all your Rules and take appropriate action. Anything else is going to just move the problem or temporarily mask it.

Also be aware that bindings are starting to use UNDEF more, which is another special case that you may have to deal with.

I can’t remember if it was you who I had the discussion about edge cases awhile back, but these are all edge cases.

And this is a great example of of the maxim in programming that “80% of the effort is spent on the last 20% of the program.” Getting the basics done only take about 20% of your effort. The remaining 80% is spent dealing with NULL, UNDEF, the time transition over midnight, daylight savings, power outages, and the like. This maxim also applies to the code itself. 20% of the code actually does work, the remaining 80% is error checking and handling. In OH the ratio isn’t that extreme, but it would not be out of the ordinary to have more than half of a Rule there to handle stuff like NULL and UNDEF.

[Andrew_Rowe]

@rlkoshak

Because the root problem is Rules can start executing before restoreOnStartup, the only truly effective way to deal with this problem is to deal with NULL in all your Rules and take appropriate action. Anything else is going to just move the problem or temporarily mask it.

that is excellent advise and I think I will follow it

@rlkoshak

it would not be out of the ordinary to have more than half of a Rule there to handle stuff like NULL and UNDEF.

wow, ok… to quote my own silly self

@Andrew_Rowe

This has led me to rethink (and question) my strategy.

me thinks the best strategy may be to leave system start up rule behind, count on restoreOnStartup to fix the missing values eventually and code for NULL or UNDEF in rules

@rlkoshak

I strongly recommend using MapDB for restoreOnStartup. It is very much faster than any of the external databases which will give you fewer of these startup type problems.

OK (light bulb goes on) … MapDB is an internal component? or something and rest are external DBs… no call to outside service… ??? faster

OK… will do!

[rlkoshak (Rich Koshak)]

@Andrew_Rowe

me thinks the best strategy may be to leave system start up rule behind, count on restoreOnStartup to fix the missing values eventually and code for NULL or UNDEF in rules

That can work in some cases but not in all cases. Take the Time of Day Design Pattern. It is not unreasonable to think that OH might be down when a time of day transition occurs. So there area few approaches to take:

1. put the system in an undefined state until the next time of day transition
2. let the system be in the restoreOnStartup state until the next time of day transition, even if it was skipped while OH was down
3. recalculate Time of Day in every Rule that requires that information (can still use a centralized Rule for this)
4. continue to use a System started Rule

All of these have advantages and disadvantages. I currently implement 4. In the Rules that care about Time of Day, if the Item is NULL of UNDEF the Rule does nothing (with warnings and alerts of course to tell me something is not right). Option 3 is one I just thought of and it’s intriguing. I’m not sure if this would be a good approach or not. The way it would work is the Rule would have a Switch added added as a trigger. In a Rule if the time of day Item is NULL or UNDEF, send command to the Switch, wait half a second and then proceed. This would be a kind of just in time calculation.

For something like the weekend, it might not be necessary to calculate it at startup. The likelihood that OH is down over night is lower. And if it is down that long, you will know to look for problems when OH comes back up.

@Andrew_Rowe

OK (light bulb goes on) … MapDB is an internal component? or something and rest are external DBs… no call to outside service… ??? faster

MapDB and rrd4j are both embedded databases. They run as part of OH and therefore there is no overhead caused by networking. All the rest are external programs that OH must communicate with over a network, which adds latency.

[Andrew_Rowe]

@rlkoshak

Option 3 is one I just thought of and it’s intriguing. I’m not sure if this would be a good approach or not. The way it would work is the Rule would have a Switch added added as a trigger. In a Rule if the time of day Item is NULL or UNDEF, send command to the Switch, wait half a second and then proceed. This would be a kind of just in time calculation.

wheels are turning in my head, I’m trying to imagine this, I’ll have to go take a peek at the DP

@rlkoshak

MapDB and rrd4j are both embedded databases.

gotcha… very important thank you Rich will follow your advise on this matter

[Andrew_Rowe]

@Udo_Hartmann

Now, is the name of the Group gHome or ghome ?

Udo… I’m not home right now but I think you cracked the persistence problem!!!

Home run, you nailed it Udo because changed one character in persist file to capital and grafana is logging data… thanks!

syntax owns me

Really really people, you can not buy support like this for any price. Folks all around the globe, helping, pitching in where they can… 24/7… = awesome!!!

one of the best community projects I’ve had the pleasure.