Solutions for Lazy Execution of Rules

lukics · October 4, 2018, 11:25am

My story:
I have a zoo of actors and sensors for Illumination at home. Lamps controlled through homematic actors, Lightify and Tradfri ones. Sensors (switches) from homematic traditional and IP with motion detection and without.

When I started it was really frustrating, pressing the switch sometime gave result sometime not, sometime it was just delayed. There was no clear picture what factor influenced this behaviour. pressing the switch too fast gave also unexpected results.

Expectation:
Certain rules can execute silent in the background and nobody cares about how real-time they are fired.

Others like pressing a wall switch and expecting the light to come on immediately, pressing again and switching it off, holding the switch down to dim, or what ever is programmed.

I have gone through a learning curve and want to summarize my experience and findings here in this thread

Topics I will touch:

Switch initial state NULL
Switch Channels v.s. Item
Thread Pool
Java Memory Management
Rule Compilation
Visual Studio Code
Sample Code

Switch initial state NULL

I had my rules defined this way

rule "LivingRoom Toggle Light"
when
    Item switchXY changed from OFF to ON
then
.......
end

This resulted in the situation that when OH was rebooted the 1st pressing was ignored, because switch state was initially not UNDEF or NULL, going from NULL/UNDEF to ON. I have persistence enabled, though when rebooting this was the last value which was persisted .

My behaviour was then to once again and again… resulting weird things.
I fixed this after analysing the log.

Switch Channel v.s. Item

What I have seen is that when I pressed the switch in sequence too fast, expecting on/off of lamp I had to wait certain time between each step.
The reason was that the item had to go through states OFF-ON-OFF and this takes some time.
Some Things generate channel messages for particular messages. These channel messages come more then less real-time. You can verify in the log messages if this particular channel generates such

my particular wall mount switches did and I changed the rules to

rule "LivingRoom Toggle Light"
when
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered SHORT 
then
......
end

The channel trigger PRESS has also the values LONG, CONT, LONG_RELEASE, which can be used for dimming rules

Thread Pool
I have increased the thread pool to a higher value
editing /etc/openhab2/service/runtime.cfg and adding line

org.eclipse.smarthome.threadpool:RuleEngine=20

This might give some relieve, though programming the rules the right way should be a better way to go, e.g using timers instead sleeps.

Java Memory Management

I am running OH on a RPI3, memory is some sort short, in order to give the max to OH edit /etc/default/openhab2 and add

EXTRA_JAVA_OPTS="-Xms400m -Xmx650m"

This gives min and max memory of 650 MB which is 2/3 of total , figures differ for other platforms, 2/3 could be a rule if OH is the only app running.

Rule Compilation

I have witnessed that when I reboot the OH, just edit a rules file or after some period of time, the execution of an action is delayed by some seconds (5-10). The reason is that a rules files gets compiled just in time. Just in time mean when the first rule needs to be executed. That means that the rules file is just pre-parsed at boot/save for the when clause, the actual compilation and loading in memory happens when needed. That might be fine for memory management, for the real-time execution it’s a no go.
As a result I added in each rules file a almost empty rule triggered by system started event

rule "Initiate 4711.rules"
when
        System started
//   or Time cron "	0 0/1 * 1/1 * ? *"
hen
        logInfo("Initiate 4711.rules", "finished")
end

the good thing is that the system started event gets fired at boot time (multiple time !?) but also when a rules file is saved, and the best, just for that particular file saved (other rules files are not triggering)

I did not implement yet, but as an idea a cron statement additionally to system started could keep the rules file in memory. Not sure if there would be a side effect of flooding the thread pool if at once to many refreshes start.

Visual Studio Code

I love Visual Studio Code, but when run, it’s killer for OH performance. Seen that it even stops some processes when existed (workaround should be there already). I have witnessed that when I had a VSC session, during but also after one, rules got really unresponsive. That might be because the language server consumes much memory and rules are recycled, I do not but I also for the moment do not care.

I have created on a other server (rock64, RPI3+ should be fine too) a shadow OH instance

From the production node I exported by NFS

/etc/openhab2 veveohab-vss(rw,sync,no_subtree_check,all_squash)
/var/lib/openhab2/jsondb  veveohab-vss(ro,sync,no_subtree_check,all_squash)

and mounted to the same folders in the shadow node

veveohab-pro:/etc/openhab2  /etc/openhab2                       nfs rw  0  0
veveohab-pro:/var/lib/openhab2/jsondb /var/lib/openhab2/jsondb  nfs ro  0  0

I use NFS, samba should fine too, instruction on details how to can be found on the web.
The VSC now I point to the config directory of the production as openhab server (language server) I set the shadow one. The syntax check is not faster, though I do not care as long it does not affect the production.
There are issues with new added item in items file, these are not seen by the shadow node server and restart of OH-shaddow is needed (have find out how fix that)

Sample Code - Lambdas for Light Control

I have created following lambdas

val dimmLight= [ GenericItem dimLampItem |
    	try{
		sendCommand(dimLampItem,  (dimLampItem.state as Number + 5).toString)
	} catch(Exception e) {
		sendCommand(dimLamItem, OFF)
	}
    	dimLamItem.state.toString
]
val  toggleLight= [ GenericItem dimLampItem, GenericItem dimLampItemColor |
    try{

	if ( dimLampItem.state == 0) { 
		if (isBedTime.state == ON)
			sendCommand(dimLampItem, "5")
		else
			sendCommand(dimLampItem, "100")
	if (dimLampItemColor !== null) 
			sendCommand(dimLampItemColor, "2500")
	}
	else
		{sendCommand(dimLampItem, OFF)}
  	} catch(Exception e) {
		  sendCommand(dimLampItem, OFF)
	}
    dimLampItem.state.toString
]
val  motionToggleLight= [ GenericItem motionSensor, GenericItem motionSensorIlum, GenericItem dimLampItem |
    try{
		if (isBedTime.state == OFF) {
			if ( motionSensor.state == ON && motionSensorIlum.state < 50) { 
					if (motionSensorIlum.state < 20)  sendCommand(dimLampItem, "5") else sendCommand(dimLampItem, "100")
			}
			else {
				sendCommand(dimLampItem, OFF)
			}
		}
  	} catch(Exception e) {
  	sendCommand(dimLampItem, OFF)
  	}
    dimLampItem.state.toString
]

sample rules

rule "StairsHallway motion light "
when
    Item g_TreppeGangObenMotion changed 
then
	motionToggleLight.apply(g_TreppeGangObenMotion, g_TreppeGangObenMotionBrightness, TreppeGangOGLicht_1LEVEL)
end

rule "StairsHallway dimm light"
when
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered SHORT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered SHORT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered SHORT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered SHORT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered SHORT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered SHORT 
then
		toggleLight.apply(TreppeGangOGLicht_1LEVEL, null)
end

rule "StairsHallway toggle light"
when
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered CONT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered CONT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered CONT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered CONT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:1#PRESS" triggered CONT or
	Channel "homematic:HM-Sen-MDIR-WM55:3014f711a0001f58a992fb22:4711:2#PRESS" triggered CONT 
then
		dimmLight.apply(TreppeGangOGLicht_1LEVEL)
end

isBedTime is an Item which is set when going to bed, night time.

Toggle, dim and motion light works well. The timer for motion light is managed by the homematic motion detection devices.

rlkoshak · October 4, 2018, 5:38pm

Excellent write up. There is a lot of good information here. I’m sure I’ll be linking back here often. Thanks for posting!

Some comments to clear up some of the raised topics.

So the reason is that all Items get initialized to NULL. Items get initialized when OH first starts or when OH reloads a .items file. NULL stands for uninitialized and is used to represent an Item that has not received any updates since the Item was created.

To avoid this most of the time, you can use Persistence with restoreOnStartup which will populate the Item with the most recently saved state in the database. I say most of the time because at the time of this writing there is a bug in OH that can allow Rules to start executing before Items have been fully loaded and had their state restored. So it is still a good idea to always check for NULL.

Also note there is another state, UNDEF, which is used to indicate the Item doesn’t have a usable value. By default, the Expire binding will set an Item to UNDEF. This can be used, for example, to clear out old sensor readings from a sensor that has failed and isn’t reporting. How is this useful? For example, I don’t want to have a Rule making home heating decisions based on old temperature data. I’d rather fall back to a different behavior that doesn’t rely on that sensor instead of blindly assuming that it is valid when it could be hours old.

As for the Rule trigger, one way to handle that is to get rid of the from OFF from the trigger. A Switch can only ever change to ON from NULL, UNDEF, or OFF. In all three cases the Rule should trigger so don’t restrict the trigger.

Note that channel triggers are not always available. In fact they are rarely available.

See (OH 1.x and OH 2.x Rules DSL only] Why have my Rules stopped running? Why Thread::sleep is a bad idea for more details.

How did you come to this value? Do you have any advice for how others can determine the best number on their platform?

Also, mention what this does/what problem it addresses. I believe where this has the biggest impact is OH startup times, rule’s parsing times, and reducing potential delays in some Rules running because they were garbage collected meaning the Rule needs to be reloaded and parsed. Correct?

I think you mentioned in another thread that it might be a good idea to add a cron triggered Rule to keep the file’s Rules in memory too.

OH instance?

Are you adding them by editing .items files or through PaperUI?

It is possible that NFS doesn’t report the file events when the files are saved so it doesn’t know to reload.

Though jsondb does not get reloaded on file change. A restart is required for OH to pick up changes there.

As of OH 2.3, a simpler lambda syntax is once again supported.

val dimmLight = [ GenericItem dimLampItem |
    try { 
        ...
]

The Rules DSL will figure out the return type based on the type of the last line executed.

lukics · October 4, 2018, 5:55pm

Thank you for the great feedback, I will update the original article correspondingly (hope this is the way it has to be done)

mstormi · October 25, 2018, 2:56pm

Well as that’s the initial and maximum sizes of the Java heap (not the whole process) that’s too much on a Pi.
I’m running with -Xms400m and have patched that into openHABian as the default. -Xmx possibly can remain.

lukics · October 26, 2018, 4:33am

Well, I adapted the value above.
Thanks for input.

yves · November 7, 2018, 12:55pm

How can I find what is a good size for a pine 64, that has only running openhab?

mstormi · November 7, 2018, 2:36pm

depends on how much memory it has, if 1 or 2 GB then the values for the Pi should do.