Problem: Rules can take 10-20 seconds to trigger if they not have triggered very recently. My theory is that a JIT compiler runs over them before they are executed, and after some time of not using it, the result is purged from memory. Thus it has to be compiled again, which takes time on my Raspi (3+).
My solution: I put this in the rule file that I want to prevent from being recompiled:
rule "keep this thing in memory"
//use the compiled code in this rule file every minute
when
Time cron "0 0/1 * 1/1 * ? *"
then
logInfo("Motionrules","cron execution to prevent gargabe collection of compiled rules")
end
It triggers every minute. Any thoughts on this? I can even replace it with an even less expensive function, and will cost next to nothing in term of processor time. It will block some memory, but it is for stuff that is needed instantly and the motion sensor stuff also triggers regularly, so it will live in memory often.
Seems to work for the hour I have been testing it.
Even trying to fill up the memory for the whole minute between calls by getting a list of items, addons, controls in paper UI, calling the classic UI, it doesn’t seem to be purged from memory. Rule triggered in about 0.02-0.05 seconds after the event.
You could also simply increase the Java heap size to affect all rules files in one go.
Add options -Xms400m -Xmx512m to e.g. EXTRA_JAVA_OPTS in /etc/default/openhab2
Does that keep all my rules in memory over night? There are situations where the rule is not triggered for many hours, but still needs to start up immediately.
I am also running Grafana and Innodb on the Raspi, wouldn’t that reduce available memory to them? Or is that not likely to affect them?
I played with those memory settings, but after applying the cron rule to all my rules files I see I need to krank them up. Top now shows RES memory consumption to be 2.2GB. I only set 1.5.
Very interesting, I am also facing the issue of long delay in rule triggering. Will be very happy to know what is the final conclusion and how to reduce triggering time
To any others trying this out, all rules must have a unique name! (as the spec also says)
I noticed having put several cron rules with the same “keep this thing in memory” name in different .rules files that only one of them were firing.
If I’m happy with the performance with all the cron rules, I will try out -Xms2200m -Xmx2500m (sorry pi’ers) and take the cron rules out, but it will take a few days. Will report back.
Yes the values I gave were for Pis. For 64bit x86 you need to about quadruple them. Be aware that it doesn’t prevent the OS from paging out unused rules/pages so the RES value of the Java process is a good orientation but not the exact figure, then again paging back in is fast.
What’s taking so long is recompilation of rules. Java throws out compiled code on garbage collection so when you’re in need of such a dropped rule OH needs to parse it (which takes even a lot longer on Pi than on x86).
Note I wouldn’t be sure your extra cron rules help as it’s not them you want to remain but the other rules in the same file. OH/Java doesn’t do keep-or-drop-all-or-nothing. If it did, any rule execution would prevent garbage collection from taking place. And you would encounter total lock downs on recompiling everything.
I can’t really discuss on what Java does or doesn’t do. But after a few days of looking at my logs, I can’t see any delays any more, without changing the heap size. I have the rule only in one of my rule files, but I also can’t see any delays in rules from other rule files any more. So to me, it does seem that Java does garbage collection on all the rules as a whole, unless some rules are used regularly.
It could also be that something else is going on, which has solved my delay problem, I’ve just tested this idea and it seems to work.
My code is now completely useless, but it seems that the compiler still compiles and runs something:
rule "keep this thing in memory"
//use the compiled code in this rule file every minute
when
Time cron "0 0/1 * 1/1 * ? *"
then
return;
end
I’m wondering about the JIT interpreting/compiling theory. Based on known Timer behaviour.
Example;
var Timer myTimer
rule "context test"
when
System started
then
myTimer = createTimer(now.plusMinutes(20)) [
logInfo("testing", "executed")
]
end
If you edit and reload the rules file after 10 minutes, the timer will carry on running and then fail after another ten minutes with a kind of null error.
. I do not know, but guess the error is because the myTimer variable has been destroyed and recreated by the file reload. Maybe it’s because the execution block has been destroyed. But the orphaned Timer has lost it’s context.
I speculate that a complete rules file cannot be ‘recompiled’ during normal running without this happening in the same way, and it doesn’t seem to happen? ( I’m a poor test case as I host OH on more capable boxes than most. )
Raises two possibilities:
It’s not a recompile/reparse thing going on, it’s just swap/paging out thing of precompiled code.
or - “The system” is smart enough to know it needs to preserve the context for the running Timer, and locks out the timeout/dumping mechanism.
This is not to shoot the general theory down, just understand better. It’s not just the code in play, there’s the whole context of variables etc. Are they always retained in memory?
There are the possibilities that the parent rules file, or the parent rule, or the timer’s code block could get “rolled out” of memory while a timer is running (actually just Quartz waiting for an alarm datetime I think)
It would take further experiment to find out more detail. There may be another method to retain ‘rapid response’ involving Timers.
Having the above piece of code in one of my .rules files seems to have done away with the multiple second delays I was experiencing. Now all rules react within tenths of seconds. Even saw one case where a rule was putting something in the log before the triggering event was logged.
So if you are experiencing multiple second delays on a Raspi 3b+, I think it is safe to say that making sure a rule runs every minute keeps everything readily available in memory.
I have added this code to all my rule files and have been running it for about a 3 weeks now, and haven’t experienced any delays. So it seems it have done the think, thx u
I think you need it only once, not necessary to put it in all your rule files. Or if you have a rule that actually does something useful every minute, also good. I have a rule that triggers every five minutes (on sun position update), that wasn’t frequent enough.
