Rule execution lags, possibly Java GC

@milo

To to summarize, you’re getting a delay on actions/rules processing…not just on the 1st time a rule runs, but intermittently after the rule hasnt been run e.g. the system has been booted for 24+ hours… the rule you are experiencing the delay with HAS been run before in that 24 hours… but now its being slow/delayed again?

If thats what you’re experiencing, obviously there are a ton of variables involved e.g. have you a massive rule-set etc… but for the purposes of this reply, Ill just assume theres nothing too special going on… that your device doesnt have a lack of memory or is busy doing much more than running Openhab… But those are things you would need to consider.

The seeming root cause with my setup was Azul Java v8 not having enough memory given to it to keep all my rules stored in memory… so this would result in garbage collection occurring … which means the rule is removed from memory and has to be loaded from disk again (causing the delay).

In my case, the rules would be removed from memory because either:

  • Java didnt have enough memory to store all rules being used, so had to swap them in/out of memory.
  • They rule in question just hadnt been used in a long time so was dropped out of memory for memory management purposes.
  • Java v8 wasnt as good at memory management as later versions of Java seemingly are.

So… Things you can try:

  • Currently, at the bottom of this long post, you can see we are testing later versions of Java (v11) which Marcus is involved in the testing/approval process of that. So you could try out a later version too (instructions are a few posts up from this one). Please remember though, this is a test, so you could experience other issues.

  • Use the XMX and XMS settings to increase your java memory available. You could also use this with the Java 11. Certainly this is what I am doing and things seem to be better for me. It may provide you some improvement with Java v8. Suggested settings are in the above posts as well as how to implement it (just a quick edit of a config file).

  • There are other settings which are listed in the above posts, and you edit the same configuration file, these basically alter the way that Java Garbage collection works… and I did have some successes with these. I at least reached a satisfactory state with Java v8… though I am now on testing Java v11 because its (in my view) just a better version of Java… and it seems to be working better at this time.

You will always get the initial delay on a reboot where the rules are loaded for the 1st time, but after that, things should be smoother.

My recommendation overall, would be to check you have spare memory and your system isnt overloaded with other things taking cpu/memory…and if all looks good there, Id suggest change the EXTRA_JAVA_OPTS settings to add your XMX and XMS settings to something like 400 800 (look at the above posts for the my settings)… try that on Java v8.

If that doesnt relieve things and you dont have any other issues with trying it, Id go to Java v11 and keep in the XMX and XMS settings and feed back to Marcus if you still have problems.

EDIT: Just adding this to the post
EXTRA_JAVA_OPTS in the /etc/default/openhab2 file.

EXTRA_JAVA_OPTS="-Xms400m -Xmx800m"

Something like the above is quite a lot of memory to give Openhab on a small setup (in my view)… but it might be a good place to start testing and probably work your way in 50MB chunks if it resolves things e.g. -Xms350m -Xmx750m etc… If the 400 and 800 have no impact… def try the Java v11 with these settings… otherwise, assuming no other cpu/memory issues… youre looking at adding other Garbage collection settings and checking there are no issues in your rules.

Thats my thoughts at least.

Thanks