Avoiding delays as rules recompile

Hi all

openHAB V3.2 running on Windows
All config done using files
99% off functions done via MQTT

I have noticed that when a rule runs a while after a rule in the same .rules file was run, there is a pause which ranges from 2 - 8 seconds) as (I believe) the .rules file is recompiled. The delay is definitely within openHAB.

This is VERY painful if you are using remote light switches, monitoring IR codes etc. and it removes vital spouse approval of the system.

A previous poster (thank you Arno - ā€œAFromDā€) suggested a work around where you add a rule which does something pointless every minute, thus ensuring the the rules stay loaded in memory.

e.g.

rule ā€œKeepRulesInMemoryā€
when
Time cron ā€œ0 0/1 * 1/1 * ? *ā€
then
DoNothingItem.postUpdate(ON)
end

This is nasty, but it worksā€¦

Has anyone found another way to beat the delays? Have I got something wrong?

Thanks
Barry

Rules only get compiled when theyā€™ve changed or when theyā€™re bad. If thatā€™s happening to you every time itā€™s likely the latter.
If so you should be seeing error messages in openhab.log assuming proper logging debug level.
The former may happen inadvertently when file access properties change. Not sure about your windows setup in that regard. Consider openHABian on Linux/Raspi.

Hi Markus

Thanks for the reply. Much appreciated. Thereā€™s nothing wrong is the rules and no error messages in the logs, and they all function perfectly, just with a delay, sometimes.

File access properties may possibly get changed by backups etc.

Do compiled rules get moved in and out of memory to make space? that might add a delay? If yes, can I increase the amount of memory allocated for them, to reduce the frequency of space recycling? Please assume I have infinite amount of RAM available.

I originally ran on Raspian for a couple of versions (over 4 years) but moved to Windows after various Pi related reliability issues - and I donā€™t regret it at allā€¦ Nothing would drag me back to Pi. (at least for this use - I have others in use as HABPanels, Squeezeboxes and CCTV monitors). They are great, but not for something mission critical. I like my VMā€™s, raid disks, dual redundant power supplies and dual redundant UPSā€™s - and no SD cards or USB power suppliesā€¦ So, my setup is rather more server farm than most peoples. Itā€™s there running already for work, so the additional power for another VM is minimal.

I do love the way that the OH files configuration can be moved between platformsā€¦ and having the key information in files would make the transition to another platform very easy (not that I intend to) Everything is via MQTT, most recently some Zigbee stuff via the Zigbee to Tasmota project running on a Sonoff Zigbee Bridge Pro device. Itā€™s really nice. The one downside of my server based Windows setup Iā€™ve found is the inability to plug a Zigbee or ZWave dongle into the openHAB server, however having played with ZWave in OH3.0 on Pi, I have decided I like externalizing the Zigbee handling, so itā€™s a lose that pushed me towards a win. Iā€™ve also played with Zigbee2MQTT project, which is very, very good, but requires a Pi or a Windows box with a real USB portā€¦

I rambleā€¦

Cheers
Barry

Your OS manages ram not applications or both which is annoying

Doesnā€™t Java manage itā€™s own RAM usage etc ? Official Java certainly does

Iā€™m not familiar with the Azul Zulu JDK which is the OH recommended Java for Windows installs. I might google that :slight_smile:

1 Like

Yep, and you will not be finding understanding advocating use of PCs and Windoze in particular.

BTW openHABian used to but no longer uses Azul java, one of many ā€˜collectiveā€™ experiences shaping openHABian to now be a stable 24x7 basis to OH users. Thereā€™s others such as SD mirroring to compensate for RPi weaknesses.
FWIW, you donā€™t need to use a Raspi, it is also available on debian on x86. Iā€™d suggest to at least move over there, Windows is a PITA. Java on top even more so, thatā€™s quite unexplored territory and I believe thatā€™s what youā€™re right now getting hit by. Not many (if any) that can help you here.

To stay on topic, yes this delay-after-a-period-of-disuse was a real measurable effect in OH2. Not clearly understood, to my knowledge, not even as to whether its about ā€œrecompilingā€ or ā€œswapping outā€.
It may well have to do with the host environment (Windows, Java version) but openHAB remains supported in many environments - brushing it off as ā€˜oh Windozeā€™ is not helpful.

OH3 comes with a ā€œnew rule engineā€ hoped to cure many ills - but I do not think you are the first to observe this old delay effect under OH3.

Of course in OH2 days more of us used DSL from xxx.rules files, and that is no longer true. This might be the key.
As you can see the delay problem, you could usefully experiment by replacing an affected rule by a GUI version. The rules are managed in a different way that might be unaffected. (It might even be worse!)
(There are just a few pitfalls revolving around if your files-based rule uses shared ā€˜globalsā€™, typically timers, try to choose another rule for testing)

You might even make a dummy GUI rule triggered by the same event as one of your ā€˜problemā€™ rules, just to make a log entry so that you can see/compare timing.

I do not think you need anything like every minute.

When I reviewed a fairly extensive OH2 system for this noticeable effect, I think I only needed to add a ā€˜keep aliveā€™ rule to one file, set it for ten minutes maybe. Carefully juggling rules between files that used ā€œnaturally occurringā€ frequent events like sensor updates took care of most files.

1 Like

Hi rossko57

Thanks for your helpful response. Much appreciated. Itā€™s good to know that OH remains supported in many environments,ā€¦ belittling the choices of users is never going to help anyone or win the platform new users. Itā€™s also good to know that others have seen and investigated the effect before.

I will try the GUI rule to see what happens.

Thanks
Barry

This statement confuses me because the same can be said for all of OH configs, not just text configs. Thatā€™s not a knock against using text based configs but I find leaving statements like this unaddressed creates FUD when it comes to UI configs. All the UI configs are stored in text files too and are just as easy to back up, port, check into source control and all the rest as .items et. al.

But back to being on topic, 2-8 seconds is huge. Unless youā€™ve written poor Rules DSL (i.e. over specified types and lots of forced use of primitives), even if the rules file is being reloaded it shouldnā€™t take that long. But I donā€™t think thatā€™s whatā€™s happening. You would see ā€œloading model blah.rulesā€ in the logs if it were reloading and reparsing the .rules files.

@rossko57 is probably the biggest expert on this rules delay topic though. There were a number of very extensive threads where this was explored pretty deeply.

The UI rule experiment is a good one. I donā€™t think Iā€™ve seen this problem reported with UI rules. However, there might be a slight delay the first time a given rule is triggered because the script actions and conditions are not actually parsed until that first trigger. But that delay should be much less than a second unless you have poorly written Rules DSL code.

My worry is this could be a pervasive problem, that is mostly hidden. No-one will notice (or care) if dusk-to-dawn lighting is some seconds late.
Itā€™s only if you get up for a pee in the night and the bathroom light is sluggish coming on that it would get noticed. We humans notice 2 secs there - but then just grumble, and it works fine next time.

The GUI rule comparison should be interesting - it wasnā€™t possible to do like-for-like in OH2, but can get closer in OH3.
Really Iā€™m expecting DSL GUI to be well behaved, and this is some files management issue.
If not, seemingly DSL linked - a comparison to say a javascript GUI would be in order.
Or, if seemingly file linked, a test with javascript rule from file.

Not that I think anyone is interested in fixing it, but if we understand the scope we have the avoiding recommendation. Itā€™s all vague now.

2 Likes

I have a bunch of switches that trigger some dumb devices via IR where I cannot capture the state. Each switch has then been implemented as 2 switches - an on and an off switch with a rule that says ā€œwhenever one of these change to on, set them both to offā€.

I donā€™t think I have ever noticed a delay with the rule setting the switches back to off.

This is of course highly subjective and most of the time the interaction is via alexa so I donā€™t actually look at the switches in the UI, but when I have used theUI, it has never been an issue.

rule "Living room - dumb AC switching" do
  changed AC_COLD, to: ON
  changed AC_WARM, to: ON
  run { all_off [AC_COLD, AC_WARM] }
end

I experience a recompile delay too - but only once after I changed a rule, and it is far away from the time you experience.

To get some numbers I did a measurement.
Itā€™s a reed contact ā†’ MQTT ā†’ OpenHAB (Rules) ā†’ MQTT ā†’ Relay (primary side) which I measure here.

Normal (compiled) behavior:
image
80 ms - which is good enough and not really noticeable. This is quite reproducible (had 56 ms and 72 ms. Rarely I get values of 160 ms.)

Behavior after a rule has changed (once):
image
Around 820 ms - this is noticeable and varies a bit (had shorter times around 504 ms too).
So recompile takes around 740 ms

Technical information:
Debian GNU/Linux 9.13
Zulu 11.0.14-1
OpenHAB 3.1.1-1
conventional (spinning) HDD
Rules file: 59 lines, 4 rules

Hi all

Thank you for the replies, comments and tests.

I took on board Rickā€™s notes about bad DSL ā€œover specified types and lots of forced use of primitivesā€ (always the font of wisdom) and re wrote my rulesā€¦ which had evolved over several years.

This definitely has made them snappier, but only after any compile delay. Still, worth the effort (Thank you, Rick)

I tried a comparison against a GUI built rule. There was milli seconds difference between the two, even after leaving them dormant for 24 hoursā€¦ in fact the file based rule often won.

So whatā€™s different between that and my day to day rules. The clear difference I can see is in the size and complexity of rules files.

I have 185 rules - spread across 10 files (by room) Some rooms have 3 rules, some have 30 complex rules. Iā€™m wondering if the garbage processing only effects the bigger files. Some of the bigger rules files will go unused for 8+ hours, every day. I want to avoid 184 individual rules files.

Iā€™ve added a pointless action every 30 minutes into the time critical rules files and that does provide a solution a real problem. Itā€™s not pretty, but it works and doesnā€™t require anyone to expend energy on old codeā€¦

Cheers
Barry

1 Like

Iā€™ve never run OH on Windows, only Debian and Ubuntu, so Iā€™m not sure if this is possibleā€¦

In my installs I always increase the amount of memory available to the Java OH process by adding the following to /etc/default/openhab:

EXTRA_JAVA_OPTS=ā€œ-Xms1024m -Xmx2048mā€

Maybe if you can increase the amount of RAM available to OH then rules wonā€™t get paged out? (If thatā€™s whatā€™s happening.)