On Concurrency
In OpenHAB/JSR223
Concerning concurrency in scripting (rule) engines I created rules_overview.md: elaborate about concurrency support in rule engines by dilyanpalauzov · Pull Request #2565 · openhab/openhab-docs · GitHub , stating that for GraalVM (JavaScript and Python) only a single rule or transformation can be executed at any time. Is this correct?
For clarification, if automation/jsr223/seven.py (or seven.js) installs seven rules and automation/jsr223/three.py (or three.js) contains three rules, does the implements Lock in org.openhab.automation.pythonscripting.internal.PythonScriptEngine (and org.openhab.automation.jsscripting.internal.OpenhabGraalJSScriptEngine) ensure that at most one out of ten rules are executed at once, or does it mean, that one of the seven rules is executed, and at the same time one of the three rules can be executed? That is, does splitting one file in automation/jsr223/.(js|py) into many, independent files, each of which installs a rule, increase the parallelism?
Locks in threads, explicitly started from a automation/jsr223/.java file
The current statement is that for the files automation/jsr223/.java allowInstanceReuse cannot have side effects, as these files are executed only once. But these files can install rules, contained in the jsr223/.java files, and rules can be executed in parallel. For me it is unclear, when a jsr223/.java file installs and thus contains a rule, is then allowInstanceReuse relevant for the rule inself, but irrelevent for the jsr223/.java file?
Locks in Rules
As an example for the above, I have a DSL rule, which changes the color of a sitemap item, depending on how long a device is on:
Text item=e_power labelcolor=[e_color=="yellow"="yellow",e_color=="gold"="gold",e_color=="olive"="olive",e_color=="orange"="orange",e_color=="red"="red",e_color=="maroon"="maroon"]
I have the items DateTime e_last_change, String e_color, Number:Power e_power.
var eLock = new java.util.concurrent.locks.ReentrantLock
var Timer timer
val ir = org.openhab.core.model.script.ScriptServiceUtil.getItemRegistry
rule "E Power Change"
when
Item e_power changed
then
if (previousState != NULL && Math.abs(e_power.getStateAs(Number).intValue - (previousState as Number).intValue) > 10) { // the device is on, if it uses 60 W
val prev = (previousState as Number).floatValue < 60
val cur = e_power.getStateAs(Number).floatValue < 60
if ((!prev && cur) || (prev && !cur))
e_last_change.postUpdate(new DateTimeType)
}
eLock.lock
try {
if ((e_power.state as Number).intValue < 5) {
if (timer !== null) { timer.cancel; timer = null }
if (e_color.state != NULL) e_color.postUpdate(NULL)
return
}
timer = createTimer("Set E color", now.plusMinutes(4)) [|
eLock.lock
try {
val last_mod = (e_last_change.state as DateTimeType).getZonedDateTime(ZoneId.systemDefault())
var b = ZonedDateTime.now
for (StringType c: newArrayList(new StringType(''), new StringType('yellow'), new StringType('gold'), new StringType('orange'), new StringType('maroon'))) {
b = b.minusMinutes(4)
if (last_mod.isAfter(b)) {
if (timer !== null) timer.reschedule(now.plusMinutes(1))
e_color.postUpdate(if (c == '') NULL else c)
return
}
}
} finally {
eLock.unlock
}
if (e_color.state != "red") {
e_color.postUpdate("red")
timer = null
}
]
} finally {
eLock.unlock
}
end
The above starts a thread from a rule and then loops and sleeps in that thread, changing the color in each iteration. I do not remember the exact reasons, but I had to use locks to get this right. If I convert the above to a Java223 rule (automation/jsr223/e_color.java), provided that allowInstanceReuse is not relevant for automation/jsr223/*.java, do I still have to use locks, irrespective of allowInstanceReuse setting?
Another example for possibly required lock in a rule - Rules · openhab/openhab1-addons Wiki · GitHub - “If a rule triggers on UI events it may be necessary to guard against concurrency.” … in DSL. How about in Java223?
Removing allowInstanceReuse option from Java223
I think the multi-threading considerations in transformations and rules are all the same for all JSR223 rule engines, with the exception of GraalVM (Python and JavaScript), which are single-threaded. For all the other rule engines users have dealt so far fine with multi-threading/locking. For this reason I suggest removing the allowInstanceReuse option. If multi-threading in rules requires clarifications to get right, I think the right place is to describe this at JSR223 Scripting | openHAB or Rules - Other Technologies | openHAB. Honestly, having a choice between writing OH rules in many languages, who is at the same time going to choose Java and have difficulties with concurrent programming? The option allowInstanceReuse is only meant for those people, who choose Java, write code, which shares values between executions, and at the same time cannot programme threads.
My current understanding, based on the current README.md on github (last changed by me), is that allowInstanceReuse allows sharing values between invocations, but only when a value is shared between invocations, there can be multi-threading issues, so in this case - when allowInstanceReuse offers advantages - allowInstanceReuse should not be used. If this option is not removed, having an example in the add-on documentation, explaining when a problem with allowInstanceReuse can happen, and how this problem can be solved (no allowInstanceReuse or demonstrated thread synchronization) would be very good.
Loading automation/lib/java files
The log is erroneous. I changed it to
Received ‘{}’ for path ‘{}’ - ignoring (wrong extension or unused event kind)
I see no changes uploaded on github. In the case, where a user puts a file at automation/lib/java/u.java this file should not be ignored, until the newly created file is changed, but instead read. Currently it is ignored.
Inconsistent logging of syntax errors
I agree that openhab-core logs inconsistently ScriptExceptions (syntax errors) - sometimes logs and sometimes does not log. For transformations in channels I have opened ChannelTransformation: log exact syntax errors in transformations by dilyanpalauzov · Pull Request #5043 · openhab/openhab-core · GitHub. It does add special handling of ScriptException from TransformationException, but also says, that syntax errors in transformations, which are not from a rule engine, and originate e.g. from the JSONPATH transformation, are still not logged.
About dependencies for transformations vs scripts
Changing a lib dependency SHOULD force recompilation of script that declares depending on this library. It works for “standard” script (i.e. I tested a script created from the GUI in the “script” section, and it runs every time with updated lib), but it doesn’t work for a transformation script.
I debugged a little, and I found that transform scripts are not registered in the dependency tracking system, at all. …
I don’t use other JSR223 services, but by reading code I’m pretty sure it won’t work for them either.
I filled at Transformation with ScriptEngine do not have dependecy change and do not recompile on dependency change · Issue #5046 · openhab/openhab-core · GitHub that in case of JavaScript(GraalVM) changing dependencies of a transformation does not update the transformation, until the source code of the transformation is updated.
Here a different test:
I create automation/lib/java/u.java with content
public class U {
public static String v() {
return "R";
}
}
and under automation/jsr223/m.java I put:
import java.util.*;
import org.openhab.automation.java223.common.InjectBinding;
import org.openhab.core.automation.Action;
import org.openhab.core.automation.Trigger;
import org.openhab.core.automation.module.script.rulesupport.shared.ScriptedAutomationManager;
import org.openhab.core.automation.module.script.rulesupport.shared.simple.SimpleRule;
import org.openhab.core.automation.util.TriggerBuilder;
import org.openhab.core.config.core.Configuration;
import org.slf4j.LoggerFactory;
public class M {
@InjectBinding(preset = "RuleSupport", named = "automationManager") ScriptedAutomationManager automationManager;
public Object main() {
automationManager.addRule(new SimpleRule() {
@Override public List<Trigger> getTriggers() {
return Collections.singletonList(TriggerBuilder.create().withId("trig1").withTypUID("core.ItemStateChangeTrigger").withConfiguration(new Configuration(Collections.singletonMap("itemName", "r"))).build());
}
@Override public Object execute(Action module, Map<String, ?> inputs) {
LoggerFactory.getLogger("logger1").warn("U is " + U.v());
return null;
}
});
return null;
}
}
When item r changes, a message is logged: U is R. Afterwards I change in u.java R with T. When item r changes again, the old message U is R is repeated, instead of the new one U is T.
Private-Package: freemarker.ext.ant,freemarker.ext.rhino
After Exclude directories freemarker/ext/{rhino,ant} from the bundle by dilyanpalauzov · Pull Request #9 · dalgwen/openhab-addons · GitHub the Java223 jar file does not include the directories freemarker/ext/ant/ and freemarker/ext/rhino/. But org.openhab.automation.java223-5.0.1.jar:META-INF/MANIFEST.MF contains Private-Package:freemarker.ext.ant,freemarker.ext.rhino,…. I tried utilizing org.apache.maven.plugins/maven-dependency-plugin to avoid in first place extracting the two directories under bundles/org.openhab.automation.java223/target/classes/freemarker/ext/rhino/ (or ext/ant/), but I had no success with this.