@dpa-openhab Maybe you can help me with the problem I’m currently having. It’s about creating the Triggers in the “DSL model”.
Here is an excerpt from my current “work in progress” code:
case "core.ItemStateChangeTrigger":
value = trigger.getConfiguration().get("itemName");
if (value instanceof String str) {
ChangedEventTriggerImpl result = (ChangedEventTriggerImpl) factory.createChangedEventTrigger();
result.setItem(str);
value = trigger.getConfiguration().get("state");
if (value instanceof String state) {
ValidState st = /*createValidStateFromDsl(state); */factory.createValidStateString();
st.setValue(state);
result.setNewState(st);
}
value = trigger.getConfiguration().get("previousState");
if (value instanceof String prevState) {
ValidState st = /*createValidStateFromDsl(prevState); */ factory.createValidStateString();
st.setValue(prevState);
result.setOldState(st);
}
return result;
} else {
throw new SerializationException("Invalid trigger: " + trigger); //TODO: (Nad) Find suitable exception }
The problem is creating a ValidState that is indeed “valid”. The Trigger itself that I get from the factory seems to be fine (this works for “simple” Triggers like system start), but as soon as I need to reference a State, like ON or OFF, I get exceptions when I try to serialize the rule. There must be some kind of “magic” that I can’t figure out, it’s not enough that the ValidState contains the correct string value. It probably needs to reference the actual enum instance or something like that, but I have no idea how to actually achieve that. Trying to debug the parsing to see how they create these aren’t easy.
I assume it’s the same reason you can’t do an import in a Rules DSL Script. Actions | openHAB
Scripts are small pieces of Rules DSL code that can be called from Rules. However, Scripts have limitations. Scripts cannot accept arguments. Scripts cannot return a value. Any reference to a class that would normally need to be imported in a .rules file, the class needs to be referred to by its full package as imports are not supported.
Obviously, I could use the full package name instead of the import, so I’m not saying that such a rule is impossible to translate. But the imports are not allowed.
There are tons on the forum. Most of the DSL rules you find posted will have global variables.
A simplest example could be soemthing like
var globalVariable = "Global variable"
rule "Test rule"
when
Item MySwitch received command
then
logInfo('test', "Global variable is " + globalVariable);
end
My understanding is that here the idea is to create “managed YAML files”, which create rules- rule triggers, script actions, conditiorns, etc in any automation language.
With Groovy/JavaScript/Python there is just one implementation of `javax.script.ScriptEngine . It is provided with input and this input at the same times can contain a set of statements and can create rules, e.g. by addig SimpleRule instances to the automation manager.
I think the direction should be to abolish the difference between “DSL Rules” and “DSL Scripts” by converting the construct “Rule … when … then … end” to an expression and allowing both rules and scripts to just list expressions in any order. This would mean merging the six bundles org.openhab.core.model.{rule,script}{,.ide,.runtime} into three bundles. This would make the code simpler. However I am not sure in case such a change is proposed, that the change will ever be reviewed.
I don’t follow your logic here. A Rule and a script are two very different concepts, a Rule has a certain structure, and can contain any number of scripts, both as Conditions and Actions. A script is just a script, a piece of code. You can “create a Rule programmatically” from a script, which is done when you use e.g. the “helper libraries” to create Rules, but then the Rule and the script aren’t the same. The Rule is what the script creates, not the script itself.
I mean there should be one JSR223-javax.scripting.ScriptEngine DSL implementation, which can execute both Scripts and Rules. This is how it works with JRuby/Nashorn/Jython - the javax.scripting.ScriptEngine implementation handles both scripts and rules. From JSR223 perspective a Rule and a Script is the same - just input (code). All other languages allow creating a rule within a rule: DSL should be able to do the same. This would allow to use import in both DSL Rules and DSL Scripts, because there should be no more difference between DSL rule and DSL script.
I think the new YAML format is the only one that will have the ability to define/use all aspects of an OH rule.
Nashorn and Groovy can use all aspects of OH rules, too, because they are not limited by helper libraries and have to use the OH API directly. Clevet, huh?
Example usage for global variable:
I use global variables to define a list of items. One rule adds or removes items in the array, e.g. based on a toggle for the thing, or when the linked thing goes online/offline. Another rule iterates over all items of that array, when state of sun changes, and for each item performs a check. (If the sun shines, turn all items in the array on. When clouds appear, turn all items in the array off. If I want to exclude an item from the array, I send a command and the item is not in the array anymore).
Cache could be used as substitution for global variables, but I guess it will be slower and not so convenient to type.
I need to figure out how to detect this after the rule has been parsed, not in the “source”. That’s why I need some examples, so I can let the system parse them and inspect the result.
If the APIs are available than yes, all of the JSR223 languages could create a rule with a condition. In my experience, working with the OH API at that level is completely miserable so it doesn’t surprise me that the helper libraries don’t actually expose it. The docs seem to imply that they are available with the ConditionHandlerFactory and all the rest. But the docs are certainly not good enough to actually use those.
but as soon as I need to reference a State, like ON or OFF, I get exceptions when I try to serialize the rule. There must be some kind of “magic” that I can’t figure out, i
Yes, except that they won’t be “managed”, since that means “managed by the system” in OH, which means that they are writable and the user don’t handle their storage.
I don’t understand why you think it would be better to try to force DSL through JSR223. JSR223 is an abstraction, with limitations, that “works magic” for allowing non-JVM languages access to the JVM, but it’s still more limited, and slower, than working directly through Java. Why would we want to impose those restrictions on DSL, that already works using Java?
You could probably create a Rule using the OH API with a “DSL script” too, but why would you want to? The “advantage” of DSL is that it has a defined format that, when provided, is parsed and turned into a rule, without having to call all the different builders and put everything together. If I had to do that, I would certainly choose a language that is “more friendly” than DSL, which I think of as some “troubled bastard breed” of Java and some scripting language like Python. It’s never obvious to me what the syntax should be, and I would never choose to use it for “general coding”.
This is true for all JSR223 languages, with or without helper libraries, and it’s most likely true for DSL as well. The point is that this isn’t very practical, in reality you need to read and understand the Java APIs to effectively use them, and even if you do, it’s quite verbose.
Yes, that’s the kind of use I imagined, and it will be completely thread-unsafe unless you manually use synchronization, since every rule executes in a dedicated thread. This in a way is also the reason why the JavaScripting add-on has introduced the “context lock” that means that only one thread can use a given context at any time. But, this slows things down and makes rules have to wait for each other. Actually, this challenge isn’t special for JavaScript, the thing is just that GraalJS has decided to refuse to use shared objects because it’s thread-unsafe, while others allow it. This has “forced” the application of the lock, but in reality, every use of shared context should have such locking.
As I understand it, this is just what the shared cache is meant for, but it also has its challenges. I don’t think that accessing the variables are that bad/verbose in most languages at least, but you have no way to lock the cache while doing a “transaction” for example, which makes some things impossible to do consistently without potential bugs. I made some suggestions for different cache implementations to address different use cases, but it never went anywhere.
I kind of doubt that it will be meaningfully slower, and it won’t be slower than using the shared context safely, using locks.
The challenge here is that all I have is the Rule instance, I don’t have a reference to the corresponding rule model. So, I can’t inspect e.g DefaultEvaluationContext. I’ll have to test some of the scripts you (both) have posted and see, but at this point, I’m unsure if I’ll be able to detect “shared context” in DSL. It’s easy in other languages because shared context means that it’s a SimpleRule. I doubt that’s the case for DSL, but it would be very convenient for me if it was.
OSGI usually gets in the way, meaning access to all the OSGI APIs. This is why we cannot access nor use Item metadata from Rules DSL. It’s impossible to get to the MetadataRegistry (or at least I was told so many years ago, today? ).
This is why the JS Scripting helper library goes to great lengths to never present a Java Object to you unless you explicitly want one. It’s all JavaScript. This improves compatibility with third party libraries too.
For jRuby, my understanding is its all Java to begin with. But I know less about how all that works.
For Python, the author made the decision to have as light a touch as possible, but in my past experience it’s less problematic dealing with Java Objects in Python.
I want to reiterate that this is only a problem for importing a Rules DSL rule from a file to make it managed or converted to YAML.
Are you trying to go both ways? As I understand it Things do not go both ways (yet?) so maybe this is a problem that can be delayed or skipped.
If you already have a rule that’s managed, it’s already impossible to have a “global” variable so there’s nothing to detect. If it’s already loaded from a .rules file, the user already has the DSL version of the rule. If it’s loaded from a YAML file, it’s the same as managed and the user couldn’t be using “global” variabled in the first place.
I don’t feel like digging into the details just to find the answer, but that’s a “general challenge” that should be there across languages: The way you usually obtain references to managers, registries etc. is to declare (in the Java code) that you need that instance, and then OSGi figures (or doesn’t) it all out for you. But, there’s always the challenge of circular dependencies, and also the fact that if you ask for a reference to something, you yourself will “be held back” from starting until all the stuff you ask for references to have started (it’s not entirely true, you can have optional references, in which case you can start without it, but then you also have to deal with the fact that the reference might or might not be valid).
This system will have its impact on scripting languages as well. They can’t “ask for” any of these, so I assume that there has been made some arrangement to have the “scripting engines” be assigned refererences for “the most common” registries/managers. DSL, being deeply wound into this dependency hierarchy itself, might have particular challenges (with circular dependencies or having to start before something else) that might not apply to the other scripting languages because of they they are “coupled” with OH.
But, there is another way to get instances, you can ask for them programmatically. I think that is possible “at any time”, but I haven’t really studied how you do this very much, so let me remain vague. I would think though, that you might be able to get the registry that way.
My starting point is to try to mirror what has been done for Things and Items. That means going both ways, because you can edit the rule either from the UI controls, or from the “Code” tab. Every time you switch tab, the conversion must take place (if anything has been modified). Since file based rules are read-only, there’s no need to do the conversion in their case, as they can’t be modified. So, you only need to convert Rule → YAML or DSL for read-only, but both ways for managed. So, here it “solves itself” because those with shared context will be read-only.
But, there’s also the “export”/copy definition button at the bottom (don’t remember the exact phrase). This is where I need the check. That button will have to be hidden for all SimpleRules, because even though they could technically be processed, and a file generated, the resulting Rule when parsing this content would be invalid (the Action would be missing). Similarly, I think the button should be hidden for DSL rules with shared context, if at all possible, because the resulting Rule when parsed back in wouldn’t work either.
I don’t think it’s user-friendly to let them generate “useless” YAML files. If they copy what they see in the “Code” tab and use that, I guess they’re on their own. For SimpleRules we actually hide the “Code” tab altogether, because it was decided that it would only be confusing to see what was shown there (essentially an incomplete Rule). I guess the same argument could be made for “shared context” DSL files.
To compensate, I made the “Source” tab that is shown if the source is embedded in the Rule instance. But, this must be done by the code that “creates” the rule, which means the helper library. I’ve tried to figure out a way to do this for DSL in the past, but gave up figuring out how to acquire the “source text” at the point where the embedding would need to take place. I’m betting that’s possible though, I just grew frustrated with it all - and don’t know if anybody would appreciate it that much anyway. But, by “embedding” the source for DSL rules, you could see “the whole file” with context and all rules in the “Source” tab in the same way as for helper library created rules.
I don’t understand why you think it would be better to try to force DSL through JSR223. JSR223 is an abstraction, with limitations, that “works magic” for allowing non-JVM languages access to the JVM, but it’s still more limited, and slower, than working directly through Java. Why would we want to impose those restrictions on DSL, that already works using Java?
JSR223 offers interface how to plug-in (insert) automation implementations. The JSR223 interface allows writing automation in Java language by the users and without JSR223 writing automation in Java by the user would be more complex to implement.
OSGI usually gets in the way, meaning access to all the OSGI APIs. This is why we cannot access nor use Item metadata from Rules DSL. It’s impossible to get to the MetadataRegistry (or at least I was told so many years ago, today? ).
This should be possible, and access the ThingManager (to enable or disable things) too, by adjusting org.openhab.core.model.script.ScriptServiceUtil.java.
It might be that we use the term “DSL” wrong, but Xtext, or whatever it is, is definitely used to define the file formats. All the “traditional file formats” for rules, things (channes++) and items (metadata?) use this. To me, from looking at the rule code, it doesn’t appear to be “a clear separation” between this use and “DSL scripts” either, but even if there is, I very much doubt that the file formats are affiliated with javax.script.ScriptEngine.
In openHAB everything, which uses Xtext, is called DSL. Xtext is a parser and lexer, like javacc, bison/yacc/lex/flex. .things, .items, .sitemap, .persist files are parsed with Xtext and are called DSL (Domain Specific Language).
XBase defines a java-like language, which allows writing program logic. Under the hood, XBase uses Xtext. “DSL Scripts” use XBase and implement JSR223. For items, .things, .persist, .sitemap files JSR223 is not implemented.
Back when I last looked into this for Rules DSL, Rules DSL didn’t have access to one or more of the classes or Objects needed to do it, and it couldn’t just be imported. So you couldn’t declare that you needed it in the rule.
I don’t remember what specific class or call where it broke, just that it had something to do with OSGI. I think it was trying to use FrameworkUtil to get the bundle. For anyone who wants to try, example code to get to the Item MetadataRegistry from Nashorn is as follows:
That should be enough to show the Java classes involved.
I was always against that decision. I don’t like to hide things from users, especially when it creates confusion (e.g. what’s the difference between a Script and a Rule? Nothing really except the UI treats them differently).
I originally made the PR so that it showed the “Code” tab also for these - but during the review, I was told to hide it. Since I agree that there’s very little you can actually use the information in the Code tab for in the case of SimpleRules, I didn’t really resist.
They are treated differently though, there is a “hack” in the code that makes SimpleRules work at all. Their Action is missing, or its content is missing. Instead, it contains a reference number. When a SimpleRule is registered, a pointer/reference to the compiled action is passed along (it’s just a memory address that will be different every time it’s compiled). OH assigns a number to this reference, and stores it in a map. It then injects this reference number into the action’s configuration in the rule. When the rule is executed, OH looks for such a reference number, and if found, it doesn’t execute the action normally. Instead, it simply invokes the code pointed to by the map using the reference number.
As a result of all these “hacks”, the Code view is quire confusing, because the action only contains this reference number. Most people have no idea how this is done or what the reference number means, and frankly, I’m not sure it’s worth it trying to explain it. They can’t really do anything with it anyway, the reference number means nothing and is different every time. The action itself isn’t in the rule at all, the rule just “blindly invokes” the referenced compiled code/function.
At the time we also couldn’t add tags, change the status of a rule from a “Script” to a “Rule” because I accidentally added the Script tag to one of my rules, etc. We can do all that now (I don’t remember when that was added) so those technical limitations are not so concerning anymore.
But it still confuses end users. To this day I help users who get confused or assume things about Scripts that are incorrect (e.g. they must write their Script Actions as “Script Rules” and configure their “real” rule to call the Scripts.
The term “Script” is horribly overloaded in OH which doesn’t help.
And the same arguments would apply to Scenes and you can get to the code tab for those. It’s inconsistent. We are not concerned about seeing incomplete rules there? I don’t care enough to push hard about the issue though.
I’m sure there are reasons for this, but from the end user’s perspective, there is nothing functionally different between defining a Script under Settings → Scripts and defining a rule with a single inline Script Action under Settings → Rules. It’s too late now to change it, but it seems like the different way it works is unnecessary.
Ok, this is what I called “find the instance programmatically”. I thought that should work regardless, but I’m sure there’s some reason why it doesn’t. I’m just not “enough into” all the scripting stuff to properly remember, but isn’t there “presets” or something that provide references to some of the most common registries? Are these not available through DSL, or is my memory way off?
As I said, I originally made it visible. I’m “full transparency by nature”, but I didn’t really have any good reason to push back. The “Code” tab was shown also for SimpleRules in the past, but after I made the “Source” tab, I guess some felt that it was a bit too much with 3 tabs, and “hard to explain” exactly what the difference between “Code” and “Source” was?
It may work now in Rules DSL. I don’t know. But it didn’t used to (OH 3 timeframe was the last I messed with trying to get it to work with Rules DSL).
Indeed, but the presets that are available in Rules DSL and those available in the other languages are not the same. It seems something additional needs to be done beyond adding something to the preset to make it available as an implicit variable in Rules DSL. Currently, based on the docs, the ItemRegistry, ThingRegistry, and RuleRegistry are included in the default preset. But Rules DSL doesn’t have direct access to any of them.
For the other languages, Links, metadata, widgets, and all the other registries need to be acquired separately. Sometimes the helper library does this for you, sometimes it doesn’t.
JSR223 presets are ignored in DSL Scripts: model/script/runtime/internal/engine/DSLScriptEngineFactory.java:public void scopeValues(javax.script.ScriptEngine scriptEngine, Map<String, Object> scopeValues) {} does nothing, and in model/script/runtime/internal/engine/DSLScriptEngine.java:public void put(String key, Object value) {} also does nothing.
actions.config.type should be a MIME-type returned by the JSR223’s ScriptEngineFactory.getMimeTypes(). At least this is how UI rules are stored in jsondb So instead of DSL → application/vnd.openhab.dsl.rul. Or what is the plan in the current form to map DSL and JavaScript to the right JSR223 engine factory?
For me in the above examples is unclear why for multi-line YAML after script: once is used literal style | and once folded style >.
).