OH 3.2 new JS scripting - time to leave Rule DSL?

[Oliver2 (Oliver)]

I want to re-start this topic and like to hear your thoughts whether it is time to leave Rule DSL especially in the context of what Kai Kreuzer posted yesterday here (blog on openhab.org):

“Javascript Script Engine
In openHAB 3.2 we are proud to announce a greatly enhanced version of the GraalVM based JavaScript engine, JS Scripting (opens new window), which brings the same ease of use and powerful built-in library of the Rules DSL, but in a modern ES2021 version (opens new window)of JavaScript that will be familiar to power users and novices alike. The enhanced JS Scripting add-on brings together several existing projects into a single, easy to use automation environment.”

Furthermore I read posts about Xtend like the future maintenance of Xtext is a risk. or Bye bye Xtend, Welcome Java

I know that Rule DSL will be supported by openHAB probably for some more years, that there are many many code examples here on this forum (as always there are pros and cons).

As of now I have the impression it is about time to leave Rule DSL and move on to JS or Java (because it is kind of native scripting language to OH). As this “impression” is by far not profound because I am not a programmer I’d like to add some facts from you to my impression.
As said, this topic has already been discussed in other threads so let’s focus on the new aspects to the topic in relation to Kai’s blog.

[rlkoshak (Rich Koshak)]

@Oliver2

Kai Kreuzer posted yesterday here (blog on openhab.org) :

That section of the blog post was written by @digitaldan who did the bulk of the work to push JSScripting over the hurdle to make it a good choice for both UI rules and text based rules. But all due credit also goes to @jpg0 who wrote the JSScripting binding in the first place as well as the helper library that the add-on now comes with.

First let me say I speak for no one but myself. Everyone seems to think I have some sort of authority I do not possess on threads like this.

I would personally have listed Rules DSL as deprecated as of the release of OH 3.0. Deprecated means “it’s probably going to go away at some point.” Nashorn ECMA Script 5.1 as well as Jython are in the same category.

If you are just now getting started with OH, do not choose any of these three languages for rules.

But what if you already have a bunch of stuff written in these rules?

Nashorn ECMA Script 5.1

You need to start looking now about how to migrate your rules to JSScripting. It’s not actually that difficult to make your existing scripts work for both Nashorn and JSScripting though you would be much better off refactoring your rules to take advantage of the library that now comes with JSScripting which will greatly simplify your rules and bring them closer to standard ECMAScript.

It is now possible to run both Nashorn and JSScripting side-by-side so migration should be relatively straight forward and something one can do gradually.

Nashorn will likely be the first language to go since:

1. there is a suitable replacement
2. OH will eventually (year or so I’d guess) need to move to a Java later than 11 and Nashorn is gone in versions of past 11

Jython

The future here is less certain. While it’s certain that Nashorn is going away, Jython may stick around. The problem here is that the upstream project is kind of abandoned. There are not many people working on it, Python 3.x support is nowhere in sight, and there are better alternatives now.

Python 2.7, which is what Jython currently supports, has been end of life for almost two years now.

The future of Python support in OH is not completely bleak though. There is a GraalVM Python which is up-to-date and could be supported by OH. JSScripting is itself GraalVM so there is even an example for how to make it work. But from what I can see, there is no one working on implementing it.

Will a GraalVM Python add-on be created before Jython dies? But when it does happen you’ll still have to do some very significant rewrites because:

• it will be Python 3
• the current Helper Library will probably need to be rewritten and it likely won’t work exactly the same way.

So what’s the Pythonista to do? While the pressure is not high yet, consider looking into HABApp. It’s not built into OH but runs as a separate process along side OH, using the REST API to interact with OH. It’s pure Python 3, supports standard Python 3 libraries and all that good stuff.

Or if you don’t want to run something separately, consider looking to JSScripting.

Rules DSL

At this point I can’t see a case where OH will voluntarily remove Rules DSL by choice. There is too much legacy right now to remove it without a ton of pain on users. So it will probably not go away for quite some years to come. Xtend is pretty mature and stable and therefore even if the upstream project stops we can continue to use it for some time I’m sure. Therefore, of the three, Rules DSL has the least pressure on it.

If you have a bunch of Rules DSL rules hanging around, don’t feel like you must right now start rewriting them. You’ve probably years of time.

If you are writing new rules, I encourage you to use Blockly in the UI if you are not that much of a programmer, or any one of the non-deprecated rules languages that you prefer.

Over the coming years, consider moving those Rules DSL rules to Blockly or one of the non-deprecated languages too so you are ready when Rules DSL eventually does go away.

But Rich, just tell me what language to use!

I don’t think it’s my place to advantage or disadvantage any of the rules languages. It’s also a moving target with new languages being added and new support in those languages happening all the time. But if I had to pick one right now it would be JS Scripting. Here are the reasons:

• It is built into OH and doesn’t run as a separate service.

• It comes bundled with it’s own helper library to abstract away the Java solving a lot of those type problems that plagues even Rules DSL. And that library makes interacting with OH as simple and straightforward as possible.

• Supports npm for third party libraries. In general it can present as pure and unadorned of a JS environment as you need (see below).

• There is a setting that lets those who want to write absolutely pure JavaScript with full control over what’s imported (which might be important for some third party libraries) to have that, or let the add-on import all the openHAB library stuff automatically so you can just start writing your rule without a bunch of lines at the top of every one of your Script Actions and Conditions in the UI rules.

• Has full and excellent support for UI rules.

• Right now it is the only language that supports the sharing of variables between rules even for rules written in the UI through the cache (this is being looked at to move to core so all languages can get this support but adoption of that is not certain).

• Eventually, Blockly, which currently “compiles” into Nashorn JS will compile to JS Scripting at which point, JS Scripting will probably need to be made a default add-on (like rrd4j).

• Any new docs that I personally write (such as the rest of the Getting Started Tutorial) will be JS Scripting based. All new examples and tutorials I personally write will be JS Scripting based. Any rule templates that I write and publish to the Marketplace from this point forward will be JS Scripting based. As it is, most if not all of the rule templates already published either are not ECMAScript or will run in either Nashorn or JS Scripting, to ease migration.

• Can support intellisence for code completion and other IDE type features.

Will I always recommend JS Scripting? Probably not but who knows. New stuff is added to OH all the time. But as of this writing, all the other options have limitations which make them, in my opinion, less viable candidates to be “the default” language for OH. For the most part, the feature they all lack is good support for UI rules in addition to text file based rules.

But if you don’t care about UI rules and never will care about UI rules (can I ask you to reconsider since UI rules is the only way to write Rule Templates) then any of the other non-deprecated languages are perfectly fine candidates for use.

[Oliver2 (Oliver)]

@rlkoshak

Everyone seems to think I have some sort of authority

Your humility speaks for you. To me, you ARE “some sort of authority” or in better words “a capacity” and I am really glad that you answered to this topic and found clear words and provided a lot of detailed and profound facts. Thank you very much for this.

@rlkoshak

If you have a bunch of Rules DSL rules hanging around, don’t feel like you must right now start rewriting them. You’ve probably years of time.

True. But if one is deciding to move on to e.g. JS scripting I think the best path will be to start migrating existing rules one after the other starting with the least complex ones thus learning the new specifics and syntax.

More feedback more than welcome.

[rlkoshak (Rich Koshak)]

It’s really up to each individual user to decide. There are some good reasons not to jump head first right away.

• There are not a lot of examples on the forum yet. Many of the example you will find might be out of date.

• There are some docs (woohoo!): JavaScript Scripting - Automation | openHAB, but they are mostly reference docs. While thats really all we have for Rules DSL too, to be honest, those reference docs are augmented with thousands of practical examples posted to the forum. JS Scripting doesn’t have that and a bunch of new features were added in just the past couple of weeks that changes a lot in how it works. So you’ll be forging new ground.

• I am really really encouraging people to write Rule Templates and for users to look at the marketplace first before deciding to code rules on their own. Why migrate your Rules DSL rule to JS Scripting when you can just install a rule template and create and configure a rule that does what you want. Only with rule templates it really doesn’t even matter what language it’s written in since you don’t ever even have to look at the code itself. I hope the Marketplace will make conversations like this irrelevant.

So for some users, it may make sense to wait a bit before diving head first. But I know there will be lots of users who do want to dive in, and many of those are already doing it, or at least planning on it.

And even if you decide to migrate now, do you want to stick with files or migrate to the UI at the same time? I would push users to using the UI but that’s a more selfish reason. The more people we have writing meaningful rules in the UI, the more rule templates and Blockly libraries we will get published to the marketplace, the fewer rules most users will have to even write. I want the average user out of the business of writing rules and widgets. Of course that will be impossible but they don’t have to write everything.

And with templates you may get something that does even more than you could implement yourself. For example, I have a Threshold Alert Rule Template. You configure:

• a Group of Items to trigger the rule on change
• a threshold value
• the comparison (e.g. >=)
• another rule to call when the Item that triggered the rule meets the comparison

So you can write a simple rule that sends an alert when one or more batteries fall below 25%. But all you have to write is the alert. Everything else is handled in the rule template.

But wait, what if you want to limit the alert to occur no more than once a day? There’s a config for that where you can supply “1d” for one day, or “3h” for three hours. Even if more Items meet the threshold, your script won’t be called more than that duration.

But wait, there’s more! What if you don’t want to ever receive alerts between 23:00 and 08:00? The template has parameters for that too.

So any user of any skill level would just need to write a script/rule that sends a Telegram or whatever. All the rest is done for them in the template.

That’s what I want to get to. Users are not limited to simple scenarios that they have to code themselves. They can have very complex behavior with hardly a line of code to mess with.

NOTE, the script that sends the alert could be written in Blockly making things even easier.

[Andrew_Rowe]

Great topic Oliver!
At first I didn’t get your thread title, you are asking if you personally should migrate away from DSL rules, not suggesting openHAB itself should dump DSL right?
I see you’ve been around for almost two years so you’ve witnessed some of the changes in openHAB. The rule language situation in openHAB is one of the few things that really remains in flux right now. I think Rich did (as ever) a terrific job of summing up the current situation.
It is at this point I would like stand up and applaud what a great project this is. openHAB advertises itself as being ‘hardware agnostic’ . Well I’d like to add that it also is pretty dog gone ‘language agnostic’. In other words, when it comes to writing automations, you have a lot of choices. This is because there are a awful lot of very talented people who have contributed to the project, talented programmers who used their expertise and talent to make openHAB better and more capable.
You would need a crystal ball to make a better choice then Rich’s advise provides. All our contributors are volunteers and unfortunately for myraid reasons come and go. With them, again unfortunately, often goes the maintenance of some very good ideas.
So… in response to your request for comments in reference to the blog post, Dan also introduced the newly merged jRuby option and Yannik and Stefan’s recent enhancements to the Blockly languages
Lots of horses for courses
As far as DSL going away… not to worry… not to worry DSL isn’t going anywhere any time soon IMHO

[Bernd_Ritter]

I migrated all my rules from “Rules DSL” to “ECMA script”, after upgrading to OH 3.

Now after reading this thread, I am confused.
Are my rules in the outdated “Nashorn ECMA script”?
Do I have to migrate again from “ECMA script” to “JS scripting”?

I am currently on OH 3.1 and upgrade to 3.2 is planned for next week…

Regards
Bernd

[Oliver2 (Oliver)]

No. Both versions coexist. You can continue with Nashorn ECMA script 5.1 for now.
Migrating should be feasible as code gets easier.
Please re-read Rich‘s section - everything is in there

@rlkoshak

Nashorn ECMA Script 5.1

[rlkoshak (Rich Koshak)]

@Bernd_Ritter

Are my rules in the outdated “Nashorn ECMA script”?

Yes

@Bernd_Ritter

Do I have to migrate again from “ECMA script” to “JS scripting”?

Eventually. There is no big hurry but eventually you’ll need to. But it could be as simple as adding one line to the top off the script.

[Bernd_Ritter]

Can you use “Blockly” to generate “JS scripting” code? I found it quite useful in OH 3.0 to use Blockly and look at the generated ECMA script code…

[rlkoshak (Rich Koshak)]

not yet. It’s backed by Nashorn for now.

[Marcus_Carlsson]

Hi all

I will just write my thoghts about this. I’m pretty much a poser or wannabe, that lacks experience in coding and just got addictive to homeautomation. I’ve no hard feelings of learning some’ new skills but going to java seems like a huge learning curve.

With that beeing said is there a risk that unexperienced users will choose any other platform?

And is there any good place to start learning java scripts? I just recently started playing around with blockly and watched the code and that doesn’t look like java as mentioned above. On the downside it all felt like you need a dsl rule to trigger blocklyscripts and blocklyscrips can’t be added to textual scripts atleast not in my VS code.

Maybe there’s a better way to trigger rules, and yes I prefer having my config as text so I can edit them easily.

Cheers y’all

[NCO (E. Gerland)]

@Marcus_Carlsson

dsl rule to trigger blocklyscripts and blocklyscrips

I agree, that looks for me confusing as well (trigger by regular rule and do the rest in blockly).
(By the way, I am a wannabe as well
I tried once NodeRed, because I liked the “visual approach” this is what’s interesting on Blockly.
However, I was not able to replace all my DSL code with NodeRed, so I wonder if this will be possible with Blockly.
And: if there will be an option to create a trigger “block” in OH blockly soon as well (like if item XY changed to ZZ)

[rlkoshak (Rich Koshak)]

@Marcus_Carlsson

I’ve no hard feelings of learning some’ new skills but going to java seems like a huge learning curve.

Blockly is now pretty complete and way easier to use and learn than even Rules DSL.

@Marcus_Carlsson

With that beeing said is there a risk that unexperienced users will choose any other platform?

Perhaps but I don’t see how that can be helped. But it’s worth noticing that we now have a rules language even more beginner and non-technical user friendly than Rules DSL in Blockly. We also have rule templates and blockly libraries in the Marketplace so most of these users may not even need to write rules at all eventually.

And as I said, there are no plans to remove Rules DSL. But we don’t own the upstream Xtend project upon which Rules DSL is based. When the upstream drops support, we will have no choice but to drop support too. There really isn’t any other option. But it’s not going to be our choice when/if that happens.

@Marcus_Carlsson

I just recently started playing around with blockly and watched the code and that doesn’t look like java as mentioned above

Do not confuse Java and JavaScript. They are completely different programming languages. Also note that the JavaScript that the Blockly blocks convert to is going to be more complex and hard to read than code you would write yourself. So it’s perhaps not the best comparison.

Currently it also converts to Nashorn JavaScript, not JSScripting. And with the helper library that comes with JSScripting, the code will look significantly different too.

@Marcus_Carlsson

On the downside it all felt like you need a dsl rule to trigger blocklyscripts and blocklyscrips can’t be added to textual scripts atleast not in my VS code.

Blockly only exists in MainUI. You cannot easily define them outside of MainUI. So you will configure a rule, choose “Run a Script” as the Action and code your Blockly there. If you want to stick to files, Blockly isn’t for you, nor are custom MainUI Widgets.

@Marcus_Carlsson

Maybe there’s a better way to trigger rules, and yes I prefer having my config as text so I can edit them easily.

This is probably going to be controversial and the opinions expressed are my own.

1. Most of the objections against using the UI are bogus. I fully agree there are some cases where sticking to text files for configs is absolutely required. In all other cases it pretty much boils down to “because I want to” is the reason. That’s fine and well on it’s own. However…

2. There are compromises that come into play when choosing files or the UI for configs. Some of those compromises are covered in the Getting Started Tutorial but they can entirely be boiled down to the following: you get no help in text file configs; you will have to fight syntax errors and spend lots of time looking stuff up.

3. Given 2 inexperienced users and non-technical users have no business trying to use text based configs. It’s going to be a waist of their time and it’s going to be a waist of our time supporting them. There is a whole world of mistakes that are simply impossible to make in the UI. Much (not all) of the config in the UI is self descriptive and self documenting; there is no need to look stuff up.

If you are “pretty much a poser or wannabe, that lacks experience in coding” and just getting started with OH, IMO you have no business messing with text configs in OH. It’d be like trying to win a bike race with flat tires.

Since you @Marcus_Carlsson already have started with text files it may not be worth the effort to try to convert. But ultimately choosing to do everything in text configs is a choice that comes with a cost and compromises and I don’t have a lot of sympathy when those costs and compromises come due. It’s up to you to decide if the costs and pain of dealing with text configs are less than what ever benefit you perceive to get from using them.

@NCO

I agree, that looks for me confusing as well (trigger by regular rule and do the rest in blockly).

Navigate to Settings → Rules in MainUI.

Click the + icon.

Fill in the metadata (UID, name, description, etc).

Add triggers as desired.

Click add Action under “then” and choose “Run Script”. Choose Blockly as the language and code what you want the rule to do.

@NCO

so I wonder if this will be possible with Blockly.

As of 3.2 it should be possible.

@NCO

And: if there will be an option to create a trigger “block” in OH blockly soon as well (like if item XY changed to ZZ)

This is really kind of nonsense.

Blokly only exists in UI rules. In UI rules there are three sections: triggers, actions, and conditions.

• Triggers are where you define the events that trigger the rule. There is no code here at all in UI rules. Compared to Rules DSL, this is what goes between the line that starts with rule and ends with then.
• Actions are where you define what the rule does when it triggers. Compared to Rules DSL, this is what goes between the then and end.
• Conditions are unique to UI rules. This is where you can define under what conditions the rule will execute (e.g. only run when MyItem > 20 °C). This is the equivalent to Rules DSL rules that start with an if(MyItem.state <= 20 | °C) return;. Separating out the conditions from the actions is a very powerful capability.

NOTE: There might be a confusion between the Rules section and the Scripts sections of MainUI. Rules are fully capable rules with triggers, actions, and conditions. In very simple cases there may be no code at all in these. In more complex cases you’ll have a Script Action (“Run script” chosen as the Action) and sometimes a Script Condition (the last line of a Script Condition needs to evaluate to a boolean). Scripts are a special type of rule that consists only of a single Script Action without triggers or conditions.

Blockly can be used in both Scripts and Rules.

[NCO (E. Gerland)]

@rlkoshak

This is really kind of nonsense.

That’s possibly true and my impression about this comes from NodeRed, where everythig was visible in the graphical view (like item xx changed).
Now that you explained it your way, it totally makes sense (to create the code itself between then and end with blockly (only).

I will play around with the mainUI possibilities to get a better idea.
Thanks, @rlkoshak

[smhaller (Stefan M. Haller)]

Actually I would love to switch from DSL to ECMA 11.
But for sure I started with something simple, that had to fail .

SendNotification does not work in my OH env conditions (no openhab-cloud).
JavaScript Scripting - Automation | openHAB

What I don’t like is the dependency to OH external stuff (npm), while with Rules DSL everything is part of it and known where to be found (but that’s because of I would need to get used to). And as @rlkoshak stated there is a lot of stuff and examples for the old Rules DSL, while there is room for improvement on documentation of JavaScript stuff.

[Oliver2 (Oliver)]

@smhaller

dependency to OH external stuff (npm)

hmm. Normally you do not have to care about as it is installed with the binding -if I got you right.
I have not only decided to move from DSL rules to JSscripting - I am also moving from text based files to UI scripts.
I think once you have successfully migrated one rule, the rest can be migrated easier.

[rlkoshak (Rich Koshak)]

@smhaller

What I don’t like is the dependency to OH external stuff (npm)

npm is an option, not required. Everything you need to interact with OH is built into the add-on including the helper library. The reference docs for the helper library that comes with the add-on is at JavaScript Scripting - Automation | openHAB. Be sure to click through the links which will lead you to more detailed reference docs and eventually to the actual code if desired.

By default, all of the helper library stuff is imported into your rules by default so you don’t even need to import it manually. For those who do want to use npm and external libraries and such that flag can be turned off to avoid naming conflicts where necessary. I think this is a good compromise between the ECMA Script experts out there and most of the rest of us who just want to write rules for OH.

One final note about npm though. This is going to be a really powerful and easy way for people to distribute additional rules libraries and even perhaps rule sets. Instead of 1. clone the github repo, 2. copy the files you want to the right folder, 3. test that it’s working, 4. repeat we will have 1. navigate to $OH_CONF/automation/js, 2. run npm install openhab_rules_tools. With the former not only do you get the library but you get everything that library depends upon as well.

I’m not expecting this to replace the Marketplace and I’ll still push to get support for libraries in the Markteplace, but for now it’s a much better experience than what we used to have.

@smhaller

stated there is a lot of stuff and examples for the old Rules DSL, while there is room for improvement on documentation of JavaScript stuff.

This addresses two different things. As reference docs, I find JS Scripting’s docs to actually be far superior to what we have for Rules DSL. They are more comprehensive, complete, and you can actually get down to the actual source code where needed. Furthermore, because it’s almost fully standard ECMA 11, there are way more resources out on the wider Internet and hundreds of books to help you learn the language. But that’s just the reference docs.

Where Rules DSL excels is that it has nearly a decade’s head start when it comes to examples specific to openHAB. There really isn’t a whole lot that can be done about that except give some time for more and more people to post tutorials and examples.

While I cannot promise timeliness, I would ask anyone who is struggling to convert a rule to post a new thread, tag me in it, and I’ll do my best to show you what it should look like in JS Scripting. But note that I’m far from an ECMAScript coder and my code is likely riddles with anti-patterns and poor practices so what I post may not be the best, but it should at least work.

However, I’m strictly using UI rules these days. My primary goal is to write rule templates which are only supportable in the UI. So I won’t be much help in creating rules in .js files, but I can help with the actions part.

[Oliver2 (Oliver)]

Rich, from a learning curve’s perspective, I think it makes sense if we had one subforum for JSscripting so that all users who start with or move over to JSscripting have all they need at one place.
It is painful to find JS examples (even NashornJS). Sure, there are tags available but those are not consistent nor are they maintained.

[rlkoshak (Rich Koshak)]

If we create one for JS Scripting we’d need to create one for Groocy, Jython, jRuby, Rules DSL, JScript, and any new ones that come along. What about HABApp and Node Red? This forum already has too many categories and this will create a whole new set.

And even if there is a category, I guarantee you it won’t be used any better than the tags are. So I don’t see how adding new categories is any better than using the tags that already exist.

[Marcus_Carlsson]

Hi

Thanks for alot of input, my reason for having textual co figuration as a favorite is in somehow related to my start with Openhab for several years ago in version 1.8 if i recall correctly.

Textual has been my safe metod to easily restore old config on new setup across windows or linux os. Maybe there is a way that I don’t know about to easily backup and restore config across different plattforms.

I am now cliking around in the ui designs and can figure some of it out but not all yet. I might get it soon and woukd really like to learn how to convert my rules and I have no regrets for learning new stuff I’m eager too learn new stuff. I’m no educated programmer but have a big interest and have coded basic, html, power query and read a cojple of css and java books.

And a least I will say that everybody in the community is really helpfull and there is alot of examplecode to get an idea from or reuse it as is .