FINALLY: I migrated all my rules DSL files to Main UI rules + JSScripting - and I like it!

First of all I would like to thank you all for the great project and the experience I made over the years with OpenHab. I checked my backups and just saw that I started already with OH 2.3 in 2018…

Back then, I had a Raspi 3 running piVCCU (Homematic server), openVPN, pi-hole and Openhab - and if I remember correctly, I even ran InfluxDB + Grafana on that Raspi. And it was working nicely, which seems to be really surprising right now when I look at my current setup.
Today I Have 1100+ Items, 100+ Things and 100+ rules running on a Raspi 4 standalone with only Openhabian, separated from the Raspi 3 piVCCU and a Raspi 3 InfluxDB+Grafana (+MongoDB occasionally).

I really liked the file based configuration of OH 2.x, so I was a bit concerned about the approach the project is heading to, the UI based configuration. I was even looking at other home automation projects to be prepared for a migration. But as I’m curious and in general really liked the OH way of configuration and the app integration, I gave 3.0 a chance. So when migrating to 3.0 I re-created all my things, imported the items and was surprised that it was working quite smoothly. It was just a pain to remove all the UI transformation patterns from the labels after importing the items and then adding them to the metadata. And creating the semantic model out of the imported items took quite some time and is still not completely done.
In general I like the semantic model, I just don’t want to see everything in that - like read-only switches or virtual variables which are just set and used by rules.

This leads me to the UI rules - as a developer I’m used to use an IDE or editor to write my code and making heavy use of code completion, intellisense and linting. So I was very concerned if I should really migrate my rules DSL files to the UI. But there are some killer features which convinced me that I really want to use the UI:

  • conditions: this is really nice. I had a lot of rules to set switches only if another item had a special state and now I can create the UI rules even without writing code, it’s just a simple action which is executed once the trigger is fired
  • run now: I missed that everytime I wrote a rule in VSCode. Now you can immediately see if there is an error in the rule (which is mostly caused by missing linting)
  • time based triggers: creating these triggers is a charm - the UI based creation of cron rules saves a lot of time, if you don’t have to always search the internet for the syntax (I know a linux user should know the format of cron expressions…)
  • JavaScript: I never got warm with the rules DSL - I’m a Java developer, but the code you write with Xtend is far from standard Java and I never googled so much when developing my regular Java projects… I worked a lot with node.js so it is really nice that I can now use JS in my rules (even if it is not the latest version, but that’s OK).

It took some time until I migrated all my rules, but now I made it - and thanks to the great support of the forum I could really migrate all rules to JavaScript, where it is required. Almost half of my rules I created just by clicking the respective actions… and the number of loc is reduced significantly compared to before.
And I’m now thinking about what should I automate next as it is fun working with the new features :slight_smile:

After the migration to 3.2 the only files left now in my conf folder are the

  • sitemap: I like that as it is easy to maintain and easy to use in the Android app
  • icons: I’m using a lot of material icons
  • transformations: translations, data transformation
  • some external scripts triggered by the exec binding,
  • and the exec whitelist

There are just a few things which in my opinion could be better especially compared to the VSCode integration (but I know not all of them can be implemented in the UI, it is just no IDE):

  • with so many items the semantic model item picker becomes very slow when creating new rules. Especially choosing an item outside of the model takes ages. I started using the code editor for the rules and copy/pasting the parts between them. So a code completion or just a flat list might be a bit quicker
  • auto-formatting of code would be nice
  • linting would be nice
  • searching in rules is not possible without IDE. I opened the automation_rules.json in parallel to the UI to perform searches
  • copy/paste of rules is not possible. A “clone” feature for rules would be nice. The clone action could open the editor pre-filled with the same code like the original rule
  • it would be nice if I could maintain the transformation files via the UI

Additionally it seems like there is a kind of memory leak when editing a lot of rules. I couldn’t find out any root cause, but I observed that, especially when an error was thrown, the system becomes slower and slower until you can’t use the UI anymore. I had some heap space errors as well, but these were resolved when I set the heap to 512MB.

Anyhow, thanks again for all the effort to make OH such a pleasure to use!

10 Likes

You absolutely could continue using the file based configs. I just want to make it clear for future readers.

This is absolutely a pain, so much so that I recommend against it. If you want to use the semantic model, my recommendation is to use the “Create Equipment from Thing” and “Create Points from Thing” options to recreate your Items instead.

But you caught on to one of the most important thing of all. Many Items have no business being in the semantic model. The model should be focused on those thing you want to expose to your end home automation users only.

I completely agree. Conditions are are one of those things that at first you think “what’s the point, I can just use an if statement” but when you start using them you can’t imagine not having them.

Note, you can “run now” rules that are written in files too. You just can’t edit them in the UI.

Note that the material icons are available in MainUI Widgets by default. You don’t need to download and maintain them yourself. And you can write expressions to chose between them and select color and stuff in your config.

Here’s a trick I discovered. When you go to select an Item it first shows the Semantic Model. Select any random Item. Then when you click to change that Item you’ll get a flat list. Two clicks is a pain but it may be less of one if the dialog is really slow for some reason.

I discovered this because for some reason a few of my Items that are not in the model won’t appear even after checking the “show non-semantic items” checkbox.

This is more of a “how to get by” suggestion, it still needs to be fixed of course. It shouldn’t take that long and it should show all Items when that check box is checked.

There is a post somewhere where someone wrote a script that can tell you everywhere an Item is used which might be expanded to help with this. But I agree, it’s a significant limitation, especially when you get an error from a rule but it doesn’t say what rule it came from. I’ve used the REST API and the raw json file to find stuff on a number of occasions.

It kind of is through the “Code” tab. You see the whole rule (minus the rule metadata like UID, name and description) there and can copy that and paste it into a new rule. But I agree, a clone would be better.

Though that begs the question, why are you duplicated code instead of making one more generic rule?

Another option, if you don’t mind the rule being public, is to create a Rule Template on the Marketplace. Then you can instantiate any number of rules from that template as needed. For example, I have a Threshold Alert rule template that I instantiated once to get alerts when the humidity gets too low in a room and a second time to tell me when one or more of my batteries gets low.

Don’t ignore the possibility of libraries and that rules can call other rules as options as well.

I just saw the first half of the two PRs required to implement that this week. So it’s coming soon.

Also, as of 3.2 (I think) the JS transformation allows the definition of the code inline without the need to write a separate .js file in the transformations folder.

This is known and it relates to errors in rules. As the errors build up the amount of memory use expands. It can take hours to recover of lead to an OOM eventually. So if you’ve a lot of errors in your rules you’ve got problems. If not it runs nice and smooth.

When I’m editing rules and generating lots of errors, I make sure to restart OH when I’m done. The problem is known and I’m pretty sure there is an issue open for it. But waiting or rebooting can be a work around for now.

I’ve got a couple of other tidbits you might find helpful.

  1. Check out my GitHub - rkoshak/openhab-rules-tools: Library functions, classes, and examples to reuse in the development of new Rules. library. You can install it via npm and it provides library functions that implement a lot of the design patterns and common stuff like timer management and such. I’m moving some of it to openhab-js but most of it will remain in this separate library.

  2. Look in the Marketplace, there might be a rule template there that you can just install and use to replace some of your own code. Even better, as you work on your rules consider templates you can publish yourself. :smiley:

  3. The -Scratchpad- rule is really helpful. You can create it from the developer sidebar (more on that below) and it appears at the top of Scripts. Scripts are just rules with a single Script Action. When you need to simulate something (e.g. change states of Items to trigger rules) you can just write some arbitrary code there and click “run now”. It’s a great place to write and run temporary code to explore and stimulate your rules and Items.

  4. Use the developer sidebar (alt-shift-d). Here you can pin all the stuff you are currently working on (e.g. if you are working on your heating, pin all your heating Items, Things, and rules here). There you have access to everything you need one click away. No need to have lots of tabs open or switching back and forth. You can see the current state of your Items, status of the Things, and you can see your rules running as you code and test and configure.
    Also available there is an event stream that lets you see the full event stream. The stream can be filtered to specific items, things or rules and it includes lots of stuff you won’t find in events.log by default.
    If you are working on MainUI Widgets, there is an expression tester there also. You can create the -Scratchpad- Script from there too.

  5. Take advantage of enabling and disabling rules. You can manually enable/disable a rule or you can have a rule that enables/disables another rule based on some trigger. This can be even more straight forward than using a condition. For example, I’ve a switch that I turn on during the Christmas season. I “steal” some of my humidifier smart outlets to drive the Christmas lights. So when that switch goes on it disables the rules that control the humidifiers and enables the rule that controls the Christmas lights. A disabled rule will not be loaded and not consume any resources.

Thanks for posting and I hope you find some of this useful too to make it even better.

11 Likes

thank you very much - there is so much information in your recommendations which is really helpful for my configurations!

You absolutely could continue using the file based configs. I just want to make it clear for future readers.

I agree and it was working fine before I migrated. I just think the UI brings so much benefit is is not worth to stick to the old way.

But you caught on to one of the most important thing of all. Many Items have no business being in the semantic model. The model should be focused on those thing you want to expose to your end home automation users only.

Good to know I’m not the only one having this opinion…

Note that the material icons are available in MainUI Widgets by default.

Nice - can you give me an example how to refer to them? In the documentation I couldn’t find anything about that…

Here’s a trick I discovered. When you go to select an Item it first shows the Semantic Model. Select any random Item. Then when you click to change that Item you’ll get a flat list. Two clicks is a pain but it may be less of one if the dialog is really slow for some reason.

I observed this as well. But I ended up using the code directly and you already recommended to use that for copy/pasting rules. I just think for beginners that this is not the best way as you have to be careful about the syntax and the numbering of the items inside the code…

I just have a lot of rules like “if that item changes to on then set another item to off if time is between x and y” and these I copied as it was quicker than doing all with the UI…

Rules execution as action is one of the nice features I forgot to list :slight_smile: Regarding Rule Templates and publishing them to the Marketplace is one of my next tasks to look at. I didn’t consider this so far…

Also, as of 3.2 (I think) the JS transformation allows the definition of the code inline without the need to write a separate .js file in the transformations folder.

Oh, this is something I didn’t know. Have to check.

Your tidbits sound very promising. I didn’t know about the advantage of the developer sidebar and I will for sure take a close look at it!

1 Like

More on expressions to choose the icon based on the state(s) of item(s) can be found

and in the Getting Started Tutorial.

You can also find tons of examples and installable widgets in the Marketplace.

One example from a custom widget on the marketplace that changes the color and icon based on the state of the configured Item:

uid: rlk_lock_list
tags:
  - list
  - lock
  - marketplace:126596
props:
  parameters:
    - description: Door name
      label: Name
      name: name
      required: false
      type: TEXT
    - context: item
      description: Control Item
      label: Control Item
      name: control_item
      required: false
      type: TEXT
    - context: item
      description: Sensor Item
      label: Sensor Item
      name: sensor_item
      required: false
      type: TEXT
  parameterGroups: []
timestamp: Feb 3, 2021, 2:35:39 PM
component: oh-list-item
config:
  action: toggle
  actionCommand: true
  actionCommandAlt: false
  actionItem: =props.control_item
  badge: '=(items[props.sensor_item].state == "OPEN") ? "open" : "closed"'
  badgeColor: '=(items[props.sensor_item].state == "OPEN") ? "orange" : "green"'
  icon: '=(items[props.control_item].state == "ON") ? "f7:lock" : "f7:lock_open"'
  iconColor: '=(items[props.control_item].state == "ON") ? "green" : "orange"'
  subtitle: '=(items[props.sensor_item].state == "OPEN") ? "open" : (items[props.control_item].state == "ON") ? "closed and locked" : "closed and unlocked"'
  title: =props.name

Pay attention to the icon and iconColor properties. I used f7 icons but material or iconify icons will work the same.

Sounds like a great candidate for a Rule Template.

2 Likes

sorry kind of old thread but… GOLD!

Ahhhhh… AWESOME!!!
thank you Rich

Right? I had the same problem and now I see all these duplicate groups I had created because they wouldn’t show up in the list. Hate scrolling all they way to the bottom to check the show non-semantic box as well

edit: added above for context

off to scratch around on git and see what became of this. any link to issue (open or closed) would be a help

This has been merged for some time. It’s definitely in the latest release.