Experimental Next-Gen Rules Engine Documentation 1 of : Introduction

This tutorial post is very out of date and ultimately did not make it into the official docs. Do not use this except for historic purposes. See the Getting Started Tutorial sections on rules for how to get started with rules. Rules - Introduction | openHAB

EDIT: This is getting too long for one posting. I’ve split it up (thanks @ysc’s write-up for HABot for giving me the idea).


This is a series of postings to start writing and gathering what we know works and what doesn’t yet work in the Rules that can be created through PaperUI. In particular I’m interesting in what we have access to and not in self written scripts.

I’m making all of these a wiki so please add your contributions to the OP instead of in comments if you have something that works or spot something wrong or that has changed. For now it is kind of stream of consciousness and not organized.

I’m currently testing with OH 2.4 SNAPSHOT Build #1405


Background

The Experimental Next-Gen Rules Engine (NGRE) is a brand new automation engine available as a separate install on OH 2.3. Note the “Experimental” in the name. While NGRE has matured significantly in recent months, it is still a work in progress. Use at your own risk. The document below makes up the bulk of the available documentation.

This document will primarily focus on creating and managing NGRE Rules through the Rules tab in PaperUI. This is not currently the only way and expect new NGRE editors in the future.

What’s New

There are some significant differences between the Rules DSL and the NGRE. For one the NGRE supports new concepts and capabilities including:

  • supports the creation and management of Rules through the REST API meaning we can have UI based Rule Builders (currently only PaperUI exists but there was a Node Red/Yahoo Pipes like UI started)

  • Rules can be enabled or disabled, and they can be enabled and disabled through other Rules // TODO how?

  • Rules can be created that all use the same script as the action

  • a Rule can be manually run from PaperUI (don’t do this right now, there is a bug that will kill your Rule)

  • Rules are written in main stream programming languages (JavaScript supported now, others to follow)

  • if you are using JSR223 then you are already using the NGRE (more on the implications of this below)

  • support for Rule Templates // TODO How do I create a template? How do I use a template?

  • super simple Rules like sending a command, playing a sound, or saying something do not even require manually entering a script

  • there is a new concept called “Conditions” which allows one to define the conditions under which the Rule should trigger; no more need to add if statements at the beginning of your Rule; conditions are fully scriptable allowing for very complex and thorough pre-checking

  • support for libraries, functions, and classes

  • it will run your old .rules files (not yet implemented but promised)

And there are more. But the main takeaway is the NGRE is a wholly new thing with new concepts and capabilities.

Two of the most exciting additions for power users is the support for libraries, functions, and classes as well as the fact that you’ve likely already been using it, perhaps unbeknownst to you.

For new/non-technical users the most exciting part is the built in support for creating Rules in PaperUI, the promise of a Node Red like Rules editor, and the ability to load libraries and have Rules call each other. This last point means no longer will you have to copy and paste and edit code from the forum but will instead be able to download a library and call it with your Items.

There is much to be excited about the NGRE!

What’s Missing or Broken

There are some features that are missing for those used to Rules DSL for PapeRUI created Rules (note, most of these are available in JSR223).

  • Time cron triggers or combination of triggers and conditions to replicate the full cron expression. The “fixed time of day” trigger is a little confusing. The format description says HH:MM but I could only get it to work by also supplying AN/PM. The Day of week condition is currently broken as the dialog doesn’t let us click on the days to select them, or it is not showing that we have selected a day.

  • send command for all the command types. Currently CLOSED/DOWN/OFF/ON/OPEN/UP are the only ones listed in the dropdown. Appears we can type in anything but that is not clear from the way the UI works causing confusion.

  • Syntax checking and highlighting in the JavaScript entry box. There is also a bug in how the script entry box resizes when first creating a Rule; it doesn’t resize when pressing enter on a line with text. Pressing enter on a blank line does resize it so one must add a bunch of blank lines to open the dialog up enough to work.

  • Global variables alternative? The main issue is that many of the use cases of Timers requires retaining a handle on the Timer that can be accessed across multiple Rules or multiple executions of the same Rule. Another use case that can’t be supported easily without global variables are the use of the storeStates and restoreStates. Everything else can be stores in Items.

  • Member of Rule triggers are missing.

  • Group update event triggers are currently broken. The only way to trigger a Rule using a Group is by changed triggers. The combination of these last two means it is impossible to trigger a Rule on an Item’s update or command unless the Item is explicitly listed as a trigger.

  • There is no way to add tags to Rules except by editing the JSONDB files manually.

How NGRE Is Related to JSR223

JSR223 is a way to write OH Rules using one of three more mainstream programming languages: Jython, JavaScript (Nashorn), or Groovy. For experienced programmers, this restores many of the important programming tools that were purposefully removed from the Rules DSL to create a simpler programming environment for non-programmers. The lack of those tools can be frustrating and overly constraining for these users.

With the creation of the NGRE, now JSR223 rules and PaperUI created Rules are run on the same engine. This means that one can run a mix of JSR223 and PaperUI Rules, one can write JSR223 Rules but set up the triggers using PaperUI, once can create a library of classes and functions that are useful for both JSR223 rules and PaperUI created Rules.

In short, the NGRE will give the experts a chance to create complicated and reusable Rules logic and make that available to all users through a simply download instead of a copy/paste/edit from some forum posting.

If you are looking for how to do something and the answer is not in this guide, look at the JSR223 docs and at the JSR223 openHAB libraries. Both will be rich with examples.

This guide will focus on creating Rules in PaperUI. If you need to write complicated Rules that need good syntax highlighting, a fast save/test cycle, or if you are just more comfortable using text based files for your coding I recommend following the instructions for coding with JSR223. You will still be using the NGRE and for super simple stuff you can use PaperUI.

For those who are not coders and are just getting started, using PaperUI may be a more comfortable choice. Realize that there is nothing that can be done in JSR223 rules that cannot be done through the PaperUI interface, but the current script entry interface is very frustrating to use.

But if I use PaperUI, where are my Rules saved? All Rules created in PaperUI get saved to the JSONDB. After you create your first Rule you will find a new automation_rules.json file in $USERDATA/jsondb. The format of this file is loosely documented here. I can tell you, it’s pretty ugly, you will not want to edit this file by hand.

Here is an example JSON Rule
{
  "7a6ebf3b-6f0c-4a82-b582-5e7ded2dae42": {
    "class": "org.eclipse.smarthome.automation.dto.RuleDTO",
    "value": {
      "triggers": [
        {
          "id": "1",
          "label": "an item receives a command",
          "description": "This triggers the rule if an item receives a command.",
          "configuration": {
            "itemName": "Test",
            "command": "ON"
          },
          "type": "core.ItemCommandTrigger"
        }
      ],
      "conditions": [],
      "actions": [
        {
          "inputs": {},
          "id": "2",
          "label": "execute a given script",
          "description": "Allows the execution of a user-defined script.",
          "configuration": {
            "type": "application/javascript",
            "script": "\u0027use strict\u0027;\n\nload(\u0027./../conf/automation/jsr223/jslib/JSRule.js\u0027);\n\nlogInfo(\"Using the library!\");\n\n//var myLog \u003d Java.type(\"org.slf4j.LoggerFactory\").getLogger(\"org.eclipse.smarthome.model.script.Rules\");\n//myLog.info(\"createTimer---------------------------\");\n//var ScriptExecution \u003d Java.type(\"org.eclipse.smarthome.model.script.actions.ScriptExecution\");\n//var LocalDateTime \u003d Java.type(\"java.time.LocalDateTime\");\n//var DateTime \u003d Java.type(\"org.joda.time.DateTime\");\n//var fnc \u003d function(){\n//    myLog.info(\"Timer completed\");\n//}\n//myLog.info(\"Creating timer...\");\n//ScriptExecution.createTimer(LocalDateTime.now().plusSeconds(5), fnc);"
          },
          "type": "script.ScriptAction"
        }
      ],
      "configuration": {},
      "configDescriptions": [],
      "uid": "7a6ebf3b-6f0c-4a82-b582-5e7ded2dae42",
      "name": "Test4",
      "tags": [],
      "visibility": "VISIBLE"
    }
  }
}

Next step: Experimental Next-Gen Rules Engine Documentation 2 of : Installation.

18 Likes

Edit" A nearly complete rewrite. Please review for accuracy and intelligibility.

@5iver, @lewie, I’m having a heck of a time figuring out how to use event. Unlike with JSR223 Rules I don’t have access to that input Object that it appears is being used in the JSR223 libraries. All I get is the event.

When I log it I get something like Item 'Test' received command ON. However, typeof String returns false and there are no String methods available like split(). When I try to print out what type event actually is I get Object.

I can’t get any keys or methods to print out on this Object.

So what the heck is event? How do I use it? To I really have to call toString and parse the text? Where can I look to see what type event may be?

Try event.itemName, which will work for most. I have a table in the openhab2-Jython repo that may help… if it is the same event object.

2 Likes

Thanks, that does help a lot. Just as you were posting I also figured out what type it is an org.eclipse.smarthome.core.items.events.ItemStateEvent.

Though clearly it is something more because both getItemName and itemName work. It is like Xtend and you can drop the get?

Trying now…

Looks like dropping the “get” works for itemName, itemState, and type. It doesn’t work for getPayload(), getSource(), and getTopic() (MQTT topic?).

That table is useful. Thanks for the help. One more speed bump surpassed. :slight_smile:

1 Like

As for the Templates, I just found some documentation over at ESH. If I undestand this and that section correctly, it seems that resource bundles provide them. I have not tested this in any way, but maybe it helps a little.

1 Like

The contents of the event object will depend on the trigger type of the trigger that triggered the rule. In this example, it would have been ‘an Item state is updated’. ItemStateTriggerHandler monitors incoming events and runs the Actions for a rule when the event topic and the states match what was specified in the trigger. If using a script Action and a mix of different trigger types, you’ll need to check which attributes are available in the event object before trying to use it. Also, there is no event when manually running the Actions of a rule in Paper UI, so JS scripts relying on the event object will fail when run like this.

Both methods and attributes are available for these in the ItemStateEvent class object.

I’m not sure why you are not seeing them. This works for me (OH 1405):

'use strict';
var log	= Java.type("org.slf4j.LoggerFactory").getLogger("org.eclipse.smarthome.model.script.Rules");
log.info( "JSR223: JS: Test Rule 1: source=" + event.source );
log.info( "JSR223: JS: Test Rule 1: topic=" + event.topic );
log.info( "JSR223: JS: Test Rule 1: payload=" + event.payload );
log.info( "JSR223: JS: Test Rule 1: source=" + event.getSource() );
log.info( "JSR223: JS: Test Rule 1: topic=" + event.getTopic() );
log.info( "JSR223: JS: Test Rule 1: payload=" + event.getPayload() );

2018-11-05 08:04:15.257 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: source=org.eclipse.smarthome.core.autoupdate
2018-11-05 08:04:15.258 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: topic=smarthome/items/Test_Switch_1/state
2018-11-05 08:04:15.260 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: payload={"type":"OnOff","value":"ON"}
2018-11-05 08:04:15.267 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: source=org.eclipse.smarthome.core.autoupdate
2018-11-05 08:04:15.270 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: topic=smarthome/items/Test_Switch_1/state
2018-11-05 08:04:15.272 [INFO ] [org.eclipse.smarthome.model.script.Rules] - JSR223: JS: Test Rule 1: payload={"type":"OnOff","value":"ON"}

Topic is the event topic. If you turn the logging of org.eclipse.smarthome.automation to TRACE, the source, topic and payload for each event will be logged by ItemStateTriggerHandler.

2018-11-05 07:49:10.701 [TRACE] [org.eclipse.smarthome.automation.module.core.handler.ItemStateTriggerHandler] - ->FILTER: smarthome/items/Test_Switch_1/state:Test_Switch_1
2018-11-05 07:49:10.708 [TRACE] [org.eclipse.smarthome.automation.module.core.handler.ItemStateTriggerHandler] - Received Event: Source: org.eclipse.smarthome.core.autoupdate Topic: smarthome/items/Test_Switch_1/state Type: ItemStateEvent  Payload: {"type":"OnOff","value":"OFF"}
2018-11-05 07:49:10.710 [DEBUG] [org.eclipse.smarthome.automation.core.internal.RuleEngineImpl] - The trigger '1' of rule '208bb8ba-d78f-4f7c-8d44-0bdf91b16fd5' is triggered.
2018-11-05 07:49:10.716 [DEBUG] [org.eclipse.smarthome.automation.core.internal.RuleEngineImpl] - The rule '208bb8ba-d78f-4f7c-8d44-0bdf91b16fd5' is executed

Yes, most of that is added to the OP above. There is also no event when the Rule is triggered based on time.

I was looking at the JavaDoc, not the source and the attributes do not show up in the JavaDoc so I assumed they were private.

OK, looking at the source code they ARE private. So this means that private/protected/public is ignored when accessing Java Classes in JSR223? That seems like a really bad idea.

I just retried it and now it works. Maybe I forgot to remove the () when I went from getTopic() to topic.

I’ve an idea.

PaperUI created Rules does not yet have Member of triggers so the event will always be the Group, not the child of the Group that triggered the Rule. However, I notice that the topic says, for example:

smarthome/items/TestGroup/Test/statechanged

So if I’m not missing something, I can get at the name of the Item that triggered the Rule by parsing the topic, right? Or is there something I don’t understand about the topic yet?

You could do this, but it is already done for you… event.memberName is derived from the topic and made available through GroupItemStateChangedEvent.

1 Like

Thanks for the links and info. That helps a lot.

You probably wouldn’t know the answer to this but are you aware of any reasons why triggering a Rule by updates to a Group wouldn’t work? I can trigger a Rule all day on changes to a Group and I get the GroupItemStateChangedEvent. However, if I use an item updated trigger nothing happens.

It’s probably a problem with the JSON Rules and not something you would have seen but just in case this is a known bug or something like that…

This issue sounds like it fits your description, and I feel it is related to this one, but I’m still absorbing the response from Henning.

All of the ERE issues should have an Automation tag, but you’ll find more that haven’t been tagged yet by just searching for automation.

1 Like

Great resources. I’ve added my two cents to the two issues.

For the second issue I agree it feels related.

I can see Henning’s argument somewhat as the source of the update . But since PaperUI Rules don’t yet support Member of triggers, assuming that Group update triggers worked, we would be stuck back in the bad old days of using the persistence hack to figure out which Item caused the update.

Ideally the “correct” solution would be to get Member of triggers added to PaperUI Rules but I haven’t had a chance to explore triggers deeply yet so I don’t know why they are not already there or how hard it would be to add them.

Heads up… looks like you will want to grab a new build before diving into templates.

Templates could help a lot in early adoption by providing workarounds for gaps in UI functionality, but also by providing the ability to add a complex but standardized rule that only requires a few clicks to setup. I can see templates being setup with rules that follow each of the DPs. Even default templates included in the distro. Would be awesome to get one in before 2.4.

1 Like

Thanks, once I get the Actions documented I’ll look at Templates next. The more I’m learning the more I’m excited about the ERE. If we can fill some of the gaps (Member of triggers) and get Yannick to work on his ERE UI again this is going to be awesome! Node Red ain’t got nothing on us. :smiley:

It isn’t super clear to me yet what Templates are. My first thought was to build a version of lewie’s library as a template so we don’t have to download and separately maintain it. It would be pretty freaking awesome if all those convenience methods were just there for those who want them.

I like the idea of using some of the DPs as templates too. Especially the more stand alone ones like Time of Day or Motion Sensor Timer.

I feel like I’m starting to understand how this works from a scripting perspective. It’s now more just the tedium at this point of documenting each and every thing that people will want to/need to do to transfer their Rules from Rules DSL and finding more JS native ways to do some things and work arounds for things that are lacking.

The lack of global variables is a big hole right now that is unique to PaperUI created Rules. Without them timers and storeStates/restoreStates are all but useless. I can only do so much with Item metadata. I know that isn’t a problem for regular JSR223 Rules.

ERE Rules don’t use up threads in the Rules threadpool, right? Do you know of any limit or problems if we flip the usual recommendation and push using sleeps instead of Timers? Of course there is the Expire based timers, but that will increasingly become awkward as there remain fewer and fewer critical 1.x bindings like that. And because each Rule lives in isolation it means the code that occurs when the Expire timer goes off will be completely separate from the Rule(s) that set the timer. I’ve thought about giving it a go to build a 2.x version of it but I can’t figure out how that would work as a 2.x binding. I don’t want to have to create a separate Thing or Channel for each Item that needs to Expire.

I’m rambling. Thanks for feeding me all this good info. It has been a tremendous help.

1 Like

Correct… I know I’ve posted some examples with logs somewhere… here’s one. There is one difference with the Rules DSL that I’ve noticed, which people need to be aware of. Only one instance of a rule can run at a time, so the rule cannot be triggered again until the first instance completes. Just think of each rule as having its own threadpool with size of 1. Or that each rule has its own thread. I do not know if this should be considered a bug, as it has both good and bad side effects.

Make a rule with a JS Action that sleeps for 5s, then logs. Then trigger the rule quickly several times. There will be a 5s delay between each log.

from org.slf4j import Logger, LoggerFactory
from time import sleep
from openhab.rules import rule
from openhab.triggers import when
log = LoggerFactory.getLogger("org.eclipse.smarthome.model.script.Rules")

@rule("Test delay")
@when("Item Virtual_Switch_1 changed")
def testDelay(event):
    log.debug("Test delay: start")
    sleep(5)
    log.debug("Test delay: stop")

2018-11-08 17:49:05.577 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start
2018-11-08 17:49:10.578 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop
2018-11-08 17:49:10.579 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start
2018-11-08 17:49:15.580 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop
2018-11-08 17:49:15.580 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start
2018-11-08 17:49:20.580 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop
2018-11-08 17:49:20.581 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start
2018-11-08 17:49:25.581 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop
2018-11-08 17:49:25.582 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start
2018-11-08 17:49:30.582 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop

It should be possible to use globals in JSR223-JS, I just don’t know how to do it… or if they would be accessible through the JSON rule Actions. I just did some research on it, but I shut it down since I would prefer to put the effort into adding Jython scripts to JSON rule Actions.

I’m currently down and out with what appears to be a flu, so plenty of time to look at things… but not much comprehension possible, ATM. Although, I did have a thrilling conversation with some JS code earlier today! Coding fever dreams are my fave… makes them somehow slightly less trippy.

Templates are JSON. Here is a template with a JS Action that you can play with JS_Action_template.json (1.2 KB). (Wow… the forum takes .json files!). I couldn’t figure out what directory they are to be stored in. Copying to /conf/automation/templates/ did not cause them to show up. I ended up importing from a URL through Karaf (automation importTemplates http://localhost:8080/static/templates/JS_Action_template.json). Of course, I copied the file there. After import, I could use them to create rules, but no files seemed to be created for the templates. Will be interesting after a reboot. Still learning… hope this helps.

I will definitely be sure to add that when I expand the triggers section. This can become more of a problem in PaperUi rules because one of the way to wire them up would be to write separate Rules with their own triggers that all call the same Action. And if I encourage the use of Thread.sleep this problem will become apparent.

This sort of approach will probably become common with Yannick’s UI.

I’ll need to set up at test to see if they run fifo or in random order. That can become important in some scenarios

That’s the rub. I don’t think there is anything in JS that prevents this. But in JSON Rules you don’t have any access to a context outside the Action so there is nowhere to define the variable and no way to access it, unless there is something I’ve yet to discover. As far as I can tell, even the libraries are reloaded every single time the rule runs (though I’ve not systematically explored this yet). As far as I can tell, every time the rule runs it has a brand new and clean context.

Beware that if my suspicions are correct, Jython will have the same problem.

Hope you feel better. This year the flu is supposed to be no joke.

Check userdata/jsondb. Everything else send to go there, I would expect templates to go there too. They only reason that libraries are picked up from there for JSON rules is because that’s the path given to the load function. Everything gets stored in the JSONDB.

Always.

This should not be a problem… each would be a separate rule, even if sharing an Action.

It is FIFO…

from org.slf4j import Logger, LoggerFactory
from time import sleep
from openhab.rules import rule
from openhab.triggers import when
log = LoggerFactory.getLogger("org.eclipse.smarthome.model.script.Rules")

ruleID = 0

@rule("Test delay")
@when("Item Virtual_Switch_1 changed")
def testDelay(event):
    global ruleID
    ruleID += 1
    tempRuleID = ruleID
    log.debug("Test delay: start [{}]".format(tempRuleID))
    sleep(5)
    log.debug("Test delay: stop [{}]".format(tempRuleID))

2018-11-08 22:38:55.079 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop [1]
2018-11-08 22:38:55.080 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start [2]
2018-11-08 22:39:00.080 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop [2]
2018-11-08 22:39:00.081 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start [3]
2018-11-08 22:39:05.081 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop [3]
2018-11-08 22:39:05.082 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start [4]
2018-11-08 22:39:10.083 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop [4]
2018-11-08 22:39:10.083 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: start [5]
2018-11-08 22:39:15.084 [DEBUG] [org.eclipse.smarthome.model.script.Rules] - Test delay: stop [5]

I forgot to include that I had searched there too. Actually, I searched the entire directory (it’s a manual install), but found nothing. And removing the templates through Karaf said it was successful, but they were still there in the list and could be used for new rules. May be another bug, but I’m still on 1405 and do not have the template PR.

Except if the rule takes a long time to run and lots of triggers are running the rule we can end up with a backlog of triggers waiting to run.

:+1

Where are the rules created saved?