I am having a really frustrating time trying to use the Xtend scripting for rules. Lack of clear documentation, no support for functions, odd (to me) syntax, and no real tooling are making it difficult for me to write what would be simple logic in a more common scripting language.
I saw the JSR223 section of the wiki, but the note at the top makes me wonder if that is going to be a supported solution in the long term.
Currently this scriptengine cannot be used in OpenHAB2. A new smarthome compatible functionality is in development. Scripts from this engine can be used in the new addon without significant changes. A compatibility layer will be provided.
Is there any documentation about the OH2 way to do this? How much does it differ from the JSR223 way? Are rules in other languages supported in OH2 as part of the core?
Many people do as well. I did as well when I started. Often the frustration comes from trying to write rules using a structural or OO approach which Xtend really fights against.
I found the combination of the Rules wiki page coupled with the Xtend documentation and the many many examplkes posted on the wiki to be adequate. In particular when I am trying to figure out specific syntax the Xtend documentation has been exceptionally helpful.
There is indeed support for functions. Please see Taking Rules to New Heights. In Xtend they are referred to as lambdas and if you search this forum for ālambdaā or āFunctions$ā you should find dozens more examples.
The Rules DSL is a dynamically types language and I find the tooling (openHAB Designer) to be as good as any other dynamically typed language IDE like JavaScript or Ruby. One thing to note though is that Designer is based on Eclipse and it appears that the developers assumed that the users would be familiar with it also so a lot of really really important features are hidden if you donāt know they are there. In particular CTRL-SPACE at the end of a partially completed line will give you a list of all the valid ways to complete the line.
This is something I completely understand. When you try to write rules the same way you would in a language like Python or C or Java the Rules engine really fights against you, making certain things that seem like it should be easy very difficult. But the rules engine does provide a way to do it more easily if you bed to the rules engine instead of trying to make it bend to you. In particular:
Where ever and as much as possible put state and data structures into Items, not global hashMaps or arrayLists etc
Put your items into Groups
Use the sorting, filtering, and looping methods on a Groupās members. For example:
// Find an item by name and get its state
val s = gMyGroup.members.filter[i|i.name == "Name"].head.state
// Find the most recently updated Item in a Group, requires persistence
val i = gMyGroup.members.sortBy[lastUpdate].last
// turn off all Switch Items in a group that are ON
gMyGroup.members.filter[i|i.state == ON].forEach[i | i.sendCommand(OFF)]
Where possible trigger off of the Group updates instead of individual Items with one caveat that a Groupās state is updated multiple times for each change to one of its members so this only works for rules where multiple executions do not matter or you add logic to avoid it. But at the worst you can change the rule trigger to use each individual Item but still use the Group logic above
When the rules engine seems to fight you, step back and reconsider your approach, there is likely a better way to do it
By following these rules of thumb I eliminated 75% of my lambdas, 90% of my hasMaps, and reduced my lines of code by over 50% across the board. And my rules are easier to read, easier to debug, and easier to maintain.
To go back to this point. You are unfortunately coming to OH at a transitional period. OH 2 is rapidly approaching a full release so the developers have cautioned against spending too much time on documentation. But OH 2 is still maturing rapidly with changes almost daily so it is slightly premature to start on the OH 2 documentation in earnest. There will also be a change in how the documentation is done for OH 2 (it will be configuration controlled instead of a wiki). So addressing this need for more clear documentation is temporarily on hold. However, I hope to take a stab at writing up something more clear, comprehensive, and self contained for the Rules DSL as part of the OH 2 documentation effort soon.
My reading is that the binding will not be supported on OH2 as is but a new scripting engine is being implemented to replace the JSR233 binding and there will be a compatibility layer that will let your Jython or JavaScript rules written for the binding work in OH 2.
In general the developers are going to great lengths to ensure that whatever you do in OH 1 will work in OH 2.
Thanks for the detailed response! I just installed the JSR223 binding and I am finding it much easier to write my rules in javascript and that they are ending up with about the same length as what I was trying in Xtend anyway.
I guess I donāt really see where Xtend offers value over using javascript or python given that the API needed to interact with OH is so small and simple. JS and Python have huge knowledge bases behind them and solutions for almost every problem under the sun which makes it very easy for even a novice programmer to find solutions to their problems. On the other hand I had never even heard of Xtend before trying out Openhab and the resources and reusable solutions seem to be quite limited.
Regardless, JSR223 seems to work very well for me and I am grateful for whomever wrote the binding. I will probably just continue to write all of my rules in JS.
When you look at event based systems: Home automation (e.g. HomeGenie, openHAB, etc), PBX and telecom switches (e.g. Asterisk, Freeswitch, etc.), Phone Automation (e.g. Tasker, Locale, etc.), you will find that almost across the board they will use an Actor based programming language (e.g. Erlang in telecom) as opposed to a Functional programming based language (e.g. Lisp, Clojure, Scheme, etc.) or a Structural programming language (e.g C, etc.) or an OO programming language (e.g. Java, C#, etc.). Often this language is highly customized to the product (e.g. Asterisk rules are its own language) and other times they go with something relatively general purpose (e.g. HomeGenie has you write rules in C# or JavaScript). Most of the time they run a middle road where they have a Domain Specific Language that has hooks to use more general purpose languages. This is the path that openHAB has taken.
The advantage to having a DSL is that the language itself can be highly customized to the system environment it runs in, a lot can be handled for you in the background, and you can put limits in place to help guide and direct developers away from approaches and practices that may be suboptimal or dangerous. These DSLs are also well designed (usually) to handle the āevent responseā type processing that these systems perform and they allow certain assumptions to be made about how rules interact with the rest of the system that you cannot make if you just open it up to general purpose programs.
Personally I would give the openHAB DSL a C to a B- grade in how well it accomplishes what a good DSL should do. My main areas of complaint are:
it is difficult within Designer or looking at examples which packages need to be imported to use certain types of Objects. For example, if I were king we wouldnāt need to import anything from Joda, nothing in order to cast to and from Item and State types (e.g. I shouldnāt have to add an import to cast a Switch to a SwitchItem
working with Date Times can be maddening. Sometimes I need to use Joda, sometimes I need to use java.util.DateTime, converting between the two isnāt always obvious and when and where I have to use one verses the other is not always apparent
Designer hides a lot of functionality which you wouldnāt know about if you are not familiar with Eclipse
This isnāt to say that the JSR233 is all roses either. Iāve seen several postings about unexpected and weird behavior using it (mainly with Jython but JS has been mentioned too).
But despite my complaints, the Rules DSL is particularly well suited and adapted to openHABās architecture and, when used following the rules of thumb I outlined above, are shorter, easier to read and follow, and easier to maintain than the equivalent rules Iāve written in Jython when I experimented with JSR233, or when I tried to code in the DSL as if it were Java or C. This is one reason why I tend to defend it on this forum more so than most. Its different but once you get over its differentness it becomes a very good language for this problem. However, on the other side of the coin, Iām not sure I would choose Xtend for general purpose programming.
That huge knowledge base is not necessarily all a good thing. For example, there was just in the last day a post about how you cannot import JavaScript libraries into your JSR233 JavaScript code. Is the same true for Jython? I donāt know. Also, because the rules are event driven, there are many solutions and approaches out there that will not be suitable for this environment which a novice programmer may not know how to identify and avoid.
Because of that Iām a bit ambivalent about whether to advise jumping to JS233 first for people familiar with other languages or whether I would recommend slogging through learning Xtend first. I see good arguments either way. I probably fall more on the side of learning Xtend first but not as drastically as one may assume given how I defend the DSL on this forum.
This is indeed a known gap which Kai is addressing in the new rules engine under development for OH 2. For now copy/paste/edit is the approach for OH 1 rules sharing.
One thing I am really stumped on is how to write a library of simple reusable functions for my rules to abstract away common logic. I have a bunch of simple logic which I would like to reuse across rules. DRY (Donāt Repeat Yourself) is always a good coding practice.
Here are a couple of scenarios:
I have a dimmer which instantly jumps to a certain brightness when using sendCommand. I would like to have a function to which I can pass a dimmer, a desired brightness, and a time value and have that function smoothly ramp the brightness to the desired value. This does not seem to be possible with Xtend as far as I can tell. As an example, I would want the method to look something like this:
function setBrightnessSmooth(dimmer, desiredBrightness, speed){ ...do logic to gradually change dimmer brightness }
Or another simple example ā it would be useful to have a function which just returns a boolean for āis it dark outā. In my Xtend rules I have to write
if(now.isAfter((Sunset_Time.state as DateTimeType).calendar.timeInMillis) as Boolean || now.isBefore((Sunrise_Time.state as DateTimeType).calendar.timeInMillis) as Boolean)
While in my javascript version of this rule I can simply do
if(isNightTime())
becuase I have defined the following functions
//This function resets at midnight -- at 12:01 isAfterSunset will return false
function isAfterSunset(){
var sunsetTime = ir.getItem("Sunset_Time");
return sunsetTime.state <= DateTime.now();
}
//This function resets at midnight -- at 11:59pm isBeforeSunrise will return false
function isBeforeSunrise(){
var sunriseTime = ir.getItem("Sunrise_Time");
return sunriseTime.state >= DateTime.now();
}
function isNightTime(){
return isAfterSunset() || isBeforeSunrise();
}
Can you provide an example of how I might achieve these two use cases (or more generally just a library of reusable functions to avoid copy pasting logic) with Xtend rules?
Iām afraid we have to wait until OH 2ās new rules engine to support reusable generic rules and rule fragments. You can do it now for all the rules in a single .rules file but you canāt share them across .rules files. This is one reason why I organize my rules by function instead of other organization schemed (e.g. organized by room).
Agreed and so far, for my setup at least, Iāve managed to apply DRY quite successfully. I canāt think of a single case where Iāve had to repeat more than two lines of code. So it can be done.
val Functions$Function3 setBrightnessSmooth = [DimmerItem dimmer, Number desiredBrightness, Number speed |
...do logic to gradually change dimmer brightness
]
// To call from within your rule:
setBrightnessSmooth.apply(dimmer, desiredBrightness, desiredSpeed)
There are caveats. Only rules in the file where this is defined can see and call it. Also, the lambda does not have any context so you have pass to it anything defined in your rule or globally as a command line argument. Finally it only goes up to Functions$Function7 I think, but if you find you need more arguments than that it is a code smell and there is probably a better way.
If you organize your rules such that all your Dimmers are in the same file everything that needs to see that lambda can.
This one is harder as lambdas donāt return a value. But for things like that I create a Night switch which gets switched from ON to OFF based on Astro or Time events. See this post for details.
This is partially what I mean by āWhere ever and as much as possible put state and data structures into Items, not global hashMaps or arrayLists etcā. In this case I created a time of day state and some simple rules to set the state. In my other rules that care about time of day I simple check to see if Iām in a time period I care about by checking the Item.
So rather than creating a little function that returns true or false when it is night, I just
rule "Dispatch Notification"
when
Item Notification_Proxy received update
then
logWarn("Notification", Notification_Proxy.state.toString)
if(Night.state == OFF) sendNotification("rlkoshak@email.com", Notification_Proxy.state.toString)
else sendPushToDefaultDevice(Notification_Proxy.state.toString)
end
One of the maintainers on this forum (watou) prefers to use a single String Item that is set to āDayā, āNightā, etc instead of using separate switches, but the concept is the same.
In other cases where I need to create a boolean out of whether a bunch of Items have a certain state (e.g. if I want to see that all my Presence switches are OFF) Iāll use a Group and filter command.
val isPresent = gPresent.members.filter[i|i.state == ON].count > 0 // isPresent is true if one or members of gPresent are ON
Itās a one liner so I donāt consider it to be a violation of DRY if I reuse it elsewhere. Though the need to reuse it elsewhere rarely happens because I usually create rules to maintain these sorts of flags as Switch Items instead of recalculating the state all over my rules. Seems like a good application of DRY to me.
Search this forum for ālambdaā and āFunctions$ā and you will find all the examples that have been posted to this forum by me and others. Search the wiki for āFunctions$ā for some more examples. However, if you find yourself writing lots of little functions that also is a bit of a code smell. You shouldnāt write little one or two liners in a lambda. You should think bigger. Write the entire logic for how you handle all the logic around your dimmers into a lambda and have one rule that figures out which dimmer you need to control and calls that lambda.
Here are the lambdas Iām using, though to be honest they are pretty unique to my rule set because they basically centralize the logic for all Items of a given type (e.g. both of my garage doors use the same lambda for opening/closing) and are basically the rule body of multiple rules. And frankly a couple of them should probably be eliminated and the multiple rules that call them be merged into one rule.
// I have two garage doors so I merged the logic that triggers them into one lambda
// This lambda can probably be simplified, I don't think I really need it anymore and instead of having two rules that
// call the same lambda I can merge everything into one rule
val Functions$Function1 openGarage = [ String garageNum |
logInfo("Garage Controller", "The garage door " + garageNum + " has been triggered")
var url = "http://192.168.1.201:8000/GPIO/"
switch garageNum {
case "1" : url = url + "17"
case "2" : url = url + "22"
default : url = null
}
if (url == null) {
Notification_Proxy.postUpdate("Received cmd to trigger unknown garage door: " + garageNum)
}
else {
url = url + "/sequence/500,01"
logDebug("Garage Controller", "The URL is " + url)
logInfo("Garage Controller", "HTTP Post result " + sendHttpPostRequest(url).toString)
if (Present.state != ON) {
Notification_Proxy.postUpdate("Garage door " + garageNum + " has been triggered.")
}
if (gGarageNet.state != ON) {
Notification_Proxy.postUpdate("The garage controller may be offline, trigger may have failed!")
}
}
true
]
// If you look at other postings on that link I provided above you will see my lighting rules which includes a few
// lambdas. The logic is a bit subtle if you don't have your head wrapped around the Rules DSL yet so ask questions
// Here is a lambda that I use to detect and reset one of my Raspberry Pis when it falls offline
// Note: This lambda can be simplified I think, there is surely a better way for me to track which ones I need to
// send a notification on
val Functions$Function5 procChange = [ GroupItem gr, String name, Map<String, Timer> timers, SwitchItem reset, Map<String, Boolean> notified|
logInfo("Network", "Detected the " + name + " controller state changed to " + if(gr.state==ON) "ON" else "OFF")
// New state is OFF
if(gr.state == OFF) {
if(timers.get(gr.name) == null || timers.get(gr.name).hasTerminated){
logDebug("Network", "Creating controller offline timer for " + name)
timers.put(gr.name, createTimer(now.plusMinutes(1), [|
// Still off
if(gr.state == OFF) {
logInfo("Network", name + " is still offline after one minute")
Notification_Proxy.postUpdate("The " + name + " controller is still offline after one minute, resetting")
notified.put(gr.name, true)
// reset the power
try {
reset.sendCommand(OFF)
Thread::sleep(5000)
} catch(InterruptedException e) {
logWarn("Network", "Interrupted while sleeping to give switch time to power off")
} finally {
reset.sendCommand(ON)
}
// if it is the garage, wait five minutes then reset mqttReporter
if(name == "Garage") {
logWarn("Network", "Resetting mqttReporter in five minutes")
createTimer(now.plusMinutes(5), [|
logWarn("Network", "Resetting mqttReporter on Garage")
S_N_GarageMqttReporter.sendCommand(ON)
])
}
}
// came back while timer was asleep
else {
logInfo("Network", name + " came back online before timer expired")
}
timers.put(gr.name, null)
]))
} else {
logInfo("Network", name + " was detected offline again but a timer is already set")
}
}
// New state is ON
else {
if(timers.get(gr.name) != null) {
logInfo("Network", name + " is now ON, canceling timer")
timers.get(gr.name).cancel
timers.put(gr.name, null)
}
// If we sent a notification that it went offline, send one so we know it is back
if(notified.get(gr.name)) {
Notification_Proxy.postUpdate(name + " is back online")
notified.put(gr.name, false)
}
else {
logInfo("Network", name + " is back online and there was no timer and no notification was sent: " + notified.get(gr.name))
}
}
]
// As you saw above in the link to my time of Day posting, I also use a lambda to determine what time of day it is and switch
// the right switches ON and OFF accordingly. It mainly gets called
That is pretty much it. I used to have soooo many more lambdas and lambdas that called lambdas and my rules ran to the thousands of lines. But by applying some Rules DSL best practices which Iāve already mentioned my entire home automation runs on 667 lines spread across seven files, none of which exceeds 200 lines. And as my comments above imply, I could probably trim a good number of those lines out if I just applied my own rules of thumb more rigorously and cut out some of the extra comments and white space I donāt really need anymore.
If you have a specific set of Items and Rules you want to do something and want to see how I would implement it in Xtend feel free to post them. Iāve helped others before in this way, and working on problems like this keep my Rules DSL coding skills sharp.
I totally agree, I try to adhere to the DRY principle whenever possible.
Indeed, Iām used to coding PHP where I could easily set variables that would work wherever they were imported, and define functions for reusable code.
Iāve been slightly surprised at how little Iāve actually needed to do in order to have something useful for my uses in terms of ādaylightā and AM/PMā¦
rule "Daylight On"
when Item daylightStart changed from OFF to ON
then {
sendCommand(daylight, ON)
}
end
rule "Daylight Off"
when Item daylightEnd changed from OFF to ON
then {
sendCommand(daylight, OFF)
}
end
rule "Morning ON"
when
Time cron "0 0 0 * * ? *"
then {
sendCommand(morning, ON)
logInfo("cron","Morning ON.")
}
end
rule "Morning OFF"
when
Time cron "0 0 12 * * ? *"
then {
sendCommand(morning, OFF)
logInfo("cron","Morning OFF.")
}
end
And THAT is what I mean when I say the Ruleās DSL is a good fit for HA. Coding all of this logic by hand in rules would take dozens of lines of code even if using a language like JavaScript plus it would require a lot of polling and such (NOTE: I donāt want to imply that this approach would not work in JSR233, it would and it would work well there too but it is not an approach that Jython or JavaScript makes obvious would be a better one).
This code is nice and simple and clean and it does what it is supposed to in a decoupled way so if you decide to completely change up how you calculate your Morning and Daylight you can and the rest of your code can remain unchanged.
The biggest thing I see when I see rules like these (and my rules files are riddled with them) is how much boiler plate code you need to write for each rule. It really becomes apparent when you have three one or two liners in a row.
The only comment I would have to make is that the curly brackets are not required. The āthenā and āendā defines the scope of the rule for you. If you are more comfortable with them in that is perfectly fine too. I just wanted to point out they are not required.
The other thing I would point out is you should probably have a System started rule which figures out whether you are in the Morning state or the Daylight state or else in the very likely rare event that openHAB is not running when the state should change from one state to the other you will miss the change and, for example, Morning will stay on. Of course you could just ignore it, it would be a very rare error case and unlikely enough to occur to not be worth the five lines of code or so it would take to implement.
I was just looking through the comments I have bookmarked and noticed about 3/4ths of them are written by you @rlkoshak. So thanks for writing clearly written, thorough explanations!
Like you said, that approach works well with JSR223 scripting as well. With Jython and JSR223 support, I have the ability to create rules as simply as the DSL examples above but I also have much more flexibility for packaging and reusing components of my code. Iām not saying that the Rule DSL isnāt sufficient for most tasks, but I donāt see any clear advantages and do see some specific disadvantages to the DSL. One disadvantage is the requirement to learn a programming language that is not used widely (compared to Javascript, Python or even Groovy). Another disadvantage are the DSL performance issues that have been reported several times in the forum. IDE support was already available for Javascript, Jython and Groovy and there are multiple choices for editors and IDEs that have specific support for these languages. There are vast quantities of useful code like device access (in cases where thereās no OH binding), unit test frameworks and other utilities. For example, I use third-party Python code with JSR223 for controlling external devices like wifi cameras.
Iād say about 70% of my rules could easily be written in the DSL, but why? Even if someone doesnāt already know Javascript, Python or Groovy it makes more sense to me to learn a language with wide applications outside of OH.
Since OpenHab2 is written in Java, I guess it should be possible to write rules also in Java. However, I could not find any starting point for this. Also I did not find the code for the current rule engine. Does someone know in which repository it is located? Or is it closed source?
I have been using node red as a rule engine and I can heartily recommend it. It is very easy to set up with openhabian and is much easier to use than clunky xtend or manually coding in JSR223. It does not sacrifice any power/flexibility either.
You can write rules in the Rules DSL, Experimental Rules Engine, or using the JSR223 Addon you can write them in Jython, JavaScript, or Groovy. You cannot write Rules in Java.
There is nothing closed source in the OH project. The source code is in the Eclipse SmartHome repo.
