Xtend Scripts vs JSR223?

Python/Jython is a programming language. Xtend is a programming language. People that write code in either one are “programming”. The declarative aspect of the Rules DSL is mostly limited to defining triggers, which can also be represented declaratively using decorators in Python. The “then” part of a the Rule DSL is a script (a program). Any nontrivial automation rule will require programming.

So, the question is which programming language do you want to use?

As a programming language, Xtend is not very popular, even among JVM-based languages. As one blog article put it, “xTend is not dead, because it has never been living.” I personally found the DSL/Xtend approach to be clunky, crippled and buggy. According to Rich, it’s just because I have had too much experience with many programming languages. According to him, if it didn’t know better, I’d be happier with Xtend. He might be correct, but that’s not a strong argument in favor of Xtend.

I agree there is more information in the OH forums about how to use the DSL and Xtend than for JSR223 Jython. However, Python is one of the most popular programming languages in the world. There is much, much more documentation about how to use Python as a language compared to Xtend. There are thousands of times more Python programmers than Xtend programmers. Multitudes of existing Python libraries can be used with Jython and OH2. And ironically, it’s even easier to integrate Java libraries using Jython. For those reasons and others (like performance), I think Jython is a net win compared to Xtend.

I’ve never said that and never will say that. I have and will say that Xtend, or more properly the Rules DSL is well suited to this specific problem space but if you feel like you are fighting it you should use JSR223. I can’t count the number of times I’ve posted people to JSR223. I’ve never once to my knowledge said anything negative about JSR223. I’ve never once to my recollection told anyone they should not move to JSR223.

But perhaps you know better what I think and what I’ve said than I do.

Agreed. But there are a large minority if not a majority of OH users who are not programmers and who will never be programmers. The barrior to those users for Python is quite a bit higher than Rules DSL. That is why Rules DSL is the default.

If we throw out Rules DSL before the Experimental Rules Engine is ready we will be abandoning a sizable chunk of the current and future OH users who will never even give OH a try let alone preservere to the point they have a working system.

My point then and my point now is the barrior to entry for none programmers is relatively lower with Rules DSL verses Jython et. al. and once past that barrior there is very little one cannot accomplish in the OH space by continuing to use the Rules DSL.

But if the limitations of the language are posing a barrior to you getting started because you are used to having programming features not available, you should use JSR223.

Though perhaps I’m wrong. Can you tell me what I think again? I keep forgetting.

You have actually made that point to me in the past (about programming preconceptions being a reason people have issues with Xtend, to be clear). It’s been awhile so it’s very possible you believe otherwise now. I never claimed you were saying negative things about JSR223 Jython or advising users to avoid it. On the other hand, I don’t agree with some of your points in favor of Xtend.

For example…

If OH users are writing DSL rules with nontrivial “then” Xtend scripts (or standalone Xtend scripts)… they are programmers. Maybe not professional programmers and maybe not self-identified as such, but they are programming and that’s what a programmer does. (We’ve had this discussion several times too.) Given that reality, it’s reasonable to discuss what programming tools are most effective for that purpose.

I’ll tell you what. If you will sign up to explain what an Object is, what a Class is, what a structure is, what a library is, what an annotation is explain why you can’t mix tabs and spaces in indents why you have to use indents in the first place and agree to do so over and over and over and over in this forum I’ll not say another word about Xtend and completely ignore every “should I use rules DSL” posting from now on. I’d be happy to let someone else take that on.

If you will agree to bring the OH docs up to at least the same level as the docs in the rules DSL I won’t even help other people to work with the rules DSL any more.

Hell if you do all that and want me to I’ll delete my account on the forum and move on. I’ll even drop OH itself if it makes you happy.

But as it stands I can’t even figure out how to call an Action in Jython without looking at source code. I’ve seen multiple questions on the forum asking really basic questions like are Timers supported? And often these questions go unanswered.

[One user is at a stand still] (How to use `itemRegistry` within a jython-**module**?) because he can’t get his rules to trigger. 7 days and no response. But he’s a programmer now and there are tons of python resources out there. I guess he can figure it out for himself. That’s what programmers do.

Honestly there have been so few answers to jsr223 questions on the forum I thought most of the users like you and spacemanspiff had moved on and jsr223 had largely been abandoned. In fact someone even posted that you had moved on in one thread. And indeed you haven’t posted since Feb.

So are you back? Are you willing to do what it takes to make Jython (let’s just pick one of the three languages) a viable alternative for new users?

As everything stands right now, saying that there should be no Rules DSL and everyone needs to be a full up programmer to get started is unreasonable and a big fu to the new users. And yes that is how I interpret your position. I’m not sure how else I could but if I’m wrong please correct me because I don’t want to put words in your mouth.

I get it. You think the Rules DSL is shit and shouldn’t exist. But you offer no viable alternative to a big chunk of OH users.

So I’ll continue to answer questions about the Rules DSL. I’ll continue to post DPs. I’ll continue to push those who are frustrated by the limitations of the language to one of the JSR223 languages. And when a viable alternative that has a low barrior to entry for non programmers comes along I’ll probably switch to supporting that. The Experimental Rules Engine is looking promising. But it doesn’t support timers and actions yet, or if it does us mere mortals don’t know how to do it. If over if the JSR223 languages was documented where I can’t figure out how to do basic stuff by looking at the docs, not the code is jump on that.

Node-red has dedicated openHAB nodes that caninteract directly with OH via the REST API.
MQTT works great in node-red (It is kind of made for it) but there are other options to interact with OH.

We’ve quickly moved quite far from an objective discussion, so I’ll make a few final comments and be on my way. Feel free to have the final rant.

Ironically, there are DSL/Xtend questions about timers in the forum within the last 24 hours (and there have been many in the past). With Jython I can point them to comprehensive documentation since timers are part of the Python standard library (along with schedulers and other time-related functionality). But that doesn’t really matter. Users will have endless questions about both approaches.

The biggest practical disadvantage of JRS223 Jython in OH is not directly technical, although it has significant technical implications. It is that it is not well understood or supported by the core developers. It took over a year of effort to get the PR merged in to OH2 and mostly not for technical reasons. They appear to not understand the potential of platform scripting. There’s a tendency to compare DLS/Xtend and JSR223 only in the context of rule creation, but that is a misleading way to compare the two technologies. JSR223 Jython can also be used to create new Triggers, Things, Bindings, Actions, and so on, and those components can be packaged and shared. Supporting this type of development with Jython would open up component development to a much larger developer community that isn’t interested in investing long hours installing, learning, and trying to keep the very complex OH Eclipse and Maven build environment working.

A clear advantage of DSL/Xtend is that it works “out of the box” where OH requires extra installation steps for JSR223 Jython (rather than bundling it). The out-of-the-box experimental rule engine API that JSR223 must use is not great. Fortunately, there are many possibilities to build easy-to-use API’s in Jython to help, but those are not endorsed by the OH team and they must be maintained external to OH. I suspect that, even for you, some your frustrations are from using the low level OH JSR223 APIs rather than using the available convenience libraries that allow “rules” (event-driven functions) to be written in a much more concise way than the DSL without needing to know about classes and objects or being a “full up” programmer (whatever that means).

You are correct that I have mostly moved on from OH. I have voted with my feet. The DSL was not a primary factor in my decision since I didn’t use it anyway. I still run a server, but OH is not the core component in my home automation system now.

2 Likes

Good to know. Thanks!

@steve1, I disagree with very little of your last post. JSR223 is a fantastic way to extend the basic functionality of OH. But that doesn’t help me help new users write automation rules. I’m sure there have been questions about rules DSL timers in the last 24 hours. But the difference is those questions have been or will get answers. Many or most of the JSR223 questions do not. And there are no easy ways to find and read docs that provide the answers.

I can’t speak to the maintainer’s. I don’t work on OH as a developer and have no insight there. From what I have seen I’ve no doubt you are correct. I’ve seen them push away a number of long time contributors. The complex maven eclipse environment is indeed a big factor in why I’ve not tried developing a binding it some such.

And I was indeed angry in my responses. From my perspective you showed up after not paying to the forum for almost half a year only to put words into my mouth that do not represent what I’ve said or believe. Replace “Rich” with “Steve” and swap Xtend for Python and tell me you wouldn’t have been angered? Especially since you left I’ve been one of the biggest promoters of using JSR223. I’m not calling blame, just explaining my responses.

1 Like

For whoever is interested, it’s easy to type “jython timer” in this community forum search form or to type “openhab jython timers” in Google. Both techniques lead quickly to related questions with answers.

Hi Rich,
I’ve been using Openhab for the past 7 months and just wanted to say Thank You.

I’m not big on posting (this being my first) but I just wanted you to know that it’s people like you, that gives folks such as myself the info to succeed. I’ve noticed in a few post you mentioning a white board would help explain the situation. Any thoughts or plans on making a Youtube tutorial?

1 Like

Yeah! We want Rich and his hat on video! We love you really.

I’m a professional software developer and I’ve been able to survive using xtend, but when I found jsr223 in openhab 1 I was so much happier. I shared a rule I wrote in Javascript that I just couldn’t do in a similar way in xtend without a lot of extra and duplicate code. Here it is: General Scene Rule for jsr223

One of the things I really appreciated about it was the fact that I could define any arbitrary data structure for a particular problem and then use that to create rules, even based on the data structure. So in the example above I had an array of objects that included item names. I could then use those item names in each object to create a rule trigger. As far as I know (and I could be ignorant) it is not possible to do in xtend and each item I want to trigger the rule would need to be explicitly included as a trigger somewhere in one of the rules.

After I upgraded to openhab 2, jsr223 wasn’t quite ready to use at the time so I went back to xtend. Most of my rules are not bad in xtend, but I just recently had a doozie. I was setting up my house to open or close vents based on temperature sensors in each room and the operating state of the thermostat. I ended up copying and pasting the same complex rule for each room, after making sure the first was well debugged. I just did a text search and replace for the copies to replace office with bedroom for example. Using jsr223, which I believe is working better now and I will probably eventually switch back to, I could create a data structure defining each room and make one rule with various triggers determined from the data structure. This is just much easier for me and the way that I like to think about problems.

I hate my speaking voice. Like really hate to hear my self when I’m recorded. Editing video of myself would be torture. So I’m really not excited about trying my hand at YouTube. I have thought about a Rules DSL book, but my fear is by the time if finish it the Experimental Rules Engine would be ready for prime time and the book would be out of date before it’s published (not that I could hope for anything better than a read the docs publication.

Not to mention I could be spending that time just working in the docs themselves.

Most of the stuff I’ve mentioned needing a white board for are really generic programming topics like how C like Classes and Objects relate to each other and the like. In certain rare circumstances having this knowledge is useful, but usually the average user doesn’t need to know details at this level.

The funny thing is that picture was taken by my wife in a hat shop in Santa Fe, New Mexico. I didn’t come home with it but everyone liked it so much I had to order one. :grinning:

Which is what I’ve been saying all along. When you are a beginner being faced with the full set of programming told is daunting. It represents a barrior to entry into itself. But if you already know all those tools, being faced with the lack of those tools is just as big a barrior to entry. So if you know how to program, please use JSR223. You’ll be much happier.

As of OH 2.3 you can put the items in a Group and use Member of MyGroup changed' (or any of the other Item triggers) and the Item that triggered the rule will betriggeringItem. Prior to that, you could useItem MyGroup received update` which want great but worked. The problem was the rule triggered multiple times per update and we had to use a hack with persistence to get the specific item that triggered the rule.

Should not be necessary. If you use Member of triggers and some of the design patterns like Associated Items you should only need one rule.

Basically, Groups are your data structures in Rules DSL.

As an example:


rule "Keep track of the last time a door was opened or closed"
when
  Member of gDoorSensors changed
then
  if(previousState == NULL) return;

  val name = triggeringItem.name
  val state = triggeringItem.state

  // Update the time stamp
  postUpdate(name+"_LastUpdate", now.toString)

  // Set the timer if the door is open, cancel if it is closed
  if(state == OPEN) sendCommand(name+"_Timer", "ON")
  else postUpdate(name+"_Timer", "OFF")

  // Set the message
  val msg = new StringBuilder
  msg.append(transform("MAP", "en.map", name) + " was ")
  msg.append(if(state == OPEN) "opened" else "closed")

  var alert = false
  if(vTimeOfDay.state.toString == "NIGHT" || vTimeOfDay.state.toString == "BED") {
    msg.append(" and it is night")
    alert = true
  }
  if(vPresent.state == OFF) {
    msg.append(" and no one is home")
    alert = true
  }

  // Alert if necessary
  if(alert){
    msg.append("!")
    val timer = flappingTimers.get(name)
    if(timer !== null) {
      logWarn(logName, name + " is flapping!")
      timer.cancel
      flappingTimers.put(name, null)
    }
    else {
      flappingTimers.put(name, createTimer(now.plusSeconds(3), [ |
        aAlert.sendCommand(msg.toString)
        flappingTimers.put(name, null)
      ]))
    }
  }
  // Log the message if we didn't alert
  else {
    logInfo(logName, msg.toString)
  }
end

rule "Timer expired for a door"
when
  Member of gDoorsTimers received command OFF
then
  val doorName = transform("MAP", "en.map", triggeringItem.name)

  aAlert.sendCommand(doorName + " has been open for over an hour")

  if(vTimeOfDay.state.toString == "NIGHT" || vTimeOfDay.state.toString == "BED") {
        triggeringItem.sendCommand(ON) // reschedule the timer
  }
end

The above makes use of Expire based Timers. But basically it’s one rule to handle notifications for all my doors.

Hey Rich,

Thanks for letting me know about the “member of changed” trigger. I’ll have to look at the rules I mentioned and see if that will help me simplify them. Your example is also helpful to me where you use the name of one item to determine the name of another item that you are modifying. When I was trying to learn xtend years ago I couldn’t seem to figure out how to do that and it didn’t seem possible to get names of items. But, I may have just missed some important docs or something.

Do you happen to know if there’s a way to determine whether an item with a given name exists? With this, I can say “if item xyz_userconfig works, do a, else do default b” and similar logic. This was a crucial use of the items registry in jsr223 for me and it allowed me to really create the flexible rules I wanted that apply the same rule to slightly different rooms or environments.

Look at the Design Pattern postings. This particular DP is called Design Pattern: Associated Items.

DPs are not really documenting the language but showing ways to use the language to solve common problems.

There is no way I know of built in to OH. However there are a couple of approaches you can use.

  1. Make sure every Item is a member of a global Group. Then if(gAll.members.findFirst[ i | i.name = "ItemName"] !== null) // Item exists.

  2. Use the REST API. `if(sendHttpGetRequest(“http://localhost:8080/rest/items/ItemName/state").contains("Item Foo does not exist!”)) // Item does not exist

There has been talk on the forum of wanting a default global Group that all Items belong to but I guess no one felt strongly enough about it to actually implement a PR to add it.

I’m only getting started with JSR223 Jython in OH2, and it’s working great. I’m about half way through converting my DSL rules.

I think you were asking about the Rules DSL, but using JSR223 Jython, it’s simply…

if "My_Test_Item" in items:
    #do stuff

I hope that you will take the lessons you learned and add to the JSR223 docs. I know you are planning on writing a side by side posting which would be awesome all by itself. But you are building up some priceless experience and are in the perfect position to write some how to get started guides.

I’m glad the transition is moving forward well for you!

You bet! I spent some time this weekend writing up the content of a lengthy post I’d referred to (with rule, lambda, etc., and will have the same in Jython). Expect a PM with a request for some proofing and suggestions! Then I’ll start adding some posts to the DPs, and try to follow up behind your posts (if I can keep up!) with some Jython examples too :slight_smile: It’s starting to click, but still a little awkward. JSR223 gives me the same “OMG, this is how it’s supposed to be!” feeling that I got when I first started using OH. So many possibilities! Using it to put together a binding would be incredible.

I can’t wait to get everything migrated so that I can then refactor to take advantage of what Jython and the JSR223 integration can offer. There are two areas left t tackle to complete my migration… using call instead of executeCommandLine, and timers (but I have some examples for this). There is definitely a gap in the docs for this, so I’ll do my best to try and smooth it out.

Shouldn’t be too hard. The OAuth2 using just OH Rules and myopenhab.org example might be a good place to start. It’s not a full up binding but building a Jython library like it would be really useful. Of course, since you are in Jython, you can/should use a Python Oauth2 library in the first place so you code would really only need to do the interactions with OH Items and such.

I would have expected the code for executeCommandLine to use subprocess or subprocess32 or the like similar to what I did here. Is call a link to executeCommandLine in OH or is it some other Python library for launching scripts. Note, what I have above blocks while the script executes so it is more like executeCommandLine with the timeout.

:+1:

call is in subprocess. Haven’t dove into it though.

Here is how I am simulating executeCommandLine in Jython…

rawMessage = subprocess.Popen([/usr/bin/curl','-s','-X','GET','https://api.wunderground.com/api/<WUAPIkey>/alerts/hourly/conditions/q/pws:KOHBRUNS25.json'],stdout=subprocess.PIPE).communicate()
(out,err) = rawMessage.communicate()
# do stuff with stderrdata...rawMessage[1]
# do stuff with stdoutdata... rawMessage[0]

Timeouts were added to in Python 3.3, but Jython hasn’t been updated in a while. So I put this into a module:

import subprocess, threading
from org.slf4j import Logger, LoggerFactory

log = LoggerFactory.getLogger("org.eclipse.smarthome.model.script.Rules")

class Command(object):
    def __init__(self, cmd):
        self.cmd = cmd
        self.process = None
        self.data = None

    def run(self, timeout):
        def target():
            #log.debug("JSR223: Command: Thread started")
            self.process = subprocess.Popen(self.cmd,stdout=subprocess.PIPE)
            self.data = self.process.communicate()
            #log.debug("JSR223: Command: Thread finished")
        thread = threading.Thread(target=target)
        thread.start()
        thread.join(timeout)
        if thread.is_alive():
            log.debug("JSR223: Command: Timeout reached, terminating process")
            self.process.terminate()
            thread.join()
        #log.debug("JSR223: Command: return=[{}]".format(self.process))
        return self.data

And I use it like this…

from openhab.command import Command
rawMessage = Command(['/usr/bin/curl','-s','-X','GET','https://api.wunderground.com/<WUAPIkey>/alerts/hourly/conditions/q/pws:KOHBRUNS25.json']).run(timeout=10)
# do stuff with stderrdata...rawMessage[1]
# do stuff with stdoutdata... rawMessage[0]

I’m still a noob with Jython, but it seems to be working well! This post helped with timers, but this has more info.

1 Like