Error with openhab(?) py script 100_DirectoryTrigger.py after Update to 4.0.1

The most important distinction is a command goes out to the binding and out to a device. An update only changes the state of the Item.

Philosophically, though commands should only be sent to actuators. Results of a command or sensor readings should be updates.

Typically such a rule would triggered by a changed trigger instead of a command.

I did the fresh installation and Jython seems to be broken now. Rule script files are loaded but do not execute. I have little time to investigate and now I am under pressure :-). Fortunately, I already ported 80% to JS, but as always the remaining 20% are the most difficult ones and also (as reported before) the most performance-critical ones. Of course I could use the old SD card and proceed until everything is ported… I will have to closely look how much is missing.

There really isn’t anything that has changed between when I wrote my reply above and now. So I doubt Jython is completely broken.

Yes, there were some rules that were working, and then it was quiet :-). I think I will get it running, but - as always - will be a pain.

OK! Everything working again, my fault: log level was changed by the update, as described in this thread here a couple of posts above. So now I can continue work on migration.
Fortunately, trigger delays of JSRule are gone.
Unfortunately, performance of JS Scripting has not improved. I will open a new thread for that. The differences are so significant that I am really interested in other experiences and opinions on that.

This weekend, I did a long deferred and overdue update from 2.x to 4.1.1. I came across this thread because I’m also using file based Jython scripts and had exactly the same error as described in the title.
TL;DR: Everything is still working like a charm, with only minimal modifications!

But first of all: you guys are amazing! The developers as well as the ones giving support here are doing such a great and ambitioned job that I cannot find in 95% of commercial software.

Let me try to wrap it up:

  • I’m on a ‘normal’ Debian, no Raspbian, Openhabian or whatsoever. So my installation is .deb-based
  • I did a manual backup of all necessary files as well as the database, because I was pretty sure that a simple update over two major versions won’t do (and even if I had been confident, it’s always a good idea to have a backup)
  • I have a wild mixture of file and JSON-DB based things and items, some simple DSL based rules, many complex Jython rules with a few system calls, and single JS-based transformations

So my first attempt was a naive update via apt install openhab - expecting that I have to repair it over the next days and weeks, maybe install a new version from scratch.
And: wow! I was so wrong about it: first thing I saw after the update was that nearly all things and items were still available, sitemaps in my OpenHab app were working, I was really happy.
I also appreciate that the Jython scripting is now an official add-on and I could remove my experimental jython-standalone configuration. Really great so far! But unfortunately, my Jython rules did not work. I have to admit it was painful and I spent a few hours to find out the reasons, but here is what I did:
I recognized that my OpenHab JSR223 helper libraries were out of date, so I followed the instructions here to install them for OH3 compatibility:

One fatal error I made: since I’m only using Jython rules, I just updated the Jython related helper scripts, not the JS helper scripts! This leads to a situation where the 000_Startup.py script is loaded, but not any other core or personal script.
The reason is most probably that there is an old script 000_startup_delay.js that blocks the whole JSR223 script loading literally forever, because it is waiting for a condition that never becomes true. Made me crazy, because depending on the timing during restart, sometimes single scripts that I renamed to 000_*.py were loaded, but most of the time they were not.
As soon as I recognized this and also updated the other helper scripts (JS and groovy, both mostly empty), I finally made quick progress.

I could not overcome the actual error about the 100_DirectoryTrigger.py. Like @rlkoshak already pointed out: the whole watchdog internals seem to have changed in OH3/4, so without someone updating the helper scripts I guess you don’t have a chance to get this working again.
But it turned out: I can safely ignore this error (or just remove the respective file from the core directory) - at least for my rules. I don’t exactly see what this script does for you anyway - modifications to my scripts are still watched and applied without a restart. And I don’t use custom directory triggers in my rules.

The only real changes I had to do in my Jython scripts were:

  • Get rid of org.joda.time imports
  • Replace the logger calls with something appropriate (I decided to use the from org.slf4j import LoggerFactory from the sample scripts, with logger names beginning with org.openhab.core.automation)

That’s it! All my rules are working again!
I defined many custom rule classes with sophisticated item and cron triggers, timer invocations and even semaphores / thread locking mechanisms - and they just work!

In my understanding, as long as future JVMs support JSR223, it is totally clear that Jython scripts works, because the Java runtime doesn’t really care about the actual scripting language that you use. It is only the helper scripts that become outdated - but even them are only to help you save some code, they are not mandatory to work with the API. You can always write and maintain your own helper libraries.

For now, despite the still broken 000_DirectoryTrigger.py, you can perfectly use it for OH4. That’s my point for today, for anyone who stumbles across this thread looking for help to get their Jython scripts up and running again.

1 Like

Yes this is only for you to create a directory trigger for your rules. It has nothing to do with your scripts being watched / loaded by openhab.

The implementation in openhab did change in 4.0 and that’s why it broke. It can/should be removed from the jython helper library.

At the moment, nobody I know is maintaining it.

1 Like

I believe it was a work around for a timing error during startup with OH 2.5. The startup behavior has undergone a lot of changes since then, maybe it’s no longer needed.

Obviously I don’t know what you are doing but the rule engine, even back in the 2.5 days, only executes one instance of a rule at a time meaning you can never have more than one instance of a rule running so the need for locks should be nearly eliminated. In OH 3 this same behavior was added for Rules DSL. In short, you get the locking for free now, at least on a per rule basis.

The real danger is that the upstream Jython library is all but abandoned. There also isn’t anyone on the OH side that is maintaining the add-on either. Jython should be considered deprecated and not used for any new development. Over time you should consider migrating your rules to a different language, one with more support. At some point, Jython is going to break and no one will be around to fix it. It may be tomorrow, it may be in five years, but it’s better to adjust now when you have time than later when you are under the gun.

It’s also worth mentioning that some new features may not be available. For example, I’m not sure that Jython has access to the shared and private cache.

1 Like

I looked into it, because I didn’t remember myself exactly :face_with_peeking_eye:
It is a simple light that I toggle with a single button of a remote control. So I have this one rule that reacts on the key press, checks the current light state and commands the opposite. And I have a second rule that reacts on a motion sensor and switches on the light, but only if the current luminance is beyond a threshold (i.e.: it is dark enough to turn the light on).
Maybe I overengineered it, but I wanted to eliminate any race conditions in case both rules happen to run at the same time. So I used a lock object for both rules to isolate them - that’s it.

Absolutely, I wouldn’t recommend that either. But I really like Python, and consider it the perfect language for small and readable automation tasks. It is the closest to executable pseudo code that I know of.
And at the same time, I could never get used to JavaScript in the last 25 years. So I celebrate each day that my syntax of choice is still supported (or at least: works). When the day comes, I might switch to HabApp like you suggested.

But let’s not focus on the things that might not work in future. My main point is that I’m very happy with the OpenHab ecosystem, and I’m overwhelmed from all the new add-ons that are available compared to OH1/2.

I found this add-on, ‘Virtual Solar Light Sensor’, that combines the current sun elevation and a ‘cloudiness’ value from a weather API to approximate the luminance outside. And this is exactly what I did in the last years with some rules to trigger my shutters in the evening. Nice to see that other people invent the same crazy workarounds, and spend hours just to save 20€ for a light sensor :sweat_smile:.

1 Like

I probably would have put them both into the same rule. Then you have more control over how to handle the race condition in the first place.

jRuby is pretty nice in that regard. It’s also more terse and reads more like English than Python in many ways.

I’m a generalist. I don’t advocate for any one language over the others, as long as those languages are roughly at parity in capabilities when it comes to OH. I use JS Scripting day to day mostly because it helps me support the Blockly users better.

Someday, when I have more time, I hope to get Clojure or some other functional language (I saw a thread where someone got Scheme working I think) working as a rules language for us old timers. :smiley:

I use that rule template myself. It’s written in JS Scripting and works quite well. The nice thing about rule templates is you don’t have to write the code yourself so it doesn’t really matter what language it’s written in.

If you like “small, readable, and closest to pseudo code”, check out
Python to Ruby Rule Conversion

Actually this looks very nice. Maybe worth to learn a new syntax.
Although the author’s obvious intent was to make look Python as ugly as possible. Python doesn’t even know the ‘&&’ operator :wink:

I think the intent was to show how to translate Jython with the third party helper library to jRuby in a way where most of the concepts have an obvious mapping.

  • How do I define a rule?
  • How do I define a trigger?
    etc.

If there are errors in the Python version on that page I’m sure that they would welcome a correction.

LOL :slight_smile:

I thought it was taken from the author’s actual python rule when he converted it to ruby. But yeah, if python doesn’t have a && operator, that does makes me wonder too. I am a co-maintainer / apprentice for the jruby library. I’ll fix that example and make the Python look as good as it can be! :slight_smile:

1 Like

I had the same idea when I thought about it a second time, and just came back here to answer to my own post - obviously you were much faster :smiley: No need to increase complexity with locks when you can just subscribe to multiple items in the first place. Didn’t think of that when I wrote them back in - must have been 2017/2018.

If I remember correctly you once documented some common patterns that cover most of the common use cases - yes, I found them:

I’m slowly getting really motivated to do a fresh start on some free weekend.

I can’t think of how to simplify or make the Python code any more succinct or nicer to read.

Yeah maybe I would have written it like this (no running code as well, but you get the idea). And it’s obvious I prefer camel over snake case - a real Python developer would sure hate me for that :smiley:

officeTemperatureItem = ir.getItem("Office_Temperature")
hallTemperatureItem = ir.getItem("Thermostats_Upstairs_Temp")
heatSetItem = ir.getItem("Thermostats_Upstairs_Heat_Set")
occupiedItem = ir.getItem("Office_Occupied")
doorItem = ir.getItem("OfficeDoor")

THRESHOLD = 2.0

@rule("Use Supplemental Heat In Office")
@when("Item Office_Temperature changed")
@when("Item Thermostats_Upstairs_Temp changed")
@when("Item Office_Occupied changed")
@when("Item OfficeDoor changed")
def office_heater(event):
  officeTemp = float(str(officeTemperatureItem.state))
  hallTemp = float(str(hallTemperatureItem.state))
  heatSet = int(str(heatSetItem.state))
  officeOccupied = str(occupiedItem.state) == "ON" or False
  doorOpen = str(doorItem.state) == "OPEN" or False
  
  difference = hallTemp - officeTemp
  
  if officeOccupied and not doorOpen and heatSet > officeTemp and difference > THRESHOLD:
    events.sendCommand("Lights_Office_Outlet","ON")
  else:
    events.sendCommand("Lights_Office_Outlet","OFF")

Going on a tangent… during our development of jruby library, we discovered that the object you got from ir.getItem could become stale, i.e. pointing to the old object that is no longer active, if the item got redefined (e.g. .items file edited / changed, or perhaps even managed UI item changed).

This is why in JRuby library, items are proxy-cached to avoid this issue, so it’s safe to do something like this in jruby (although not necessary, but here to demonstrate the idea)

door_item = OfficeDoor

rule do
  changed XXxxxxxx
  run do
    if door_item.open?
      # do something
    end
  end
end

Both the item constant OfficeDoor and the reference to it door_item will continue to point to the actual active item even if the item was updated throughout the lifetime of the script. Even items obtained through office_door = items["OfficeDoor"] is managed the same way so office_door doesn’t get stale in this case either.

So the safer way for the Python code is to keep the ir.getItem calls inside the rule, to be called every time it’s needed.

I know this is probably considered an edge case, but one that nonetheless could rear its ugly head.

From my experience, the ugly type conversions are necessary to do it the ‘Python’ way, but this due to Java/Jython object translation. Never cared much about it, perhaps there is a better way.

And since the items are already assigned by name, I would try to avoid passing the same name again in the annotations, and instead create the trigger as an object. After that, I would feel comfortable with that.

Plus: I intentionally ignored the Celsius to Fahrenheit conversion because - come on, there’s only one proper way to do it, isn’t it :smiley: Just kidding, of course this is just an example for a transformation - let me add it in a second.

But still, I see the advantages of the Ruby version.

I’m sure you’re right, so this might be a version that can be compared to Ruby:

THRESHOLD = 2.0

@rule("Use Supplemental Heat In Office")
@when("Item Office_Temperature changed")
@when("Item Thermostats_Upstairs_Temp changed")
@when("Item Office_Occupied changed")
@when("Item OfficeDoor changed")
def office_heater(event):
  officeTemp = ir.getItem("Office_Temperature").getStateAs(QuantityType).toUnit(ImperialUnits.FAHRENHEIT).floatValue()
  hallTemp = items["Thermostats_Upstairs_Temp"].floatValue()
  heatSet = items["Thermostats_Upstairs_Heat_Set"].intValue()
  officeOccupied = items["Office_Occupied"] == ON
  doorClosed = items["OfficeDoor"] == CLOSED
  
  difference = hallTemp - officeTemp
  
  if officeOccupied and doorClosed and heatSet > officeTemp and difference > THRESHOLD:
    events.sendCommand("Lights_Office_Outlet","ON")
  else:
    events.sendCommand("Lights_Office_Outlet","OFF")

I mean, the comparison is not ultimately fair, because in the Ruby example the items are already there and even proxied. Of course this could be achieved in Python as well, then the examples would be more similar. But in the context of ‘what is possible right now with Jython and JRuby integration’, I think this is a better example than the current one.