Error with openhab(?) py script 100_DirectoryTrigger.py after Update to 4.0.1

I’m not sure over the years if this script was part of openhab 3.x versions, but the error is new after updating to openhab Version 4.0.1:

2023-07-30 20:14:55.848 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py'
2023-07-30 20:14:56.398 [ERROR] [ipt.internal.ScriptEngineManagerImpl] - Error during evaluation of script '/etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py': ImportError: No module named smarthome in /etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py at line number 21
2023-07-30 20:14:56.398 [WARN ] [ort.loader.AbstractScriptFileWatcher] - Script loading error, ignoring file '/etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py'

Here the script (which I could not remember to put it there, so believe part of distribution)

# pylint: disable=eval-used
"""
This trigger can respond to file system changes. For example, you could watch a
directory for new files and then process them.
"""
from java.nio.file.StandardWatchEventKinds import ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY

try:
    scriptExtension.importPreset(None)# fix for compatibility with Jython > 2.7.0
except:
    pass

try:
    from org.openhab.core.automation.handler import TriggerHandler
except:
    from org.eclipse.smarthome.automation.handler import TriggerHandler

try:
    from org.openhab.core.service import AbstractWatchService
except:
    from org.eclipse.smarthome.core.service import AbstractWatchService

import core
from core.log import logging, log_traceback, LOG_PREFIX

LOG = logging.getLogger("{}.core.DirectoryEventTrigger".format(LOG_PREFIX))
core.DIRECTORY_TRIGGER_MODULE_ID = "jsr223.DirectoryEventTrigger"

scriptExtension.importPreset("RuleSimple")
scriptExtension.importPreset("RuleSupport")
scriptExtension.importPreset("RuleFactories")


class JythonDirectoryWatcher(AbstractWatchService):

    def __init__(self, path, event_kinds=[ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY], watch_subdirectories=False):
        AbstractWatchService.__init__(self, path)
        self.event_kinds = event_kinds
        self.watch_subdirectories = watch_subdirectories
        self.callback = None

    def getWatchEventKinds(self, path):
        return self.event_kinds

    def watchSubDirectories(self):
        return self.watch_subdirectories

    @log_traceback
    def processWatchEvent(self, event, kind, path):
        if self.callback is not None:
            self.callback(event, kind, path)


@log_traceback
class _DirectoryEventTriggerHandlerFactory(TriggerHandlerFactory):

    @log_traceback
    class Handler(TriggerHandler):

        @log_traceback
        def __init__(self, trigger):
            TriggerHandler.__init__(self)
            self.rule_engine_callback = None
            self.trigger = trigger
            config = trigger.configuration
            self.watcher = JythonDirectoryWatcher(
                config.get('path'), eval(config.get('event_kinds')),
                watch_subdirectories=config.get('watch_subdirectories'))
            self.watcher.callback = self.handle_directory_event
            self.watcher.activate()

        def setCallback(self, callback):
            self.rule_engine_callback = callback

        @log_traceback
        def handle_directory_event(self, event, kind, path):
            self.rule_engine_callback.triggered(self.trigger, {
                'event': event,
                'kind': kind,
                'path': path
            })

        def dispose(self):
            self.watcher.deactivate()
            self.watcher = None


    def get(self, trigger):
        return _DirectoryEventTriggerHandlerFactory.Handler(trigger)


def scriptLoaded(*args):
    automationManager.addTriggerHandler(
        core.DIRECTORY_TRIGGER_MODULE_ID,
        _DirectoryEventTriggerHandlerFactory())
    LOG.info("TriggerHandler added '{}'".format(core.DIRECTORY_TRIGGER_MODULE_ID))

    automationManager.addTriggerType(TriggerType(
        core.DIRECTORY_TRIGGER_MODULE_ID, None,
        "a directory change event is detected.",
        "Triggers when a directory change event is detected.",
        None, Visibility.VISIBLE, None))
    LOG.info("TriggerType added '{}'".format(core.DIRECTORY_TRIGGER_MODULE_ID))


def scriptUnloaded():
    automationManager.removeHandler(core.DIRECTORY_TRIGGER_MODULE_ID)
    automationManager.removeModuleType(core.DIRECTORY_TRIGGER_MODULE_ID)
    LOG.info("TriggerType and TriggerHandler removed '{}'".format(core.DIRECTORY_TRIGGER_MODULE_ID))

Not sure, but recognized that changed textual things configuration files not recognized in 4.0.1. Maybe the reasin is this script is not working?

That ( smarthome ) looks more like it would have been before OH3 ( as the namespace smarthome was part of OH2 or even older ). So I would expect that the same error ( no module ) also popped up under OH3.

I just checked earlier logs: No Errors in last stable 3.4 Version of openhab, it has normally loaded before migration.

2023-07-07 19:35:33.931 [INFO ] [ort.loader.AbstractScriptFileWatcher] - Loading script '/etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py'
2023-07-07 19:35:34.728 [INFO ] [ort.loader.AbstractScriptFileWatcher] - Loading script '/etc/openhab/automation/jsr223/python/core/components/100_StartupTrigger.py'

<edit: Translated, had chosen wrong language! Sorry>
Fact is, changed rules, things and items files are not loaded automatically without restart openhab :-/
</ edit>

That does not come from the OH distribution. It comes from the Helper Library.

Unfortunately the Helper Library is a third party library and was never a part of the openHAB project. And as far as I’m aware there is no one maintaining that library any longer. But it appears there is at least one breaking change for OH 4 that needs to be fixed for it to work. There is no longer an AbstractWatchService so it’s no surprise that the import fails on that line. The whole watch service parts of OH were rewritten.

Unfortunately this may mean that the end of Jython support has come sooner than I expected unless and until someone volunteers to rework the helper libraries to make them compatible with this any any other breaking changes introduced in OH 4.

Strange… I have lots of Jython scipts and I am using the Helper Library since OH 3.0 or 3.1.

I recently upgraded to OH 4.0 and everything is working. I remember I got this error message at some time, but I ignored it and I have no problems. I have also added a couple of new code lines and even added new rules to different .py files. They were all recognized as always after having saved the respective file.

Maybe you have a newer / other version of helper library?

Honestly, I took the effort once to install the helper library and then never cared about it again. So I neither remember how I installed it (I was following some tutorial to do it) nor how to check the version. Sorry! But if you have a clue how to check versions I am ready to do the check.

This is why the helper library should be a part of the openHAB project and come with the add-on itself.

Originally you at some point needed to clone the repo using git. If you do a git pull from that folder it will pull the latest version from GitHub. The fact that it’s working at all in OH 3 means that it’s pulling from the cloned repo.

But let this be a warning to everyone. Jython support is hanging on by a thread. I strongly recommend spending the effort now to gradually move your rules to something else (if you like Python HABApp is a great choice) while you are not under pressure because it broke.

1 Like

I see the same error here.
I did a test, renamed an almost empty script file so it now has the .py extension and the script got reloaded and executed immediately. From this it looks to me like the script 100_DirectoryTrigger.py is not needed for script reload.

Thanks Rich for your advice.

Somehow, I am a little bit puzzled.

Some 1.5 - 2 years ago, I got the impression from some forum entries that DSL rules could be deprecated some day (and I did not like the language) and that the new future should be JSR223 scripting, where there were three languages to choose, from which I chose Python, as it is widely used also in other contexts.

I researched how to do it in an easy way, found the helper libraries, and started. Your blog entries helped a lot to get through the chaos. I ported all my rules, developed much more, and I am really happy with it.

Now, I have a large (for my perspective) code base for rules (>5000 lines of code), and I am encouraged again to change to another “system”, HABApp. What about the Jython scripting add-on that is part of openHAB?

Will HABApp really maintained the next years? Can I really use my triggers as I did before (item changed, updated, Cron, …)? At first glance, constructing a rule looks a little complicated …

Thanks again in advance for guiding me through this.

2 Likes

All I can say is that things change.

When OH 3 was released the developer maintaining the Jython add-on and the Helper Library disappeared. A fork was made and there was a scramble to make the helper library compatible with OH 3 as only the one person who disappeared had any rights on the original repo. Since then there has been no one volunteer to support the Jython add-on and the Helper Library hasn’t seen a merged PR for at least two years.

And all that lack of work and interest from the OH side of things probably has something to do with the fact that there has been no progress on the upstream Jython project itself. Python 2.7 has been end of life for several years now yet the Jython project still shows no sign of Python 3 support any time soon. And there is very little working being done on that project at all.

No one can predict the future with 100% accuracy. At one time Jython’s future looked bright. Not any more.

All the more reason to start looking at migrating now when you have the chance to take your time rather than later when there’s an update somewhere that completely breaks Jython for good.

My understanding it there is very little that HABApp cannot do that the built in OH rules can. But keep in mind that most of the rules syntax stuff you are used to working with was provided by the Jython Helper Library. So anything is going to look different from what you are used to.

What about it? No one is really maintaining it. But as things change in openHAB core that require changes to the add-on to keep it compiling and running maintainers do step in and make those changes. However, they are not making additions to the add-on to account for new features added to core. Also, the Helper Library isn’t part of openHAB. The core Jython capability itself is not part of openHAB. Those are both third party and neither really has sufficient support any longer. All the work in the world on the add-on doesn’t matter if the upstream Jython or downstream Helper Libraries are not kept up.

Some of the features in OH 4 that I’m pretty sure Jython doesn’t support even now include:

  • new rule triggers like Time is <item> and system startlevel triggers
  • sharedCache and privateCache which lets you store stuff outside of the rule and, in the case of sharedCache share between rules. The caches are nice in that they clean up after themselves if all the rules that reference an entry are reloaded to include cancelling timers.
  • features to make coding rules in the UI reasonable like auto injecting the helper library rather than requiring the same explicit imports for every single script action/condition.

:person_shrugging: I see no reason why it wouldn’t be. But it too is not a part of the official openHAB repos. I don’t know what, if any, succession plans they have should the lead developer disappear for whatever reason. But it runs on standard Python so at least you don’t have to worry about that going away or breaking.

Most of the above lessons learned have been applied to the remaining languages in OH 4, particularly GraalJS (i.e. JS Scripting) and jRuby. In both of those cases the upstream capabilities are currently actively supported, the helper libraries are a part of the openHAB project (and come with the add-on), have multiple maintainers, and since they are a part of the openHAB project itself, the project leads can assign new maintainers should the need arise.

Blockly, the graphical programming language in MainUI “compiles” to GraalJS for what ever that’s worth.

And I suspect you might find that you can do more with less lines of code in both jRuby and GraalJS than you can with Jython, if lines of code is a concern. Also don’t forget to look at rule templates on the marketplace. Why recode a rule that you can just install and configure? Finally, look for other third party libraries which can help as well. For example I have openhab_rules_tools which provides a bunch of libraries for making working with timers a lot easier. For example, if you have a rule that needs to keep a separate timer for each Item that triggers the rule:

var {timerMgr, helpers} = require('openhab_rules_tools');
var timers = cache.private.get('timerMgr', () => new timerMgr.TimerMgr()); 

var runme = () => {
  // code to run when the timer goes off
}

var flapping = () => {
  // code to run when the timer already exists
}

timers.check(event.itemName, 'PT5M', runme, true. flapping);

That call to check will create a new Timer keyed on the Item name if one doesn’t already exist to go off in five minutes and call runme. If one does exists, reschedule it and call flapping. The last two three arguments are optional.

Thanks for sharing it.

We all need to learn a lot from you! So a very warm thank you for all you are doing.

Just coming back to original post giving an actual state:

  • Error on 100_Directory still exists, but seem to have no negative impact (at least on my system)
  • Reloading .py-Scripts after changed running again, maybe I just didn’t wait long enough after the update (about 1 or 2 hours after update, I still had the problem, now its gone)
  • Some confusion exists on my site, because my “old logging mechanism” in python (import logging from core.log) don’t cause an error, but logging not working anymore. Needed to change to Syntaxt based on import LogAction from core.actions

Still some scripts seems to have minor issues, maybe because of checking Number-Items with unspecified number-types.

I will report.

Thank you so much, Rich.

I will port my rules to something else. Even if HABApp looks great, I will also have a look at JS Scripting because my impression is that the structure (not the syntax) could be more similar to what I have at the moment.

That might be just a missmatch in the logger names and log4j2.xml. IIRC the logger names created by the helper library do not correspond to any of the loggers configured in log4j2.xml meaning the default logging level will be WARN. You can show if this is the case by logging something at the warn or error level and watching openhab.log. You should see the name of the logger as part of the log line.

When you use the LogActions, you are using the Java openHAB loggers same as you do from Rules DSL meaning most of the logger name is predefined for you based on the standard.

I started right away and tried to implement a very simple JS Scripting rule.
The add-on was already installed, I am on OH 4.0, and I tried to save the .js file to openhab-conf/automation/js. It said:no permission to make changes or add files here. So I chose “fix permissions” in the openhabian-config menu, then it worked.
First impression: It reacts extremely slow compared to Python. It took ~40 seconds to tell in the log:
Adding rule: …
Also the rule itself took 20s longer to report a changed item state in the log than the same Python rule.
Any similar experiences out there?
I mean, available memory (2 GB total) is very limited, but Python does not seem to have any problem with it…

I asked the question about performance here, think that it fits better into that thread:

I think there is a known bug that is in work being fixed regarding the slow first run of a JS Scripting rule.

But I’ll also just caution you that even if Jython is the fastest best option in the world, it doesn’t really matter. If sticking to Jython forever were an option you wouldn’t need to be going down this path in the first place.

I agree, but if there were serious performance concerns I would rather wait until either they are solved or it is urgent enough to spend more hardware.
Actually, at the moment it is also really slow in subsequent runs, not only in the first run. I am not seeing any major memory or CPU usage spikes.

There have been no reports of JS Scripting rules running slowing all the time. The first run it needs to be “compiled” and that can take some time and there was a problem with some changes in the way caching is implemented in the add-on. Are you using the helper library that comes with the add-on and is the Settings → Automation → JS Scripting ->Blue Gear Icon → Use Included Library checked? That should be the default.

Yes, that is checked.
Maybe Jython and JS scripting interfering with each other somehow?