Python3 Binding (Discussion)

What do you think would be a perfect default behavior? You can already

  1. enable/disable the scope/import wrapper. If it is disabled you have the global jsr223 variables back again
  2. enable/disable the availability of the helper libs (deployment and update)
  3. enable/disable pyc caching
  4. enable disable jython emulation

I think a perfect default behavior for UI based rules would be the automatic import of 3 things

  1. the “scope” object which wraps all jsr223 objects
  2. the “Registry” object which is more or less the entrance point for the whole API
  3. the logger object

This means I would inject 2 statements for all UI based rules by default

import scope
from openhab import Registry, logger

And I would inject these statements only if

  1. scope/import wrapper is enabled
  2. the helper libs are enabled
  3. maybe a new “inject” setting to control this behavior

I don’t really know what the best defaults would be. Ideally controlling whether or not stuff gets injected is a strong users can access through MainUI.

Beyond that, when enabled users should have access to most everything you’d usually want to do in a rule without imports like getting states of Items, commands and updates, actions, item metadata, etc. If all that is available in those two and those can be injected by default and the setting for that is accessible through the UI then it all should be good

I only posted to make sure that the UI users are not forgotten. Often choices which make perfect sense for file users and may even be programming best practices can impose a high burden on UI users, and these are the users least able to accommodate that burden.

It sounds like it’s all good.

Because I think it is a very useful feature, I just implemented it. :slight_smile:

There is now a new release in the marketplace which implements auto injection for all non file based rules (UI Rules and Transformation Scripts). And it is configurable.

There are 3 config options now

  1. Disabled
  2. Enabled for all scripts
  3. Enabled for UI and Transformation scripts (Default)

UI users were not forgotten :wink:

2 Likes

Another new release is available in the marketplace

Changes are

  • script context cleanup/close fix for compiled scripts (e.g. transformation scripts)
  • log prefix for UI based and transformation scripts fixed
  • logging cleanup and simplification

Hi

I’m trying to run my pythonscript. However, I always get the error: “ModuleNotFoundError, No module named”
I put the module in “/conf/automation/python/lib/”.
Do I have to consider anything else to continue importing modules correctly?

There is nothing special to consider.

I tried to reproduce, but without success.

I created a file __init__.py inside the folder /conf/automation/python/lib/named/ with the content

TEST = "1"

Then I created a test.py inside /conf/automation/python/ with the content

import named
print(named.TEST)

The log shows me that everything works.

2025-02-26 15:30:41.271 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/openhab/conf/automation/python/test.py'
2025-02-26 15:30:41.528 [INFO ] [b.automation.pythonscripting.test.py] - 1

Maybe you can provide more details.

  • are you using the latest binding version?
  • which openhab version are you running
  • what kind of module are you talking
  • how are you using it (how looks your import statement)
  • are you running a file based script or a script created via webui
  • have you checked the file permissions?
  • is there anything else in your openhab logs or systemd journal

Thank you for the quick response.
It is working now, was my fault.
I put the modules in /conf/automation/python/lib/openhab/

just want to mention this discussion here.

I’ve started writing a migration guide to help porting the old legacy helper lib scripts. However, it’s far from complete.

Since I’m lacking some perspective on your porting efforts and each perspective is different, I’m looking forward to receiving suggestions and experience reports so I can incorporate them into the guide and include relevant examples.

check MIGRATION.md

3 Likes

a new release is available in the marketplace

it contains a lot of cleanups and fixes, but the biggest highlight is the upgrade to graalvm 24.2.0. There, the highlight is, that a lot more polyglot.Foreign objects are inheriting from native python types. Means, they feel and behave like the native python object.

Foreign objects are now given a Python class corresponding to their interop traits. Foreign lists now inherit from Python list, foreign dictionaries from dict, foreign strings from str, foreign iterators from iterator, foreign exceptions from BaseException, foreign numbers from polyglot.ForeignNumber, foreign booleans from polyglot.ForeignBoolean, and foreign null values from NoneType. This means all Python methods of these types are available on the corresponding foreign objects, which behave as close as possible as if they were Python objects.

1 Like

Congrats on this binding, it’s very welcome for someone who’s been looking at Jython but frustrated by the lack of Python3 compatibility.
Is this binding targeting inclusion in the upcoming Openhab 5 release?

@lolando yes, it is planed to be included in the upcoming openhab5 release.

Currently only the final review and merge process has to be completed!

if you want to collect early experiences with this binding, you can already install it in openhab4. There is only a “bug” which requires additional openhab restarts after the binding is installed and activated the first time. More details can be found here. Except this “bug”, it works stable and is production ready.

I haven’t been able to get it working, unfortunately. Even with an empty test.py file, I get the following logs:

==> /var/log/openhab/openhab.log <==
2025-04-22 10:47:12.419 [DEBUG] [cripting.internal.PythonScriptEngine] - Lock acquired before invocation.
2025-04-22 10:47:12.420 [DEBUG] [cripting.internal.PythonScriptEngine] - Initializing GraalPython script engine…
2025-04-22 10:47:12.422 [ERROR] [ipt.internal.ScriptEngineManagerImpl] - Error during evaluation of script ‘/etc/openhab/automation/python/test.py’: null
2025-04-22 10:47:12.423 [WARN ] [ort.loader.AbstractScriptFileWatcher] - Script loading error, ignoring file ‘/etc/openhab/automation/python/test.py’
2025-04-22 10:47:12.424 [DEBUG] [cripting.internal.PythonScriptEngine] - Lock acquired before invocation.
2025-04-22 10:47:12.425 [DEBUG] [cripting.internal.PythonScriptEngine] - Initializing GraalPython script engine…

This is on Debian bookworm, openhab+openhab-addons 4.3.4-1; I installed from the marketplace web UI and restarted a couple of times to be sure (even though I never saw the " Native library already loaded in another classloader" message). Am I missing something?

@lolando

the log line “[ERROR] [ipt.internal.ScriptEngineManagerImpl] - Error during evaluation of script ‘/etc/openhab/automation/python/test.py’: null” means normaly that the script has a syntax error.

I tried to reproduce it with a new clean openhab docker container and I installed the binding via marked place without any issues. I tested script created via web ui and created via script files. Both cases was working. Also an empty script was not a problem.

The message “Native library already loaded in another classloader” you get only, if there are other use cases for the graal vm, like javascript usages or js/python scripts as a transformation script.

In my test scenario I didn’t get any message too.

Could it be a permission or owner problem? Are all default options enabled in your python binding settings? If the helper libs are disabled, you get not the full error trace. In this case, you have to handle the “sys.excepthook” by your own.

Normaly, there should no DEBUG messages visible, except you changed something in the default settings.

I did use the log:set DEBUG org.openhab.automation.pythonscripting command in the console to enable debugging, but I didn’t configure anything specifically. In particular, I didn’t do anything related to the helper libs (not even sure how/where to enable/configure them), which might be the problem. I ran strace -eopenat to see what files are accessed:

  • /etc/openhab/automation/python/test.py is opened fine
  • plenty of /var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca175db97c1b/lib/python3.11/os.py (and similar files) are opened fine, but followed by openat(AT_FDCWD, "<frozen os>", O_RDONLY) = -1 ENOENT (No such file or directory) (not sure what to make of that)
  • at the end, I see a openat(AT_FDCWD, "/etc/openhab/automation/python/lib/openhab/__wrapper__.py", O_RDONLY) = -1 ENOENT (No such file or directory)

The last line looks suspicious. Should there be anything in that /etc/openhab/automation/python/lib/ dir?

yes, there should be the helper libs.

when you install the binding, there should be a log line like

2025-04-22 10:40:29.772 [INFO ] [g.internal.PythonScriptEngineFactory] - Deploy helper libs into /openhab/conf/automation/python/lib/openhab.

also if the folder is deleted, it should be reinstalled after a openhab restart.


I was also able to reproduce your error by deleting the lib folder.

Now the final question is… Why you don’t have a lib folder? The following steps I already tried.

  1. New installation of openhab
  2. Install addon via markedplace => helper libs are deployed in lib/openhab
  3. manual deletion of lib
  4. openhab restart => helper libs are deployed in lib/openhab
  5. manual deletion of lib/openhab
  6. openhab restart => helper libs are deployed in lib/openhab

Here’s an excerpt of the logs when I reinstall the binding after removing it:

2025-04-22 11:23:30.306 [DEBUG] [g.internal.PythonScriptEngineFactory] - Loading PythonScriptEngineFactory
2025-04-22 11:23:30.325 [INFO ] [g.internal.PythonScriptEngineFactory] - Deploy helper libs into /etc/openhab/automation/python/lib/openhab.
2025-04-22 11:23:30.332 [ERROR] [g.internal.PythonScriptEngineFactory] - Exception during helper lib initialisation
java.nio.file.AccessDeniedException: /etc/openhab/automation/python/lib
        at sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397) ~[?:?]
        at java.nio.file.Files.createDirectory(Files.java:700) ~[?:?]
        at java.nio.file.Files.createAndCheckIsDirectory(Files.java:807) ~[?:?]
        at java.nio.file.Files.createDirectories(Files.java:793) ~[?:?]
        at org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory.initHelperLib(PythonScriptEngineFactory.java:186) ~[?:?]
        at org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory.<init>(PythonScriptEngineFactory.java:94) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
        at org.apache.felix.scr.impl.inject.internal.ComponentConstructorImpl.newInstance(ComponentConstructorImpl.java:326) ~[?:?]
        at org.apache.felix.scr.impl.manager.SingleComponentManager.createImplementationObject(SingleComponentManager.java:286) ~[?:?]
        at org.apache.felix.scr.impl.manager.SingleComponentManager.createComponent(SingleComponentManager.java:115) ~[?:?]
        at org.apache.felix.scr.impl.manager.SingleComponentManager.getService(SingleComponentManager.java:1002) ~[?:?]
        at org.apache.felix.scr.impl.manager.SingleComponentManager.getServiceInternal(SingleComponentManager.java:975) ~[?:?]
--
        at org.eclipse.osgi.container.Module.start(Module.java:486) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel$2.run(ModuleContainer.java:1847) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.internal.framework.EquinoxContainerAdaptor$1$1.execute(EquinoxContainerAdaptor.java:136) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1840) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.incStartLevel(ModuleContainer.java:1783) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.doContainerStartLevel(ModuleContainer.java:1745) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1667) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.container.ModuleContainer$ContainerStartLevel.dispatchEvent(ModuleContainer.java:1) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234) [org.eclipse.osgi-3.18.0.jar:?]
        at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:345) [org.eclipse.osgi-3.18.0.jar:?]
2025-04-22 11:23:30.359 [DEBUG] [g.internal.PythonScriptEngineFactory] - bundle org.openhab.automation.pythonscripting:4.3.3.202503251809 (390)[org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory(395)] : getting activate: activate

Looks like I should change the owner of the automation/python dir to be writable by openhab… let me try that and report.

so, it is a permission problem.

openhab is not able to deploy/install the helper libs into “/etc/openhab/automation/python/lib”

I try to add a more helpful error message.

It does work now, thanks a lot!