Python3 Binding (Discussion)

Thanks for the valuable feedback. I will improve the error logging for situations like that.


Next version will contain a error message like “Failed to inject import wrapper”. (It is already committed) :slight_smile:

Additionally I collect typical error messages with additional details in main README

…and I finally managed, in a couple of lines of Python, to do something that I had been struggling to do with the Rules DSL for years, thanks to a simple array. Many many thanks!

How do you integrate this binding with VSC ? Is there any syntax checking or highlighting or mouse over tooltips with item values like with the openhab addon ?
Breakpoints even?

not yet. I’m currently researching how to provide this.

As most of the python api is just the wrapped/proxied java api, maybe I can create dummy “placeholder” python packages, based on java packages. With these I could provide syntax highlighting and autocomplete.

But for now, I will focus only on testing, bug fixing, stabilizing and getting the pull request merged and released in the upcoming openhab5.

After that, I will spend more time on new featured like vsc support and others…

1 Like

:collision: Finally, the pull request is merged :collision:

means the binding will be official part of the upcoming openhab5 milestone 3

— UPDATE —

the new merged binding has a breaking change, compared to the previous release beta version. The transformation service is renamed from “PY3” to just “PY”

— UPDATE 07.05.2025 —

This version has a bug on windows. The helper libs are not deployed automatically. But I’m already working on a fix.

11 Likes

A new pythonscripting version arrived in the marketplace for openhab4

It contains same changes and featureset as the version for openhab5, plus the latest open pullrequest

  • fix list and set type mappings
  • fixed GenericEventTrigger
  • fixed ItemNotFound behavior
  • added addItem and safeItemName functions to Registry
  • cleanup usage of Dummy Set class => depends on latest openhab main branch

I’d like to suggest or request improved support for native module usage across multiple Python scripts in openHAB’s automation environment using GraalVM / GraalPy.

Currently, if I import a native module (such as requests, which internally relies on _ssl, _unicodedata, etc.) in one Python rule, and then import the same module in another rule (in a separate file), the second context fails to load due to GraalPy’s restrictions around native modules and context isolation. This makes it very difficult to build modular or reusable automation code.

While I understand the constraints around python.NativeModules and python.IsolateNativeModules, it would be incredibly helpful if openHAB could ensure that either:

  • all Python rules share the same GraalPy context (or at least a common native context), or
  • native module imports could be better coordinated between rules (e.g., loaded once and reused safely across scripts).

Without this, I’m limited to a single monolithic script or unsafe workarounds, which defeats much of the benefit of modular scripting in Python.

I’d really appreciate any improvements in this area or ideas on how this could be handled more gracefully within openHAB’s architecture. Thanks a lot to the team for the great work on bringing Python scripting to openHAB via GraalVM!

I tried it with following EXTRA_JAVA_OPTS:

EXTRA_JAVA_OPTS="
  -XX:+UnlockExperimentalVMOptions
  -XX:+EnableJVMCI
  -XX:+UseG1GC
  -Dpolyglot.engine.WarnInterpreterOnly=false
  -Dfile.encoding=UTF-8
  -Xms256m -Xmx1024m
  -Duser.timezone=Europe/Berlin
  -Dpython.NativeModules=true
  -Dpython.IsolateNativeModules=false
  -Dpython.ForceIsolation=false
"

What I’m trying to do, of course, is to use Python in such a way that I can also install some pip packages. It is possible that several packages are already using the same libraries in the background, for example. This often happens with numpy, requests etc.

I proceeded as follows:

Find out where openHAB has installed its automation environment:

sudo find /var/lib/openhab -type d -name "site-packages"

A typical path is, for example:

/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/site-packages

If you need several packages or work frequently, you can set an environment variable, for example:

export OH_SITE_PACKAGES=/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/site-packages

The next step is to ensure that pip3 is actually installed system-wide:

sudo apt install -y python3-pip

In the next step, we allow system packages to be overwritten.

sudo -u openhab python3 -m pip config set global.break-system-packages true

You can now install new packages as follows:

sudo -u openhab /usr/bin/python3 -m pip install --target=$OH_SITE_PACKAGES <package_name>

As example

sudo -u openhab /usr/bin/python3 -m pip install --target=$OH_SITE_PACKAGES requests

Then, of course, I created two files and imported requests. If I load only one file, no error message appears. If I load two or more, an error message appears:

SystemError, Option python.NativeModules is set to 'true' and a second GraalPy context attempted to load a native module '_cpython_unicodedata' from path '/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/graalpy24.2/modules/_cpython_unicodedata.graalpy242-311-native-x86_64-linux.so'. At least one context in this process runs with 'IsolateNativeModules' set to false. Depending on the order of context creation, this means some contexts in the process cannot use native module, all other contexts must fall back and set python.NativeModules to 'false' to run native extensions in LLVM mode. This is recommended only for extensions included in the Python standard library. Running a 3rd party extension in LLVM mode requires a custom build of the extension and is generally discouraged due to compatibility reasons.

Of course I played and experimented a bit with the parameters in /etc/default/openhab, but could not find a solution.

@Michdo93 Thanks for the feedback. I will analyze this behavior a bit more soon.

Can you try to reproduce this with the latest snapshot.

Specially with this ticket [jsscripting][pythonscripting] Use fully OSGI-ified Graal dependencies by ccutrer · Pull Request #18795 · openhab/openhab-addons · GitHub we fixed a lot related to a shared/osgiified polyglot/truffle context. Maybe this already fixes this issues.

The merge of this ticket was one day after milestone 3. So, you need to test with the latest snapshot.

I was able to reproduce it, but I think it is not easy to solve.

A shared context is not an option. Not only rules depends on their own python context. Also python transformation scripts or the upcoming mqtt homeassistant (a template parser is based on python) binding depends on their own context.

Maybe we have to wait until this ticket is solved.

I don’t know your use case, but for simple get requests you could try the HTTP action like below as a temporary workarround.

from openhab.actions import HTTP

result = HTTP.sendHttpGetRequest("https://openhab.org")
print(result)

This works if I execute requests directly in use cases. But if, for example, I use libraries in which requests would be used in the background, then I cannot use these libraries. So yes, your workaround works for many solutions, but not for all. But it does not solve the general problem.

Of course, I can also go and create a helper file with a lot of helper functions and then include all the libraries in it and import one helper function after the other into my other Python files. This may make this one help file extremely large and confusing. Of course, it also makes it difficult to reuse the code to a certain extent because it differs slightly from the original or actual functions. The legibility and comprehensibility of the code is also lost in the other files. People ask themselves why I don’t import the functions directly or possibly don’t understand what a function from the help file does.

I think there can only be small differences, because the auxiliary functions have to map all possible parameters of the actual function, otherwise the whole thing can’t work or you could only cover some of the possibilities.

And yes, I agree with you that there can be no simple solution. You probably have to actually work with such an auxiliary file and several auxiliary functions. Because the core of the problem is not the binding, but GraalPy and GraalVM. But perhaps I am overlooking possible solutions. You can of course write several help files. However, this becomes difficult from the point when requests or numpy are used in different libraries, for example. For libraries that do not rely on others in the background, you could also create a subfolder and create an auxiliary file for each library and then import the corresponding auxiliary functions into several Python files.

Nevertheless, I thought I would just put it up for discussion. It’s a problem and if there was a solution, it would bring immense possibilities.

What always works, of course, is to simply execute external Python 3 files via subprocess. Here, too, there are various ways in which data can be sent to items. So you can do a lot of tricks.

I don’t argue against your use case. I just explain the limitation of the graalvm. Currently there is no way to load a native library twice. Independently if the library is used in an imported module or directly in your script. An imported module is handled as it is directly embedded in your script. Means, every time an import statement is used in your script, the imported module is evaluated in this context again. Means if you “import requests” in 2 scripts. The native module is trying to be loaded twice.

I will play a bit with venv and hopefully get some findings how to handle this scenario better.

@Michdo93 Finally I got a prototype working. But it is not fully usable and will not be part of the upcoming release.

The following changes are needed

  1. You have to compile your own binding with this patch applied
  2. Install “patchelf”
  3. Copy .cache/org.graalvm.polyglot/python/python-home/{instanceid} to .cache/org.graalvm.polyglot/python/python-home/venv/
  4. create .cache/org.graalvm.polyglot/python/python-home/vtemp/

After all of that, you can import native modules like “requests” in several scripts. From time to time you get an exception where I currently not fully now, why. Maybe it is related to the way how it is internally initialized.

But anyway, the graalpy devs told me already that a lot of these limitation are eliminated in the upcoming graalvm version 25.0.x arriving in September.

So from my perspective, I will still watch this issues and maybe I got a better idea how to fix this in a cleaner way. But I think this will not happen, because this is currently the official recommended way.

All my hope is therefore on GraalVM version 25 in the fall, which will fix many of the current problems and hopefully provide a truly stable solution.

1 Like

Hi.
I’m trying to use the new python binding to slowly migrate rules from jython on OH 4.3.

I have encountered a few issues on the way:

  1. When installing the new python 3 binding, it tried to load the jython rule files. It was failing to do that and was spamming logs with lots of failures due to changes between jython and python3 rule files. The jython binding was installed at a time. I had to reinstall jython binding afterwards - that fixed the problem.
  2. I could not use ZoneInfo(“Europe/Warsaw”) (or any other zone I tried). Any idea how to solve this? For now I worked around this by using datetime.now().astimezone().

The log:

2025-07-06 22:18:13.337 [ERROR] [ps.Generic_UpdateLastMotionTimestamp] - Rule execution failed:
Traceback (most recent call last):
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/zoneinfo/_common.py", line 12, in load_tzdata
    return resources.files(package_name).joinpath(resource_name).open("rb")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/importlib/resources/_common.py", line 22, in files
    return from_package(get_package(package))
                        ^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/importlib/resources/_common.py", line 53, in get_package
    resolved = resolve(package)
               ^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/importlib/resources/_common.py", line 44, in resolve
    return cand if isinstance(cand, types.ModuleType) else importlib.import_module(cand)
                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ModuleNotFoundError: No module named 'tzdata'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/etc/openhab/automation/python/lib/openhab/helper.py", line 198, in executeWrapper
    status = rule_obj(module, input)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/etc/openhab/automation/python/generic-sensors-timestamps.py", line 38, in Generic_UpdateLastMotionTimestamp
    currentTime = datetime.now(ZoneInfo("Europe/Warsaw"))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/zoneinfo/_zoneinfo.py", line 43, in __new__
    instance = cls._weak_cache.setdefault(key, cls._new_instance(key))
                                               ^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/zoneinfo/_zoneinfo.py", line 70, in _new_instance
    file_obj = _common.load_tzdata(key)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/zoneinfo/_common.py", line 24, in load_tzdata
    raise ZoneInfoNotFoundError(f"No time zone found with key {key}")
zoneinfo._common.ZoneInfoNotFoundError: 'No time zone found with key Europe/Warsaw'

And the question.
I’ve had jython rules that used the following code to send telegram messages:

telegramAction = scope.actions.get("telegram", "telegram:telegramBot:BotID")
telegramAction.sendTelegram(text)

Is it possible to use telegram action with a new binding as was possible in jython?

I was not able to reproduce this. I tested it with openhab4 and openhab5 and on both systems, scripts are properly separated.

This is confirmed and is related to missing additional modules. Currently this is not easy to handle.

You must manually install this package with an external pip

  1. you must find your python cache folder by calling (replace /openhab with your path)
find /openhab -type d -name "site-packages"
  1. You must install the missing tzdata package by calling (replace --target=… with your path)
python3 -m pip install --target=/openhab/.cache/org.graalvm.polyglot/python/python-home/e41832d1c8ba537cb2b83ce5a12dcd2887406294/lib/python3.11/site-packages tzdata

maybe with the next pythonscripting version, we will install these kind of base packages by default

This is possible. You can fully access the openhab API like before.

I wanted to install the pythonscripting add on.

I am on OH 4.3.5 based on openhabian on a RasPi 4 with 4GB.

As I had been working with Jython since 2023, I have still my old Jython scripts and some libraries in the folders like /automation/jsr223/python and /automation/lib/python.

I am getting the error:

2025-07-10 00:13:01.845 [INFO ] [g.internal.PythonScriptEngineFactory] - Newest helper lib version is deployed.

2025-07-10 00:13:02.418 [ERROR] [t.ScriptTransformationServiceFactory] - bundle org.openhab.core.automation.module.script:4.3.5 (165)[org.openhab.core.automation.module.script.ScriptTransformationServiceFactory(291)] : The setScriptEngineFactory method has thrown an exception

java.lang.InternalError: java.lang.ExceptionInInitializerError

	at com.oracle.truffle.runtime.ModulesSupport.initializeModulesAccessor(ModulesSupport.java:178) ~[?:?]

	at com.oracle.truffle.runtime.ModulesSupport.<clinit>(ModulesSupport.java:57) ~[?:?]

	at com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntimeAccess.createRuntime(HotSpotTruffleRuntimeAccess.java:85) ~[?:?]

	at com.oracle.truffle.runtime.hotspot.HotSpotTruffleRuntimeAccess.getRuntime(HotSpotTruffleRuntimeAccess.java:76) ~[?:?]

	at com.oracle.truffle.api.Truffle.createRuntime(Truffle.java:145) ~[?:?]

	at com.oracle.truffle.api.Truffle$1.run(Truffle.java:176) ~[?:?]

	at com.oracle.truffle.api.Truffle$1.run(Truffle.java:174) ~[?:?]

	at java.security.AccessController.doPrivileged(AccessController.java:318) ~[?:?]

	at com.oracle.truffle.api.Truffle.initRuntime(Truffle.java:174) ~[?:?]

	at com.oracle.truffle.api.Truffle.<clinit>(Truffle.java:63) ~[?:?]

	at com.oracle.truffle.api.impl.Accessor.getTVMCI(Accessor.java:1643) ~[?:?]

	at com.oracle.truffle.api.impl.Accessor$Constants.<clinit>(Accessor.java:1497) ~[?:?]

	at com.oracle.truffle.api.impl.Accessor.engineSupport(Accessor.java:1564) ~[?:?]

	at com.oracle.truffle.api.library.LibraryAccessor.engineAccessor(LibraryAccessor.java:57) ~[?:?]

	at com.oracle.truffle.api.library.LibraryFactory.loadExternalDefaultProviders(LibraryFactory.java:439) ~[?:?]

	at com.oracle.truffle.api.library.LibraryFactory.getExternalDefaultProviders(LibraryFactory.java:430) ~[?:?]

	at com.oracle.truffle.api.library.LibraryFactory.initDefaultExports(LibraryFactory.java:213) ~[?:?]

	at com.oracle.truffle.api.library.LibraryFactory.<init>(LibraryFactory.java:208) ~[?:?]

Directly after that, it reloads the script files and then throws another error:

2025-07-10 00:13:02.456 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/fahrten.py'

2025-07-10 00:13:02.457 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/core/000_Startup.py'

2025-07-10 00:13:02.458 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/feiertage.py'

2025-07-10 00:13:02.459 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/Licht.py'

2025-07-10 00:13:02.461 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/weather.py'

2025-07-10 00:13:02.462 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/Anruf.py'

2025-07-10 00:13:02.463 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/rolladen.py'

2025-07-10 00:13:02.464 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/absFeuchte.py'

2025-07-10 00:13:02.466 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/heizung.py'

2025-07-10 00:13:02.467 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/core/components/100_StartupTrigger.py'

2025-07-10 00:13:02.468 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/smartmeter.py'

2025-07-10 00:13:02.470 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/heizungFenster.py'

2025-07-10 00:13:02.471 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/Squeeze.py'

2025-07-10 00:13:02.472 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/personal/energy.py'

2025-07-10 00:13:02.473 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/jsr223/python/core/components/100_DirectoryTrigger.py'

2025-07-10 00:13:02.456 [ERROR] [al.provider.ScriptModuleTypeProvider] - bundle org.openhab.core.automation.module.script:4.3.5 (165)[org.openhab.core.automation.module.script.internal.provider.ScriptModuleTypeProvider(300)] : The setScriptEngineFactory method has thrown an exception

java.lang.NoClassDefFoundError: Could not initialize class org.openhab.automation.pythonscripting.internal.PythonScriptEngine

	at org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory.createScriptEngine(PythonScriptEngineFactory.java:127) ~[?:?]

	at org.openhab.core.automation.module.script.internal.ScriptEngineFactoryHelper.getParameterOption(ScriptEngineFactoryHelper.java:45) ~[?:?]

	at org.openhab.core.automation.module.script.internal.provider.ScriptModuleTypeProvider.setScriptEngineFactory(ScriptModuleTypeProvider.java:150) ~[?:?]

	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]

	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]

	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

Nearly all of my .py files are commented out completely, but there is a new rule for testing I wrote. The corresponding file is loaded (I see that in the logs, not posted here), but no rule is added and I cannot see any other reaction on that file except for the information that it is loaded.

Could you help me tracking down or fixing this error?

I see. Let’s leave it there - maybe someone else will encounter something similar. I’ve already transitioned all my jython rules to a new binding and won’t be able to reproduce it any further.

That would be great!

Thanks. It’s working.

I need more information about your setup.

The path /etc/openhab/automation/jsr223/python/ shows me that these rules are not related to pythonscripting, because the pythonscripting path is /etc/openhab/automation/python/

Also in the first exception log, I can’t see anything what helps me to identify the problem. This exception could also be related to any other binding which uses grallvm.

Maybe I would simplify your setup to narrow down the problem. As a first step I would temporary remove all jython rules and all python rules except one of your testing rules. Now focus only on this single rule and check if there are still problems.

As I told, I have still all my old Jython rule files there, but all the code is commented out. Anyway, I removed the whole jsr223 folder from /etc/openhab/automation. Did not see any reaction in the logs. Then, I edited the file

/etc/openhab/automation/python/chargingoptimizer.py

and saw only

2025-07-10 08:17:34.847 [INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/etc/openhab/automation/python/chargingoptimizer.py'

The errors I had shown before have occurred after installation of the pythonscripting addon, and they always occur with a new start of openhab or the whole system.