I’d like to suggest or request improved support for native module usage across multiple Python scripts in openHAB’s automation environment using GraalVM / GraalPy.
Currently, if I import a native module (such as requests
, which internally relies on _ssl
, _unicodedata
, etc.) in one Python rule, and then import the same module in another rule (in a separate file), the second context fails to load due to GraalPy’s restrictions around native modules and context isolation. This makes it very difficult to build modular or reusable automation code.
While I understand the constraints around python.NativeModules
and python.IsolateNativeModules
, it would be incredibly helpful if openHAB could ensure that either:
- all Python rules share the same GraalPy context (or at least a common native context), or
- native module imports could be better coordinated between rules (e.g., loaded once and reused safely across scripts).
Without this, I’m limited to a single monolithic script or unsafe workarounds, which defeats much of the benefit of modular scripting in Python.
I’d really appreciate any improvements in this area or ideas on how this could be handled more gracefully within openHAB’s architecture. Thanks a lot to the team for the great work on bringing Python scripting to openHAB via GraalVM!
I tried it with following EXTRA_JAVA_OPTS
:
EXTRA_JAVA_OPTS="
-XX:+UnlockExperimentalVMOptions
-XX:+EnableJVMCI
-XX:+UseG1GC
-Dpolyglot.engine.WarnInterpreterOnly=false
-Dfile.encoding=UTF-8
-Xms256m -Xmx1024m
-Duser.timezone=Europe/Berlin
-Dpython.NativeModules=true
-Dpython.IsolateNativeModules=false
-Dpython.ForceIsolation=false
"
What I’m trying to do, of course, is to use Python in such a way that I can also install some pip packages. It is possible that several packages are already using the same libraries in the background, for example. This often happens with numpy
, requests
etc.
I proceeded as follows:
Find out where openHAB has installed its automation environment:
sudo find /var/lib/openhab -type d -name "site-packages"
A typical path is, for example:
/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/site-packages
If you need several packages or work frequently, you can set an environment variable, for example:
export OH_SITE_PACKAGES=/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/python3.11/site-packages
The next step is to ensure that pip3 is actually installed system-wide:
sudo apt install -y python3-pip
In the next step, we allow system packages to be overwritten.
sudo -u openhab python3 -m pip config set global.break-system-packages true
You can now install new packages as follows:
sudo -u openhab /usr/bin/python3 -m pip install --target=$OH_SITE_PACKAGES <package_name>
As example
sudo -u openhab /usr/bin/python3 -m pip install --target=$OH_SITE_PACKAGES requests
Then, of course, I created two files and imported requests
. If I load only one file, no error message appears. If I load two or more, an error message appears:
SystemError, Option python.NativeModules is set to 'true' and a second GraalPy context attempted to load a native module '_cpython_unicodedata' from path '/var/lib/openhab/.cache/org.graalvm.polyglot/python/python-home/3fe684bec8ba537cb2b83ce56e79ca177f1e0290/lib/graalpy24.2/modules/_cpython_unicodedata.graalpy242-311-native-x86_64-linux.so'. At least one context in this process runs with 'IsolateNativeModules' set to false. Depending on the order of context creation, this means some contexts in the process cannot use native module, all other contexts must fall back and set python.NativeModules to 'false' to run native extensions in LLVM mode. This is recommended only for extensions included in the Python standard library. Running a 3rd party extension in LLVM mode requires a custom build of the extension and is generally discouraged due to compatibility reasons.
Of course I played and experimented a bit with the parameters in /etc/default/openhab
, but could not find a solution.