Python (jython) openhab-helper-libraries update

Setup, Configuration and Use Scripts & Rules
1

I am not sure how many folks still use jython and the openhab-helper-libraries and hence not sure it is worth the effort to update/maintain for the community vs. just keeping my own version updated?

There are also a couple of repositories (crazy-ivan’s being the latest, but not maintained today). If anyone is interested, I have updated the libraries with the following changes:

  • Added/fixed the missing classes/decorators to support more recent triggers:
@when("System started")
@when("System reached start level 50")
@when("Time is midnight")
@when("Time is noon")
@when("Time is [Item]")
@when("Time is [Item] [timeOnly]")
  • I have also added an onlyif decorator which mirrors the different conditions which can be configured through the ui.
@onlyif("Item Weather_Overcast = ON")
@onlyif("Item alarmPanelDisplay is ON")
@onlyif("Item Weather_Overcast == ON")
@onlyif("Item Weather_Overcast eq ON")
@onlyif("Item alarmPanelDisplay equals 'hello friend'")
@onlyif("Item Weather_Overcast != ON")
@onlyif("Item Weather_Overcast is not ON")
@onlyif("Item Weather_Overcast < 5")
@onlyif("Item Weather_Overcast lt 22")
@onlyif("Item Weather_Overcast is less than 22")
@onlyif("Item Weather_Overcast <= 5")
@onlyif("Item Weather_Overcast lte 22")
@onlyif("Item Weather_Overcast is less than or equal 22")
@onlyif("Item Weather_Overcast > 5")
@onlyif("Item Weather_Overcast gt 22")
@onlyif("Item Weather_Overcast is greater than 22")
@onlyif("Item Weather_Overcast >= 5")
@onlyif("Item Weather_Overcast gte 22")
@onlyif("Item Weather_Overcast is greater than or equal 22")

@onlyif("Today is a holiday")
@onlyif("It's not a holiday")
@onlyif("tomorrow is not a holiday")
@onlyif("today plus 1 is weekend")
@onlyif("today minus 1 is weekday")
@onlyif("today plus 3 is a weekend")
@onlyif("today offset -3 is a weekend")
@onlyif("today minus 3 is not a holiday")
@onlyif("yesterday was in custom_dayset")

@onlyif("Time 9:00 to 14:00")
@onlyif("Time 3:30 AM to 2:00 PM")
@onlyif("Time 6:00-13:00")

I did re-factor/simplify the parsing in the decorators - so let me know if there are any issues with any of your current rules.

The code is here:
https://github.com/jsjames/openhab-helper-libraries.git

Lastly, it would be great to have the python helper libraries kept in a central openhab git repository. Not sure what would be the process to make that happen??

1 Like
2

A new developer has started work to maintain the Jython add-on. I’ve not followed the effort but I’m sure you can find their work on GitHub. My understanding is they are planning on replacing the Helper Library to bring it more inline with the jRuby and JS Scripting helper libraries and some architectural differences.

The overal intent is to make the helper libraries a part of the OH repo and automatically included when you install the add-on.

3

See Jython is not deprecated - #47 by holger_hees - maybe you can work with @holger_hees on this.

1 Like
4

I am familiar with that thread and appreciate the work that @holger_hees has put into the binding (and hope to see a Graal python version at some point :slight_smile: ). He made it clear though that he was not maintaining the helper-libraries and has developed his own version. The questions I have:

  • Is there enough interest still out there for the python libraries where it would be worth investing the time to support/maintain/…?

Assuming there is interest —

  • there is a git repository for openhab-js (openHAB JavaScript Library for JavaScript Scripting Automation), does it make sense to create one for the python libraries? Having individuals with their own forks/etc is messy and its unclear where to grab the latest source.

Incorporating a version in the jython binding could also be done - I presume that the openhab-js libraries are done on that manner?

5

The “official” repo for the helper library needs to be tightly coupled with the add-on development. It becomes the official helper library. I don’t think it makes sense to have two independent maintainers for the add-on and the helper library. At a minimum it doesn’t make sense to have two separate and independent efforts. The add-on development and helper library needs to be in lock step.

For JS and jRuby the same devs are maintaining both.

6

@jsjames currently I’m working on 2 things in parallel.

First one is the jython binding, where I follow the development process of jython itself and keep the binding up to date with the latest jython releases. Additionally I observe the progress of the python3 implementation.

My own python scripts are based on this binding. I use my own python abstraction layer to make my live easier. I would recommend this for technical experienced users, because it is very close to the openhab java API, with the advantage that you can do absolutely anything.

For long term, I work on a python binding which should base on graalpy. I don’t know when this will be ready, because I currently have problems that I don’t really understand. Help is always welcome here. The current state is, that from the source code side I converted the jython binding completely to a version which uses the graalvm polyglot api and compiles absolutely fine with dependencies to “org.graalvm.python”. But if I start the openhab demo system I get exceptions that the binding don’t find the polyglot based python language implementation. I’m already active on the graalpy slack channel and get some hint some graalvm developers. My guess is that this is related to the maven package process.

2 Likes
7

@holger_hees - Thanks again for resurrecting the jython binding!! I’m also using this binding.

The question i have is there a need for a standard abstraction layer or helper-library? I have looked at yours and since it seems heavily leveraged/streamlined from the original libraries - do we make that the official one? I would recommend adding back in the decorator support as it simplifies rule instantiation and a lot of peoples existing scripts (mine at least) heavily use those. I’m happy to help here.

Regarding graalpy, I looked at this a while back as well and it seemed quite involved. I’m not an expert, but would be happy to put some time in here as well trying to get it working. If you can post your WIP someplace, I can give it a try.

8

Hi @jsjames

I committed my code to the branch pythonscripting. Feel free to compile and look into it.

Everything compiles fine. But it is just a prototype. The goal should be, to get it running in the demo app without any exception during initial loading. Currently it looks like the polyglot implementation is not part of the jar file of the binding. But if you look into it, it is.

There is also a slack channel of graalvm devs. There is a special graalpy channel where I already started a talk about openhab and my binding. It seams to be a maven package problem.

regarding the helper libraries. They is no need for it. You can do similar thing by your own. But you have to work with the core openhab api which can change with every release. Currently I adapt this changes for every release as part of my own upgrade process. But don’t worry it is not much. Most openhab upgrades went very smooth without any changes. But from time to time it will happen.

I will clean it a bit during the christmas holidays and add decorator support as well. But I will not name it official, because it is planned to heavily refactor it soon and add it as core part of the jython binding. But if you like you can collaborate/help here too.

9

I will add helpful answers from graalpy devs here too.

the answer related to my question are

Holger Hees Dienstag um 13:45 Uhr

Hello. I try to get graalpy running in my application. Currently I’m using jdk21 and this is my pom configuration

This is the problematic code

import org.graalvm.python.embedding.utils.GraalPyResources; public final class GraalPythonScriptEngineFactory { private final Engine polyglotEngine = GraalPyResources.contextBuilder(PYTHON_DEFAULT_PATH).allowAllAccess(true) .allowHostAccess(HostAccess.ALL).build().getEngine();

the maven build creates a jar file which is used in another applicationI already checked that the jar file includes org.graalvm.* [collections,home,nativeimage,options,polyglot,python,word] and org.graalvm.python.vfs.* etcif I start the other application which includes my jar I get the following error

Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.IllegalStateException: No language and polyglot implementation was found on the module-path. Make sure at last one language is added to the module-path. [in thread "main"] at org.graalvm.polyglot.Engine$PolyglotInvalid.noPolyglotImplementationFound(Engine.java:1801) ~[?:?] at org.graalvm.polyglot.Engine$PolyglotInvalid.createHostAccess(Engine.java:1792) ~[?:?] at org.graalvm.polyglot.Engine$PolyglotInvalid.createHostAccess(Engine.java:1754) ~[?:?] at org.graalvm.polyglot.Engine$Builder.build(Engine.java:741) ~[?:?] at org.graalvm.polyglot.Context$Builder.build(Context.java:1925) ~[?:?] at org.openhab.automation.pythonscripting.internal.graal.GraalPythonScriptEngineFactory.<init>(GraalPythonScriptEngineFactory.java:69) ~[?:?] at org.openhab.automation.pythonscripting.internal.PythonScriptEngineFactory.<clinit>(PythonScriptEngineFactory.java:57) ~[?:?]

my understanding was that the language and polyglot implementation is embedded into my jar file with the maven plugin "graalpy-maven-plugin"thanks for your help (bearbeitet)

8 Antworten

Tim Felgentreff Dienstag um 14:04 Uhr

Is this a Graal JDK? Then you’ll need to use the latest JDK 23. If this is a fat jar, you’ll need some extra config, @Christian Humer can point you to it

Holger Hee Dienstag um 14:09 Uhr

no, it is the normal oracle jdk. I understood that all dependencies come in via the maven dependencies.

The fat jar already includes lib-graalpython stuff like libposix.so, libpython-native.so etc… and lib-python with a lot of python files. My understanding was that this is the embedded python runtime.

Christian Humer Dienstag um 15:20 Uhr

I’d recommend against using single fat jars. But if need to we have config for the shade plugin here:
polyglot-embedding-demo/pom.xml at main · graalvm/polyglot-embedding-demo · GitHub for the Maven Assembly plugin here:
polyglot-embedding-demo/pom.xml at main · graalvm/polyglot-embedding-demo · GitHub

Holger Hees Dienstag um 15:58 Uhr

It must be a single fat jar, because it it some kind of plugin for a smarthome system (openhab.org)The goal is to just add/install the jar to enable python support as a scripting languageThere are already jars/plugins for javascript, groovy and ruby and a obsolete jar for jython supportMy goal is to replace the jython support with graalpythonAnd thanks for the hint, I will look into theese maven plugins.

Tim Felgentreff Dienstag um 17:02 Uhr

Cool about openhab, would be awesome to get python 3 support with graalpy in there

Christian Humer Dienstag um 17:52 Uhr

And thanks for the hint, I will look into these maven plugins

You are welcome. These are the configurations we test with. If you run into some trouble with other tools, let us know. Unfortunately lots of tooling in that space just discards essential files from our jars by default. (hence my recommendation against using it) (bearbeitet)

Stepan Sindelar Dienstag um 20:21 Uhr

Once you get past that issue, then:

private final Engine polyglotEngine = GraalPyResources.contextBuilder(PYTHON_DEFAULT_PATH).allowAllAccess(true) .allowHostAccess(HostAccess.ALL).build().getEngine();

also looks suspicious.

  1. Normally you’d build Engine using Engine.newBuilder()...build() and then you pass that Engine to the contexts you create: GraalPyResources.contextBuilder(...).engine(polyglotEngine)...build().
  2. What is PYTHON_DEFAULT_PATH? If it is a path on a filesystem, then you first need to extract the GraalPy resources to that directory using GraalPyResources#extractVirtualFileSystemResources, if you want GraalPy to use the resources directly from the jar where possible, then you’d use parameter-less GraalPyResources#contextBuilder or overload taking VirtualFileSystem, which you can create using VirtualFileSystem.newBuidler()...
10

@holger_hees - I was able to get the graal engine running and submitted a pull request to your repository. Let me know if it is successful for you as well.

3 Likes
11

Thanks a lot. This works for me too. I think this is a huge step forward in providing a working binding.

Currently it is initialized in the same way as the jython binding. Means in theory it should already evaluate all python scripts located in ./runtime/conf/automation/python/, but for some reason it does not.

12

In case you have trouble with GraalPython, I would recommend to take a look on the JS Scripting add-on that is based on GraalJS. The Graal stuff is mostly the same I guess.

2 Likes
13

I believe the GraalPyResources forces the directory structure to be in a src, home and venv under the root directory. Did you try to put any files in ./runtime/conf/automation/python/src?

In looking at the GraalPyResources code, it might be better to duplicate this in the binding so we can have more control over how this looks?

14

After looking at this some more, the sample script engine factory did not return the “py” extension from the getExtension call (I submitted this as a PR). Also, with the default directory structure, it does look in ROOT_DIR/src. It now sees files in that directory, however, I think more work with need to be done on the sample engine to get things to work.