Jython is not deprecated

holger_hees · February 27, 2024, 9:41am

Hello,

over the last weeks I followed the jython development a bit.

The information that the development of jython stopped is wrong. There is a very active community talking about development of jython. The only thing which is true is that the future of jython 3 is delayed. It will come at some point, but no one can say how long it will take.

For jython 2, they are actively working on it. Fixing bugs and implementing new things to support upcoming java versions. So the argument that it will maybe not work with new java version is not true, because I saw a lot of talks about exactly this topic.

I really want to vote to remove the deprecated flag from the jython binding, I mean it is just one small class which we have to support and maintain. Also the argument that the helper libraries are not supported there does not matter for me. Just announce it as it is, without helper libraries. For me, as an example I use it without the helper libraries.

and just to avoid the discussion of habapp, This is not an option, because it does not support all the nice openhab features. e.g. actions

also javascript is not an option, because it it partially 3-5 times slower, which can have a huge impact if a role in jython already takes 200ms.

maybe I can influence the decision about the deprecated flag.

JimT · February 27, 2024, 10:37am

I was the one adding the deprecated notice.

I’d be happy to take them out and I’m glad to hear this news. Perhaps someone would jump in and either fork the existing helper library, if the licence permits, or start developing a new one.

@rlkoshak wdyt?

I also noticed the latest version was 2.7.3 from 2022, but the one in openhab addons is 2.7.2.

@holger_hees would you be able to test 2.7.3? If need be, I can provide you with a jar, or you can build it yourself.

JimT · February 27, 2024, 11:04am

Here’s a jar compiled against jython 2.7.3 for openhab 4.2-snapshot
https://github.com/jimtng/openhab-addons/releases/download/jython-2.7.3-0.0.1/org.openhab.automation.jythonscripting-4.2.0-SNAPSHOT.jar

holger_hees · February 27, 2024, 11:50am

here is a statement from one of the jython devs some minutes ago as an answer to my comment.

@JimT do you have a jython 2.7.3 version for openhab 4.1. I’m currently running this version and I like to test it in my production setup with all of my 10.000 lines of python code.

JimT · February 27, 2024, 12:03pm

https://github.com/jimtng/openhab-addons/releases/download/jython-2.7.3-openhab-4.1-0.0.1/org.openhab.automation.jythonscripting-4.1.2-SNAPSHOT.jar

holger_hees · February 27, 2024, 12:14pm

Drop in replacement works like a charm. All my rules are loaded and parsed successful and running for now without any noticeable problem

JimT · February 27, 2024, 12:55pm

rlkoshak · February 27, 2024, 3:24pm

But there is no development taking place here. The Jython Helper Library was abandoned. A fork was made but it’s not seen a change in a couple years and already has some things broken in OH 4. No one is maintaining the Jython add-on.

That’s not the argument. The argument is is may no longer work with openHAB.

If you are volunteering or know someone who is willing to volunteer to take on that maintenance of the OH add-on and the Helper Library I’m totally on board. If not then we still have the situation that something may break and no one is going to fix it.

As with the jRuby helper library, I’d really prefer that the Helper Library were a part of the OH project and were a part of the Jython add-on before I’d do that. I don’t remember if the license is compatible with the Eclipse license or not.

IIRC there were some problems with that version and the add-on. Maybe those problems have been fixed.

holger_hees · February 27, 2024, 3:45pm

The jython library is still developed. In the meantime they release 2.7.3 and for 2.7.4 there is a beta version.

Currently I’m testing version 2.7.3 and it looks good so far.

I would also like to help fixing the binding if there are some problems. But what does it mean. If someone reports a bug. I can compile it and try to find out the reason. So before you say it breaks openhab, I would like to help. But for now it works. I can not say that it works in the future, but in the discussions I saw that the jython developers spend a lot of work in fixing bugs related to new java versions. Their goal is that it runs with every LTS version. So I’m optimistic.

holger_hees · February 27, 2024, 3:48pm

the only bug I found so far is that urllib2.urlopen does not work, but it can be replaced by the openhab http action. And with version 2.7.4 it will be fixed.

github.com/jython/jython

jython-standalone 2.7.3 urllib2 send request with request body occurs memoryview error

opened 02:12AM - 15 Nov 23 UTC

closed 07:01AM - 01 Feb 24 UTC

hung001

in jython-standalone, i use urllib2 to solve sending request things , and 2.7.2 …works fine,but after i upgrade it to 2.7.3,it occurs error : TypeError: cannot make memory view because object does not have the buffer interface using "urllib2.urlopen" to send request,the problem repears on local when the reqeuest send with request body, and it works when api don't need request body. so i think the request body may be the key of this problem,please check is it a bug or something?

rlkoshak · February 27, 2024, 4:09pm

My position is simple and relatively clear.

Unless and until someone takes on the work to maintain the Jython add-on and the Helper Library as an official maintainer, the add-on is deprecated. If we cannot count on anyone keeping both the add-on and the library up to date with changes in OH, at best it’s going to be left behind (it kind of already is) and at worse it’s going to break in OH and on one is going to fix it.

Keeping it up is a lot more than just ensuring it runs on the version of Java OH is using. The OH API itself is constantly changing with new features and breaking changes. Most of the time the maintainers can deal with it but as soon as something breaks that requires detailed knowledge of how Jython works and interacts with OH, no one is going to be able or willing to fix it.

If you want to save the Jython add-on I recommend becoming or finding someone willing to:

Take on maintenance of the add-on. Go find some changes since OH 3 (that’s basically the last time the add-on had a maintainer) that need to be made in the add-on the bring it up to date. Or find something that the add-on could do better with some of the new changes to the OH API. There have been no significant changes to the Jython add-on since the release of OH 3.0 four years ago. A really good start here would be to make it so the add-on comes with the Helper Library instead of requiring users to clone a Github repo in order to have a reasonable rules coding environment.
Take on maintenance of the Jython Helper Library. At a minimum fixing the directory watcher error and implementing the Time is <Item> rule trigger would be a good start. Starting the work to move the library into the openHAB Project itself would be even better so if it gets abandoned again, it will be possible for someone else from the organization to approve PRs and such until a new maintainer can be found instead of forcing yet another fork confusing everyone as to which fork is the right one. This will be a prerequisite to making the Helper Library a part of the add-on.

The same person does not need to do both 1 and 2.

holger_hees · February 27, 2024, 4:20pm

Why not handling it like groovy, it is for advanced users.

So there is no need to implement the helper libraries at all.

Just extend the AbstractScriptEngineFactory like now and make all openhab features available. Not less and not more. Currently I use it in this way since openhab version 2.

rlkoshak · February 27, 2024, 4:32pm

Sure that’s a possibility. Though that will mean that pretty much all of the examples users will find on the forum will not be valid. It’s a pretty miserable experience to work with the raw APIs though, it definitely wouldn’t be a good choice for even most current users of Jython.

There is still the issue that there’s no one maintaining the add-on though. If you are volunteering you’ll need to start making some updates to the add-on (small ones at first for sure) to develop a bit of a reputation so you can become the Jython add-on expert (see some of the things I listed above).

Or find someone to volunteer to do it.

Note that my understanding is that Groovy doesn’t support external Groovy libraries so their only choice is to use the raw APIs.

holger_hees · February 27, 2024, 4:43pm

that makes sense, My last time I developed something for openhab was with openhab version 1. There I was the original author of the zehnder comfo air binding.

If you give me some time and keep the jython addon as long as possible in openhab, I will jump into this topic. Also regarding the helper libraries.

It should not be a big deal with the helper libs. During openhab version 2 I forked the helper libs, because they was not fulfill my needs. Since this time I made all necessary changes to the libs, needed to make it work with the latest openhab version. Maybe it would make sense to start from scratch again to match the syntax with the javascript helper libs a bit closer. But it will take time, because at the same time I have to change the way how I’m currently using them. Not much, but a bit.

And regarding the binding, I could try to simplify it a bit as my first step.

As a time horizon for everything, I could say by the end of this year, beginning of next year.

holger_hees · February 27, 2024, 5:02pm

And this is the way how my rules look like, using my own helper libs

you can enable profiling by an additional annotation, it measures the performance and it shows you always source events. The decorator for that you can find here.

github.com

HolgerHees/openhab_shared/blob/f185826217663021900f23ce3d4e23621836f8bf/python/shared/helper.py#L57


      
          scriptExtension.importPreset("RuleSupport")
          automationManager = scope.get("automationManager")
          lifecycleTracker = scope.get("lifecycleTracker")
          
          scriptExtension.importPreset("RuleSimple")
          SimpleRule = scope.get("SimpleRule")
          
          def scriptUnloaded():
              log.info("unload")
          
          class rule(object):
              def __init__(self, name = None,profile=None):
                  self.name = name
                  self.profile = profile
          
              def __call__(self, clazz):
                  proxy = self
          
                  filePackage = proxy.getFilePackage(proxy.name) if proxy.name is not None else None
                  classPackage = proxy.getClassPackage(clazz.__name__)

rlkoshak · February 27, 2024, 5:08pm

No one is looking to actually remove the Jython support. The deprecation warning is there mainly to warn new users that it may be removed at some point. But no one will remove it until it actually breaks and we can’t fix it.

So you have all the time you need.

I think that would be my preference. There has been a lot of good work in OH and the other languages over the past four years that I suspect would cause different choices to be made in designing the Helper Library if one were to start from scratch today.

The developers of the original Jython helper library preferred to keep the raw Java Objects and Classes where ever possible to keep things as close to Rules DSL (and the other languages at the time) as possiuble. In jRuby and JS Scripting they instead go out of their way to make sure that you are dealing with native jRuby/JS classes and objects (with access to the raw Java in those rare cases where you need it) where ever possible.

Having done it both ways (Jython and Nashorn for raw Java and JS Scripting for native classes and objects) I think there is a lot of advantage to the latter approach. It frees the end users to avoid that whole “is this a Java String or a Python String?” question and no matter what you are doing it will be done using the native language’s idiomatic way instead of needing to jump to Java way for this and back to a native way for that. Working with third party libraries should be easier too.

I don’t know that the syntax should need to match the JS syntax (though JS’s rule builder is kind of cool). I think it’s more important to present something that looks and feels like native Python. Take inspiration from there, jRuby, and even Rules DSL where appropriate but I’d say to follow Pythonic style as much as possible.

Also, keep the poor UI users in mind. There are some things which make good programming sense when you only use file based rules which become a huge burden when writing Script Actions and Script Conditions in the UI. For example, in JS Scripting the helper library gets automatically inserted into the script (with an option to disable that) because otherwise each and every little script you write would need to explicitly import the library. Given that, 20+% of all the code written through the UI would be import statements. That’s a pretty big burden on those users.

JS Scripting was the first to adopt to my pleas to help support these users well which is why I use it as my primary language today. jRuby has come along since then and of course Rules DSL is pretty good in the UI.

holger_hees · March 8, 2024, 9:48am

Hello @rlkoshak and @JimT

just made my first steps again with openhab. I want to share a commit to hear your opinion about.

I refactored the Jython binding a bit, with the following objectives

First I was inspired a bit from the groovy binding, because it looks simpler and easier to understand
I change the way how script types a detected to a stream api based variant like in groovy
Also the way how to PyScriptEngineFactory is instantiated is changed (No more looping over all script engines)
I simplified and reorganized the way of python path handling a bit, to make it easier to read
I changed the default python path from “/automation/lib/python/” to just “/automation/python/” to match better with the behavior of other engines like groovy (/automation/groovy/) and and ruby (/automation/ruby/)
@JimT your recommendation to make the default path, configurable… I would not recommend it, because the default path is only an addition to the python path. Means you can already add every path you want.
I removed an additional deprecated info

Before I will continue making a pull request, I will test it much more, just to avoid side effects.

I just want to hear if this are to many changes for one pull request, if I miss specific coding styles you expect, maybe you disagree that this is not easier to read or anything else. Just to remind, my last steps with openhab coding are ages ago

Additionally, my goal is also to upgrade to jython 2.7.4 as soon as it is released and tested on my side. I heard, that it will be released in Q2 2024. Hopefully not to late for the openhab summer release.

rlkoshak · March 8, 2024, 3:00pm

These are best going to be answered by the maintainers. Though overall smaller and simpler PRs are much easier to review than large ones. They have a greater chance of being quickly reviewed and approved compared to large and complex PRs.

For help in testing (and to give the maintainers a good feel that what has changed works) would be to create a posting on the marketplace where users can install and use the add-on and give feedback.

You can submit the PR with “[WIP]” in the title to give them an early view of what’s changed as well.

I personally don’t have too much to offer in terms of how it’s implemented. I would caution that because Groovy doesn’t support libraries like the other languages do some of that simplicity might be because it has less to deal with overall. I do like using “/automation/python” though wonder if “/automation/jython” might avoid trouble down the road in case someone decided to implement a GraalVM Python add-on.

holger_hees · March 9, 2024, 10:17am

thanks for the feedback. I added a pull request with [WIP]

mjcumming · March 9, 2024, 8:22pm

Would it make more sense to use GraalVM Python Runtime ?