HABApp vs JSR223 Jython

Setup, Configuration and Use Scripts & Rules
,
Tags: #<Tag:0x00007f175a4156e0> #<Tag:0x00007f175a415528>
#1

Having recently migrated quite a bit of JSR223 Jython2 code to HABapp, I’d like to jotted down some notes on my experience, and hopefully create a PR to put this in the HABApp doc. @Spaceman_Spiff recently added some notes on what HABApp is. I’d like to focus on the pros and cons.

First thing to note is that both these frameworks are based on Python (albeit different versions) and both target superusers (i.e. software developers), so in that sense there are more in common than differences. Both also have maintainer(s) that are very active on the forum.

Pros of HABApp:

  • Based on Python 3, and as such allows full usage of Python modules via pip.
    In addition, it is easier to leverage a proper IDE such as PyCharm for development. Having previously coded JSR223 code in vim on the Rasperry PI, moving to PyCharm was a big upgrade. The IDE catches many errors that I wouldn’t detect until the code path is executed at run time.
    PyCharm also allows running unit tests directly within the IDE without the need to have HABApp running (more on unit testing below).
    JSR223 code will remain at Python 2.7 version as there is no development effort to bring it up to Python 3.
    With JSR223, it is also possible to install additional 2.7 modules via pip but it is more difficult as you have to resolve the dependencies yourselves. See Michael’s comment below.

  • Much better turn-around time:
    This is achieved via multiple factors:

    1. The use of CPython and Python3.
    2. The HABApp process is much lighter weight than OpenHab with all its bindings, and scripting environments.
    3. The easier use of IDE such as PyCharm.

    In JSR223, if we make a change deep in a referenced module, sometimes the only way to ensure that OH loads the change correctly is to restart OH, and that takes very long time. With HABApp, you can just restart it in 2 or 3 seconds.

  • Provide additional deployment flexibility:
    HABApp interacts with OH via its REST interface. The HABApp process can be run as a separate process on the same machine that hosts OH, or it could be deployed on a totally different machine and / or different network.
    The usage of REST interface has drawback and I will touch on them in the cons section.

  • More resilient toward changes in the core OH:
    As the rules and modules are now in a different framework, upgrading OH will be less painful and less fragile. OH will mostly be used for the item definitions and the bindings. The chance of OH and HABapp change majorly at the same time is low.
    This strength also comes with a cons.

  • Flexibility in the helper libraries:
    When interacting with external devices or services, HABApp code has the choice of leverage the OH bindings or the devices/services’ Python API directly (when available).

  • Proper error messages.
    If one of your rule throws an exception in HABApp there is enough information in the log to pinpoint the issue.

  • The core framework is flexible enough to allow decoupling from OH when necessary.
    HABapp has a set of abstractions that map to various OH types. These can be remote and link to OH items or they can be local items that are visible within HABapp only. In the majority of cases, we work with remotely linked items, but local items are very useful in certain scenarios such as data sharing between rules and unit testings. JSR223 allows data sharing via injection into the runtime context. But a unified data types in HABapp provides much better emulation of the runtime in the context of unit testing. For example, you can write an unit test to simulate certain condition in the home such a motion sensor triggered and verify that the rule behaves correctly. And the unit tests can be run independently without the need of OH and HABapp processes.

Now here are the cons:

  • As an independent process, HABapp adds more complexity to the pipeline. We already have OpenHab, mqtt, and many other devices/sensors and services.
  • While leveraging the official OH REST API, in a sense HABApp still makes the OH eco system more fragmented dues to itself being another development choice.
  • The REST API doesn’t expose everything. Notably, actions are not accessible via REST API. This can be worked around via usage of bridging items and complement JSR223 rules, but they are not out of the box solutions nonetheless.
  • The REST API has additional latency. In the majority of cases, it doesn’t matter at all. One area that is somewhat noticeable is light management; i.e. turning on the light when a motion sensor is triggered. I do notice slight additional latency, but it is still acceptable.
  • The REST API is hidden from the user and as such receives less attention than the JSR223 engine. Being the official Python scripting environment in OH, JSR223 also has a larger user base and it is easier to find previous information on the forum.

Given the pros and cons above, IMO, HABapp is a very good contender. I am migrating all my code over to HABapp. Surprisingly, the changes needed are very minimal. The largest change for me is the naming convention. I was mainly using Java naming convention, and PyCharm flaged a lot of warnings. As both HABapp and JSR223 continue to improve, here is my recommendation on how to approach rule writing:

  1. Push as much code into modules as possible. Besides encourage reusability, it also allows abstracting from the framework.
  2. Provide an abstract layer to avoid direct dependency on the framework. For example, for HABApp item, checking if a switch is on is done via item.getState() == OnOffType.ON; for JSR223, it is item.is_on(). To give your code more flexibility, you can have a core module checking the state for you such as def is_in_on_state(item): .... The same thing applies to the other areas such as the event dispatcher, the setting of item states and so on.

Here is my original JSR code: openhab-rules/legacy-jython-code at master · yfaway/openhab-rules · GitHub.

And here is the migrated code (on going): zone-apis/zone_apis/aaa_modules at master · yfaway/zone-apis · GitHub.

6 Likes
HABApp - Easy automation with openHAB
How to get started with new scripting?
#2

If openhab moves from java 11 to grallvm python3 and some other languages are supportet. maybe this move will come in the future.

#3

excellent job breaking down the differences, thank you for this :+1:

#4

I think this is a good comparison, and it’s good to have one to help people make a descision. That said, there are some things that you have wrong about Jython/JSR223 that I want to clear up. I don’t have similar information about HABApp simply because I did not use it long enough to learn these things. I’m not advocating for Jython here, I think everyone should look at the options and pick what they are most comfortable with, but I think the comparison should be fair.

You can install modules with pip for use in Jython. You need to specify the target directory and Python version as below. When using the --python-version option you must specify --no-deps, so you will have to manually install dependencies. And yes, I know this is not ideal, but it does allow you to install and use anything available from PyPi that is compatible.

pip install -t "{OH_CONF}/automation/lib/python" --python-version 2.7 --no-deps

You can use PyCharm (or Pylance) with JSR223 rules. I am actually working on stubs for the Helper Libraries that will vastly improve the user experience.

The JSR223 script engine is what is holding this back, it only supports Jython which is only at version 2.7.2. openHAB 3 has updated to Java 11 so we now have access to GraalVM scripting. I believe there was some effort to get Python 3 running on Graal but there are issues on the Graal end regarding that so until those issues are sorted we can’t actually advance on that front. (I don’t know a huge amount about Python on Graal, this is just what I know based on forum posts I’ve seen throughout 2020)

The fact that HABApp runs on CPython and that JSR223 runs on Jython actually doesn’t change anything for most of the user base.

This no longer holds true in OH3, it automatically reloads libraries when a script is refreshed. I have not tested whether it reloads only for the script file that got refreshed or for all scripts using the library, but it is still a big improvement.

You can do item.name in JSR223 Jython and ECMAScript 5, I have done it in both.

#5

It’ll still be a java implementation of python. While it might be better than jsr233 it’ll always be a compromise.

Whilst you can install modules, you can not install all modules from pip.
Once the module ships with c files or needs to compile stuff that’s when things stop working.
Also most packages don’t support python 2.7 any more because it has been eol for quite some time.

HABApp already works and you already can create great rules with it.
It’ll likely get better over time (or at least let’s hope so :wink: ) so once something changes on the openhab side we can do this comparison agian.

@yfaway:
Imho you forgot the most important part:
Proper error messages.
If one of your rule throws an exception in HABApp there is enough information in the log to pinpoint the issue.

Con might be that HABApp has a somewhat steep learning curve. It’s different and that’s why it provides great flexibility but you have to get used to it.
Also the docs could be better (but can’t they always :wink: )

1 Like
#6

I agree that errors can be confusing, but use of the log.log_traceback decorator for any code running outside a rule greatly improves the situation. That information may not be as prominent as it should be in the docs either.

#7

Thanks guys. I’ve updated the original post to clarify pip installation in Jython. Add a point on clearer error message in HABapp and another point that JSR223 has more attention than the REST API and has a bigger user community.

I am trying to be as fair as possible between the two sides.

Michael, it is definitely possible to use PyCharm with JSR223 code. This is the reason why the wording is easier in HABApp. The reason that is the case is because HABapp is pure python 3, there is no references to Java classes. In JSR223, my rules have to import a bunch of Java classes that contain things such as SwitchItem, StringItem and so on. PyCharm will likely highlight these import as error as it doesn’t know about them. Having an abstraction on top of these objects would be helpful (similar to how HABapp has its own items package containing SwitchItem, StringItem and so on).

I emphasize CPython because it is one of the reason why HABApp starts faster. Jython requires loading the whole JVM. Obviously OH loads a so many things besides Jython so the comparison should really just focus on the scripting comparison.

I haven’t tried OH3 yet, but I am skeptical that OH3 will just reload every module. That will have a huge unexpected effects on scripts. Modules might contain states, and the system might have many rules. Imagine changing one rule, and somehow OH messes up the states of other rules. We typically do controlled reload via import.reload(...). But reloading everything has to be a deliberate act, and typically that is a restart of the process.

1 Like
#8

Absolutely! I recently started using Pylance, but I didn’t even know about PyCharm or MyPy prior to that. Now that I’m used to all that sugar in my pure Python work I want it for my rules work, so I am working on stubs (pyi files) for openhab, the helper libs, and the Java classes that we need. This will bring the IDE experience to about the same level as using a C library in Python. There will still be no way to do testing outside of OH though, there is no way around that.

Hmm, I had not thought about that, I still don’t have much of my automation in OH3 yet. I will need to test this because you are correct, the implications could cause many strange behaviours in other scripts using the libraries that just got reloaded.

#9

Lots of people are working on GraalVM support. This support is important not just to get Python3. Nashorn, the JavaScript that is currently supported is end of life and won’t be available in the next LTS version of Java that OH will eventually have to move to. It’s also a really old version of ECMAScript and Nashorn has a relatively poor way of managing import of libraries. I’m about a month or so behind on watching the GitHub issues and PRs so I don’t know what, if any progress was made on this front.

The more common approach I’ve used and seen used is something like

if items["MyItem"] == ON:

All of the openHAB state and command ENUMS get imported for you and are available to use without messing with the parent class. And most of the time, unless you are messing with Items pulled from a Group’s membership, you’ve got the Item’s name (e.g. event.itemName).

It is possible to tell PyCharm to ignore those sports of linting errors.

My understanding, which is often wrong on these sorts of things, is that nothing quite that fundamental to how they work has changed for OH 3. In other words, it probably works the same as it did in OH 2.5.

Again, I may be showing my ignorance, but doesn’t it require an explicit reload in the script to cause the module to be reloaded for that script? That was my experience at least. Without that any changes made to modules would not be picked up until OH was restarted. But my experience is getting old and all those parts of my brain are being filled with JavaScript UI Rules instead.

1 Like
#10

This is the behaviour in OH2, but I have been doing some testing with libraries in OH3. If I modify the library, then reload the script that imports it, my changes take effect immediately, no need for explicit reloads.

I have only been testing with a single script accessing a library so far though.

1 Like
#11

@yfaway Thanks for the great write up and comparison! So basically you list all the cons I also see with HABApp :grimacing: Mainly latency, limited feature support by the REST API and another external building block to take care of.
On the other hand it looks conceptually a lot cleaner than the somewhat clumsy and confusing JSR223 (this already starts with the naming, what the heck should I know is JSR223??)

One question: Maybe I’m a bit slow on the uptake… what do you mean by “actions” here?

  • The REST API doesn’t expose everything. Notably, actions are not accessible via REST API. This can be worked around via usage of bridging items and complement JSR223 rules, but they are not out of the box solutions nonetheless.
#12

The naming is moving towards “Script Engine”. The name JSR223 is from “Java Specification Request”, nothing to do with scripting in openHAB really. I agree that without context it has no meaning, but before OH3 it was an experimental feature still under development.

Actions are a way for you to trigger binding functions from rules. Things like sending MQTT or Telegram messages.

#13

What Michael said is right. There are default actions such as playing audio or stream. There are also actions that come with the bindings. Here’s the document on the work around: Advanced Usage — HABApp beta documentation.

#14

I wouldn’t put too much emphasis on that. On OH2 the typical round trip time (e.g. send a value to oh and back) is at about ~20ms. On OH3 there are some issues with the auth bundle but you can disable it.

#15

@rlkoshak et al:

I have tested whether the automatic reload works for libraries that are being used by multiple scripts: they are not reloaded automatically. As before, a forced reload will refresh the module that is in memory.

1 Like
#16

Hi - any chance you could update the link to your migrated code? Currently it’s broken. Thanks!

#17

Dan, if you are looking for the Helper Libraries I have started maintaining my fork of them in Scott’s absence. You can find details in this post:

#18

Dan, the HABapp-based code is at GitHub - yfaway/zone-apis: APIs to group Home Automation devices / sensors into zones and associate them with actions. Keep in mind that mine is more like a framework itself riding on top of the supporting libraries (Jython before and HABapp now). I found that I had to make only a few minor changes to adapt to HABapp. The PlatformEncapsulation module provides the abstraction. The idea is that if I need to move to a different framework later, I just need to change the code there. Right now it only really supports HABapp, but it could support Jython in the future (the main issue is that my code makes use of Python3 programming language features so it is highly unlikely, unless we move to GraalVM).

Here’s my legacy Jython code for comparison.

1 Like
#19

thanks!