What's the deal with Python rules in OpenHab 3?

I’m running OH3 and have installed the Python rules engine, so I can write rules in Python straight into the OH3 frontend.

What I imagined I was getting was a working method for writing rules in Python and a learning curve in how to interact with items.

What I’m finding is, from what I’m hearing, an incomplete new rule engine that might work one day, but right now can’t be used without hacking in helper libraries from an old java-python interface that was left out of OH3. And presumably, once Python rules are ready to go, will require rewriting of rules.

I’ve asked for help getting my first python rule running, and got a really prompt response that absolutely helped, but now I try and take what I learned and research further, the only nearly useful link that comes up in Google is my initial question and the thread that came of it.

The thing I’m trying to do right now is compare a datetime item state to now(). Is now() before that time, or after?

The closest I got was now() reflecting my timezone and the item state in UTC+1100 (i.e: 11pm looks like 11pm from now() and like 12pm"+1100" from the item) (I might have them the wrong way around, I’m away from my computer at the moment)

So whatever I do to get around this problem reports back that libraries don’t exist, that functions don’t exist, etc.

I’m also finding there is no definitive answer anywhere for what I need to do to make things work. I’m finding plenty of answers for things that will work in other environments, but nothing translates back into OH3 rules.

It might be me. But I’m usually better than this. I’ve tried, but there’s only so many rabbit holes I can disappear down that never get me any closer to finding out if now() is before the time in that item.

Does anyone have a working understanding that can help someone like me crash into this and make things happen?

Is there an official word on Python rules in OH3?

How do I work out what libraries I can include and what functions are available to me?

Is it true there is a major update in the future somewhere that will break anything I do now?

I’ll be happy to help contribute to documentation, because we clearly need more, but I need sources for the things I will assert. Can anyone help me get them?

2 Likes

If you want to write rules in Python (not Jython) you can always use HABApp.

I know there is currently some work done on the rules engine and the jython support but I can’t give an estimate.

2 Likes

I had a quick look at HABApp. It appears to be a 3rd party service you can install and use to write and execute Python code in, and that it uses the REST API to speak with OpenHAB.

I already have code written in python that talks through the REST API and is called from the Exe binding. Its the connection overheads for the API that I’m trying to avoid. The communication can take a few seconds to complete.

That’s why I’m trying to work out if I can use Python natively inside the OpenHAB environment, or if I should put my energy into a different scripting language.

The OH environment is Java which currently means using the Python 2 based Jython. HABapp is a separate service using the REST API just like the Main UI does. It also uses events from the API. In my testing I have not noticed any lag.

This is the benchmark done with OH2 (see rtt and rtt load for round trip times).
OH3 still has some performance issues but those will (hopefully) be fixed with the next release. I doubt that you will notice the delay even under high load.

+------------------------------------------------------------------------------+
|                                   openHAB                                    |
+------------------------------------------------------------------------------+

Bench item operations ... done!

            |  dur   | per sec | median |  min   |  max   |  mean 
create item | 2.386s | 125.726 | 7.00ms | 6.00ms | 42.0ms | 7.95ms
update item | 2.338s | 128.307 | 7.00ms | 6.00ms | 29.0ms | 7.79ms
delete item | 1.699s | 176.564 | 5.00ms | 5.00ms | 17.0ms | 5.66ms

Bench item state update .... done!

                      |  dur   | per sec | median |  min   |  max   |  mean 
             rtt idle | 4.996s | 149.111 | 7.00ms | 6.00ms | 13.0ms | 6.71ms
       async rtt idle | 4.999s | 129.418 | 8.00ms | 7.00ms | 15.0ms | 7.73ms
      rtt load (+10x) | 5.048s |  18.422 | 54.0ms | 43.0ms | 72.0ms | 54.3ms
async rtt load (+10x) | 5.074s |  12.613 | 79.0ms | 68.0ms | 90.0ms | 79.3ms

Cleanup ... complete

With the exec binding you have the overhead of starting a fresh python interpreter every time which is probably the dominant factor in your delay.

Am I correct in understanding Jython isn’t compatible with OH3 because of the move to Java 11?

I want to find out about the usefulness of the Python scripting language available in Rules in the OH3 UI.

Call me a pureist, but if I’m supporting dozens of these installations, I will want to keep everything as close to the core as possible. Extra services and other overheads are just extra potential points of failure for me.

1 Like

Thanks for the info and that makes sense :slight_smile:

My main concern is sticking as close to the core as possible. I already have mosquito running as a separate service, and a mysql server running on a different machine for persistence. Not to mention the call to python and the calls back in for item states and updates every five minutes to adjust my light bulb through mqtt for daylight.

Then if I’m managing installations for clients I’ll have the VPN server and any one of a million specialty programs for their situations. I’ll want everything to be as standardised as possible.

What happens, for example, if HABApp suddenly starts charging license fees and everything I ever wrote for one of my clients has an extra cost to run? Now multiply that risk, no matter how small, by every other non core extra I might have to use.

I do not know. Sorry.

1 Like

You could still use the last version under a non commercial license and keep stuff running.
As long as it works there is no need to change/update. It is not possible to backwards change a license.

2 Likes

And the person responding to you is the developer, so he knows :wink:

You can write Python rules in OH 3 today. However, to make doing so easier and more convenient there is a Helper Library that implements a number of annotations and library functions to make interacting with the openHAB Java classes more convenient.

The Rule engine is complete and works just fine.

For what ever reason, the Helper Libraries are not yet updated to work with OH 3. But when they are there should be an add-on that one can install to get those available. In the mean time one can manually install the library from GitHub by pulling the PR with the fixes for OH 3.

The answer to that question is going to be the same (or very nearly the same) in any of the rules languages.

Python:

import java.time.ZonedDateTime
var now = ZonedDateTime.now()
if now.isAfter(items["MyDateTimeItem"].getZonedDateTime()):

JavaScript:

var ZonedDateTime = Java.type("java.time.ZonedDateTime");
var now = ZonedDateTime.now();
if(now.isAfter(items["MyDateTimeItem"].getZonedDateTime())) {

Rule DSL

if(now.isAfter(MyDateTimeItem.state.getZonedDateTime)){

or if that doesn’t work

if(now.isAfter((MyDateTimeItem.state as DateTimeType).getZonedDateTime)){

For the most part, in Python, anything that is a part of Python 2.7 is available by default. Other libraries pip type libraries can be installed but it requires a special procedure. But most of the stuff you will be doing in rules will be interacting with openHAB. And that is generally going to be the same no matter what language you are using as you will be interacting with the openHAB Java Objects. It’s far easier to answer specific questions than to just dump the javadocs on you.

But in general:

Rules DSL: Pretty much everything you can use will be included by default. That’s why you don’t see an import for ZonedDateTime and you can just access now without calling the method on ZonedDateTime. But there are a lot of things that can’t be accessed (e.g. the Item Metadata Registry). Rules DSL also does not support libraries.

Python/JavaScript/Groovy: Except for the ItemRegistry (ir, Item states are available through the items dict), just about anything from openHAB you’ll need to use will need to be imported. That’s the sort of thing the Helper Library takes care of for you. Note that I’ve personally added an Issue to get more of this stuff added to the default context so we don’t have to import it.

Blockly: Not really complete at this time.

Probably not. At some point I suspect we will get GraalVM Python 3 support at which point you’ll have a choice between sticking with Jython (as long as it remains viable) and GraalVM Python 3. But it will likely at most require some minor edits (similar to what is required to rules to go from a major OH release anyway).

No, it’s completely compatible and there are a number of users using it. It’s the Helper Libraries on the main branch that are not compatible because OH 3 introduced a number of breaking changes (the move from Joda to ZonedDateTime, some core openHAB Actions moved, etc.). These are all fixed in a PR that has not been reviewed or accepted for some reason.

That sounds like a horribly over complicated configuration. Why this approach? Why not use the MQTT binding to talk to MQTT directly? Why involve Exec and Python at all?

tl;dr and conclusion: The Python add-on works just fine. The Helper Libraries are not yet available as an add-on but can be manually installed. The few minor changes required to make the Helper Libraries work with OH 3 are available in a PR. The Helper Libraries make interacting with openHAB easier but are not required (I’m not using them).

If your main concern is “staying close to the core” and reducing dependencies on extra stuff, I would recommend using JavaScript instead for rules. It comes by default (no add-on needs to be installed). However, at some point in the future this too will be updated from Nashorn JavaScript to GraalVM JavaScript at which point there will be some minor changes required to your existing rules mainly having to do with how libraries are loaded and used in the rules.

Your best source for “how do I do X in Python” will be looking at the Helper Library docs and the Helper Library code (if you don’t want to use the Helper Libraries themselves).

6 Likes

If you want sweet and short rules with little syntax have a look a the jRuby implementation, short sweet and fast

It does also require helper libs to be installed, but it interacts directly with openhab

1 Like

Don’t get me wrong. From what I saw, HABApp would be wonderful to have, if it was something I wanted to use. Unfortunately, in my conservation of potential points of failure, its a worst case scenario.

I appreciate your get up and go to see a hole in the community and fill it with such a great product. I wish there was more of that around. When you got to the choice of “should I put my resources into developing a standalone application that has a high risk of being superceded overnight one day?” you said “yes”, where I said “no” :wink:

This will be considerably quicker to read than it was for me to decipher and perform, so please enjoy…

SSH into openhabian machine as openhabian

Make a directory to work in.

openhabian@openhab:~$ mkdir -p tmp/py

then: (Download the OH3 version of the openhab helper libraries for OH3, extract them, isolate the ones noted as required on this page and save the whole automation directory and all its children to the conf share directory.

cd tmp/py
wget https://github.com/jimtng/openhab-helper-libraries/archive/oh3-patch.zip
unzip oh3-patch.zip 
mkdir -p exp/automation/lib/python
mkdir -p exp/automation/jsr223/python
cp -vr openhab-helper-libraries-oh3-patch/Core/automation/lib/python/core exp/automation/lib/python/
cp -vr openhab-helper-libraries-oh3-patch/Core/automation/jsr223/python/core exp/automation/jsr223/python/
cp -vr exp/automation /etc/openhab/

Then back in the OH3 UI, create one of the example scripts that came with the download

  • Go* to Rules
  • Create a new rule (+)
  • Fill in the following fields:
    • Name: Jython Hello World (cron decorators)
    • Description: This is an example cron triggered rule using decorators
    • Tags: Test tag & Hello World
    • When: (Add Trigger)
      • Time Event
      • on a schedule (cron)
      • Cron Expression: 0/10 * * * * ?
    • (and that reports back: Every 10 seconds )
  • Done
  • Save

Then in the Then section:

  • Add Action

  • Run Script: Python (2.7)

  • And on the Edit Script screen

    from core.rules import rule
    from core.triggers import when

    def hello_world_cron_decorators(event):
    hello_world_cron_decorators.log.info(“Hello World!”)

I then click Save, then Run Now and get this in the logs

2021-01-23 23:15:00.395 [ERROR] [e.automation.internal.RuleEngineImpl] - Failed to execute rule ‘a39174d329' with status 'RUNNING'
2021-01-23 23:15:10.400 [ERROR] [e.automation.internal.RuleEngineImpl] - Failed to execute rule ‘a39174d329' with status 'RUNNING'
2021-01-23 23:15:12.342 [WARN ] [jython.core.log                     ] - The 'configuration.py' file is missing from teh python.path!

I try:

import java.time.ZonedDateTime
var now = ZonedDateTime.now()
if now.isAfter(items["MyDateTimeItem"].getZonedDateTime()):

…as per your

The answer to that question is going to be the same (or very nearly the same) in any of the rules languages.

and get:

[internal.handler.ScriptActionHandler] - Script execution of rule with UID 'd7add55e50' failed: <eval>:1:0 Expected an operand but found import
import java.time.ZonedDateTime
^ in <eval> at line number 1 at column number 0

—edit
I later came back and realised I should have put in a real item. So I amended the script to:

from core.log import logging, LOG_PREFIX
import java.time.ZonedDateTime
log = logging.getLogger("org.openhab.core.model.script.MyLog")
\# Note: removing "var " from the front of this next line stops it from failing
now = ZonedDateTime.now()
if now.isAfter(items["LocalSunrise_StartTime"].getZonedDateTime()):
    log.info("after sunrise")
else:
    log.info("before sunrise")

…and got back

2021-01-24 02:23:27.487 [ERROR] [internal.handler.ScriptActionHandler] - Script execution of rule with UID 'tryJython' failed: SyntaxError: no viable alternative at input 'now' in <script> at line number 4 at column number 4

when I had that var in there, and then without it:

Script execution of rule with UID 'tryJython' failed: NameError: name 'ZonedDateTime' is not defined in <script> at line number 4

As you can see, I’m not shy. I’m not trying to get someone else to do all the work for me. I’m working with the information I have at hand and am absolutely stuck.

Your best source for “how do I do X in Python” will be looking at the Helper Library docs and the Helper Library code (if you don’t want to use the Helper Libraries themselves).

My best way of doing this is finding someone who can commit to an official source and I can go from there.

Weeks I have been pouring through discussions and I am still no closer to being able to run the most elementary scripts. I was doing fine with OH2.5 but when I upgraded because persitance just absolutely refused to work, and because I was assured it was officially upgraded, I’ve hit nothing but roadblocks in trying to write automation.

I’ll write up documentation myself if no one else has time, but seriously, I can’t find anything solid to teach.

It’s far easier to answer specific questions than to just dump the [javadocs](http://www.openhab.org/javadoc/latest/) on you.

Not at all. Have you heard the old “teach a man to fish” theory?

I’ve considered javascript, but havent even worked out how that would work yet. I might be stuck in the “but Javascript is client side” paradigm.

Except for the ItemRegistry (ir, Item states are available through the items dict), just about anything from openHAB you’ll need to use will need to be imported. That’s the sort of thing the Helper Library takes care of for you.

How about a short page with a crash course on library importing for Openhab 3? I can’t be the only one finding it difficult to just get things going.

When I try to apply the logic that its all Python and Java anyway so it shouldn’t be a problem that I’m in a particular environment, I find there’s enough issues coming up that Google has never ever heard of that its clearly not a straightforward implementation.

I come from a Javascript, CSS, PHP, SQL, Basic, VBasic, etc world where life s good because everything is documented. If I want to know how to do something, I describe it in my head and then find the way to describe it in my chosen language. References are plentiful and very verbose. Forums are full of people who will give you absolute answers to your questions, and who were baptised by fire in the coding forums where useless: why do you want to do that when… questions are flamed mercilessly.

I wish I had time to go down every dead end I find. I’ve been doing it a lot and I’m learning a lot about what doesn’t work, and often why, but I’ll be 60 in 11 years and I don’t have all the time in the world left to me.

I posted this topic last night, my time. I woke up today to your response and wanted to take my time going through everything before I got back to you. That then stretched out to working out what a PR is, wondering why you’re calling it a PR when I don’t think it actually is, wondering why it took until today before I found out that PRs existed, trying the suggested “PR”, searching to make sure the one I found that is actually called something entirely different is in fact what you’re suggesting (I still can’t be sure), documenting everything I did, addressing some of the off topic points that came up, spell checking, formatting, posting… and still I am no closer :frowning:

OpenHAB could be massive. I chose it because it showed the best potential for the future. But if we can’t make people understand how it works, it will only ever be a weird program that annoys hobbyists.

Hi Ben,

I am the one who made the PR (Pull Request) to the helper libraries. I am not the creator of the library. I am just another user of the library who faced an issue during my upgrade from openhab 2.5 to 3.0 and tinkered a bit with the helper libraries to make it work on openhab 3.0.

However, I am using the rule system completely through file-based system. I do not use the openhab 3’s GUI based rule creation, and I am unfamiliar with how the helper libraries would work within the GUI rule system. This is not to say that it’s not possible. I simply do not know because I am quite happy with my file-based rule system.

So if you want to use the file based rule system, perhaps I can help pointing you in the right direction on how to get your rules working, purely as a user-to-user assistance.

Also, whilst all my rules were written in Jython, and in RulesDSL prior to that, I am slowly migrating them to JRuby. Right now I have some rules in Jython as they were from openhab 2.5 and some migrated to JRuby. Prior to discovering the JRuby option, I have never programmed in Ruby. I found it so much nicer though. It’s like Perl and Java but a lot more fun and compact.

3 Likes

The instructions for installing the Helper Library include a a step to create automation/lib/python/configuration.py. You skipped that step. Which instructions for installation did you follow?

Also note, if using the Helper Libraries, and especially if you will be creating and maintaining configurations for lots of instances, why create the rules through the UI in the first place?

I may have shown the import incorrectly. I haven’t used Python for a few months. Maybe it needs to be

from java.time include ZonedDateTime

It’s nothing specific to OH. Look up how imports work in Python and do that.

Python support is what it is in OH right now.

So why change? Rules DSL and. rules files are still supported the same as they ever where. You don’t have to use any other language. You don’t have to use the UI. You can continue on as you always have.

For Python you’ve already done it.

from core.log import logging, LOG_PREFIX

Note that this syntax is defined by Python, not openHAB.

For JavaScript see OH 3 Examples: Writing and using JavaScript Libraries in MainUI created Rules

For Rules DSL you are out of luck.

The docs are the way they are because no one has volunteered to write them better. Id write some myself and it’s on my to-do list but I’ve even more people bitching about the docs for Pages than I do for Rules. And frankly, a lot of the problems are caused by people who have run with Rules DSL rules for years suddenly deciding they absolutely must rewrite everything in a language they are unfamiliar with in a way they are unfamiliar with despite the fact that all the knowledge and work they already have with. rules files are still valid and still work.

The Helper Libraries are not a part of the openHAB project itself. Unless and until it becomes so, it will have it’s own documentation. I can’t fix that.

Python and JavaScript and Groovy all have their own syntax and ways of doing things like importing libraries, iterating lists, etc. I can’t fix that either. One would presume, and perhaps it needs to be said, that the OH docs are not going to teach you how to program in theses languages. They will cover how to interact with OH 3 stuff but not fundamentals to the language.

2 Likes

Thanks Jim :slight_smile:

The link I posted above showing where I got my Jython libraries from, is that the pull request?

You’ve definitely piqued my interest in Ruby. I’ll be having a look :slight_smile:

@NetDirection I think you are trying to do something I find myself doing often enough: trying to run before you can walk. You are ripping through getting automation up and running in Python faster than I’ve seen anyone else in terms of all the different subjects you’ve touched on in this post, but you seem to be missing some details in your haste.

As Rich stated, you missed step #8 in the manual install instructions. I’ve lost track of how many people have missed that one.

To echo Rich, I don’t know how the libraries will work when using GUI rules either. They were written before it was possible and are largely focused on making rule creation in files easier. I’m sure most of the utility functions will work in the GUI, but it hasn’t really been tested yet.

Also it is going to take a little time to get your head around what it means to be running in Jython not CPython. The changes seem unnoticable at first, but as you evolve and get into more advanced things like interfacing with Java classes you will have to adapt.

Your import needs to be:

from java.time import ZonedDateTime

I’m working on stub files for openHAB and the Helper Libraries that will greatly improve the coding experience for Python scripts and libraries by providing type hints. They are lacking polish still and I am hoping to discuss with the maintainer of the Helper Libraries before publishing them, but if they are done before I hear from him I will likely publish them myself in the interim.

3 Likes

The page I linked to and the associated GitHub zip file

Can you please link to the instructions you are talking about?

I’m envisioning a situation where many people are running OpenHab in their homes and they pay someone like me to keep it ticking over for them.

In that scenario, they will be playing with the automation scripting themselves. They will need to be able to use the UI without having to call me every 5 minutes.

And if they are still calling me every 5 minutes, I will need to know the answers to their questions.

Let me stop you right there. I’ve learned over the past week that unless someone is talking about using Python in OH3 UI in January 2021, and has successfully done what they are saying under those exact conditions, I’m going to find that no matter how helpful they’re being, it is utterly useless to me at this point.

I’ll try your suggestion when I’m back in front of my computer, but don’t be surprised if you feel the slight hint of a swear word being mentally transmitted around the world to you a bit later.

You get how that communicates exactly nothing, right?

For those who missed it the first time, I’ll repeat it. I upgraded because a major release implies fitness for purpose. What I found was that the change in Java version broke what I had learned in 2.5 and I’m having so many problems fixing it that I’m stuck at the point of searching for a scripting language that will work.

I’ve imparted no knowledge. I have essentially repeated, like a parrot, something I saw somewhere else.

This is not a crash course.

Delegate. I’ve volunteered to help. Select a group of helpers and guide them in creating this content for you.

So a mass of people have spontaneously and individually decided to do the same thing in opposition to what you expect to be normal and reasonable? Is it possible you’ve made assumptions that don’t bare out in the real world, and these people are just flowing with where the tide takes them?

My keyboard also isn’t covered under the OpenHab project, but I need to understand how it interacts with it.

Fortunately there’s no specific things I need to do with a keyboard to function normally in OpenHab, but if there was, it would be pointless to not make it clear what has to be done.

No it doesn’t need to be said, it would be ridiculous for me to expect you to teach me how to code.

When the OH3 docs “will cover how to interact with OH 3 stuff” will be a good time and reduce the amount of posts along these lines.

I wish this didn’t have to be said, I really don’t like being the ahole in all this, but none of the trying stuff that doesn’t work, answering questions about why I’m trying to do a specific thing, responses that literally say they don’t know the answer, responses that worked in a different version so presumably they work now (but don’t), or responses that don’t get me the simple statement I need so I can move onto my next line of code all take time to process, investigate and reject. It’s very frustrating. I shouldn’t have to literally beg people to stop going off topic.

So far the best response has suggested JRuby.

If it’s cool with you, I’ll put together a summary of these responses as an answer to my initial question and mark that as the solution.

I’ll get you to sign off on it first.

It’s going to include the phrase “not yet fit for purpose and there is no development roadmap”

Woo hoo! Progress :smiley:

I worked out import instead of include myself and got this…

2021-01-25 03:34:02.632 [WARN ] [jython.core.log                     ] - The 'configuration.py' file is missing from teh python.path!
2021-01-25 03:34:02.661 [INFO ] [org.openhab.core.model.script.MyLog ] - before sunrise

So that’s positive.

Can someone please direct me to this illusive step #8? I’m convinced I haven’t seen anything with 8 steps listed.