Unreliable rule triggering (cron) with openhab 2.3

[mstormi (Markus Storm)]

@raulix

Don’t use exceptions.

That’s pure theory without practical use!

The key is in the second sentence: write code that does not fail.
You don’t need to use exceptions when your code is bullet-proof.
You don’t even need to use lambdas.
It’s just that you want to use both of them because you are used to that way of programming.

@raulix

I expect documented language features to be working and not get indoctrinate by people who are not used to that a little bit “more advanced” features

You’re actually the only person I have seen to date to use exceptions in OH rules.
Bottom line is it doesn’t work. Ok. So you can either keep expecting and insisting on this to be a bug and wait for it to get fixed some day [or never because some developer has a different opinion on whether that is supposed to work at all - I don’t know if it is a bug and in fact I don’t care].
Or you change your fundamental approach and refactor your code to get the job done as a number of experienced people here are effectively advising you to.

[rlkoshak (Rich Koshak)]

@raulix

Without exception/finally handling you cannot use reliably lambda functions.

Not necessarily true. It depends on how the lambda is written and what, if any shared resources the lambda uses.

And when you read that thread you will find that even WITH the exception handling and the locks you can’t reliably use lambda functions because you cannot guarantee that the finally will execute every time. Some errors will kill the Rule and cannot be caught.

@raulix

I expect documented language features to be working and not get indoctrinate by people who are not used to that a little bit “more advanced” features.

We should only rarely be using lambdas in the first place. Lambdas were never a documented feature of the Rules DSL. The were something that was discovered, not written with Rules DSL in mind.

Lambdas are NOT documented anywhere in the OH Docs (I just did a search, the only mention of lambda is a placeholder in the beginners tutorial). The only documentation for them are in forum postings.

Global lambdas are not recommended for general use (I should add that warning to the lambda example posting). They should only be employed in very rare circumstances. If you can’t or won’t consider an approach that doesn’t require lambdas, you should consider switching to JSR223. Otherwise you should write your Rules so they are more generically (i.e. rather than many Rules that call a common lambda, have one Rule) or trigger other Rules to implement the common code by generating an event (see Separation of Behaviors DP).

I find a bit of a stretch to call an undocumented feature of the language not working as expected being a bug. It’s an undocumented feature of the language in the first place and as the person who has probably written the most text on what lambdas are and how they work on this forum I’m saying you shouldn’t be using them. Lambdas were never really designed for our use in Rules. They just happened to be there. We shouldn’t be surprised that they don’t work well.

@raulix

The exception handling worked more stable in OH 2.1 than in OH 2.3.

No it didn’t. All the same problems with exception handling and the finally clause not executing every time, particularly in lambdas, has existed since at least OH 1.8.1. It may be the case that the stricter checking introduced between OH 2.2 and 2.3 is causing more things that have always been problems to result in exceptions instead of just silently failing, but the problem with exceptions remains the same.

@mstormi

You’re actually the only person I have seen to date to use exceptions in OH rules.

I’ve seen them lots, but it is almost always when the user is also using ReentrantLocks. If you are using locks, it is almost a must to use try/catch/finally. However, because you can’t trust that the finally will always be called and therefore the lock unlocked because of how some errors kill the Rule, locks, like lambdas, should only be used sparingly and everything possible must be done in such Rules to prevent exceptions from occurring. I strongly recommend against using locks in lambdas.

@mstormi

or never because some developer has a different opinion on whether that is supposed to work at all - I don’t know if it is a bug and in fact I don’t care

If I were to guess, I’d say that lambdas are inherited from Xtend but they were never intended to be used in a multi-threaded context like what exists in OH Rules DSL. Looking at all of the examples I can find, a new instance of the lambda is created for each thread. With a separate lambda per thread there is no need for the locks and no need for thread safeness. But if we create a new instance of the lambda in each Rule then there is no point in using the lambda in Rules DSL in the first place.

[rossko57]

@mstormi

You’re actually the only person I have seen to date to use exceptions in OH rules.

To be fair, I’ve got try/catch exception handling in my lambdas.
But it’s there as belt and braces, as I have written the code so that the exceptions don’t arise. (so far as I can make it!!)

(belt and braces = English idiom meaning protected by redundancy against trousers falling down)

As you say, this a practical approach to make a working system, whether or not there are bugs in this area of DSL. I expect it to continue working whether or not those bugs get fixed one day.

[raulix (R.R.)]

Documented in the OH doc is the “finally” handling. See here:
https://www.openhab.org/docs/configuration/rules-dsl.html#concurrency-guard
But if there is a “finally” than the “catch” should work too (technically).

Just to make clear: I don’t claim exceptions should be used as substitution for carefully programming. But exception catching and logging is very helpful for debugging and finding own bugs.

In my OH installation there are a lot of invalid items state at startup. I have no real change to debug these rule (only by a restart). At least I can see that something was going wrong…

@mstorm: If you don’t use exception blocks and logging, you can’t know what’s going wrong in your system - that could lead to feeling very comfortable. But in professional environments that is an antipattern for good reason.

[raulix (R.R.)]

Topic documentation

In my eyes OH does not have an own documentation about rule syntax, there are just some examples.

https://www.openhab.org/docs/configuration/rules-dsl.html#the-syntax

Note: The rule syntax is based on Xbase and as a result it is sharing many details with Xtend, which is built on top of Xbase as well. As a result, we will often point to the Xtend documentation for details.

So I looked in Xtend docu for language features too. And there you can find lambda functions and try-catch-constructs.

[mstormi (Markus Storm)]

[quote=“raulix, post:24, topic:51431”]
@mstorm: If you don’t use exception blocks and logging, you can’t know what’s going wrong in your system - [/quote]

Not true as a general statement.
There’s debuggers and static code analysis tools, just to name some alternative approaches.
And logging is available in OH.

@raulix

In my OH installation there are a lot of invalid items state at startup. I have no real change to debug these rule (only by a restart). At least I can see that something was going wrong

Wrong, you do. There’s multiple options:

• you can check for NULL/invalid state in your code before using (that’s part of the ‘write code that does not fail’ approach)
• you can initialize items on system startup using ‘System started’ trigger (also part of that approach)
  You can even use persistence on items.
  FWIW, it helps to make sure your system loads items before starting on rules execution.
  There’s threads on this in the forum.

[raulix (R.R.)]

@mstormi

There’s debuggers and static code analysis tools, just to name some alternative approaches.
And logging is available in OH.

I know debugger and static code analysis tools for several other programming languages, but there are no such tools documented for OH rule files. (VSCode with LSB is something else, but helpful though.)

@mstormi

you can check for NULL/invalid state in your code before using (that’s part of the ‘write code that does not fail’ approach)

By “Checking for NULL states” you don’t suppress/exclude other programming mistakes.

@mstormi

You can even use persistence on items.
…
FWIW, it helps to make sure your system loads items before starting on rules execution.

Items can be NULL despite of persistence. See here: https://www.openhab.org/docs/configuration/persistence.html#startup-behavior

Persistence services and the Rule engine are started in parallel.

[mstormi (Markus Storm)]

I rather feel you just want to avoid going down that route at all cost.
This is getting annoying so I’ll stop trying to convince you. Take the advice or leave it.

[rlkoshak (Rich Koshak)]

@raulix

But if there is a “finally” than the “catch” should work too (technically).

Only if the exception is thrown back into the Rule. It appears that in some cases the exception takes place in and is aught by the Rule Engine itself and instead of rethrowing the exceptions back up the stack trace and into the Rule it gets caught outside your Rule and the Rule gets terminated. And this problem is exacerbated by lambdas for some reason. I’ve seen, but not been able to study, that a exception that happens in a Rule can be caught in the Rule, but if it happens in the lambda it kills the Rule.

This is the root to why locks + lambdas + exception handling is such a dangerous thing in OH.

There are some times when a lock cannot be avoided, just like there are times where a lambda or a Thread::sleep cannot be avoided. But because of the way the Rules DSL works, all of these should be treated as a last resort. It is worth going to extraordinary lengths to avoid their use.

I’ll be the first to admit that there are some very significant problems with the Rules DSL. Error handling is perhaps its weakest point. I like the Rules DSL because it is easier for new users to become productive in it compared to more feature rich languages, but I would never ever recommend it for use in a “professional environment.” If you are used to and expecting something like that, I again implore you to use JSR223 instead.

@raulix

But in professional environments that is an antipattern for good reason.

Because of the limitations of exception handling outlined above, relying on exception handling in your Rules gives you a similar false sense of comfort. I think the counter argument is that the Rules DSL isn’t a processional environment. It’s not intended to be. And because of the limitations of exception handling one could make a good argument that within the Rules DSL using try/catch, and especially try/catch/finally would be an anti-pattern.

For the most part, the number of things that can cause an exception in a Rule is relatively low.

1. The biggest one is type problems (e.g. an Item is NULL or UNDEF but the Rule just assumes it is a Number without checking first).
2. The next biggest one is trying to parse a String to a Number or a DateTime that is not correct for that type.
3. Next comes trying to sendCommand or postUpdate an invalid state to an Item (e.g. a Number > 100 to a Dimmer Item).

1 is the most troublesome because this exception usually just kills the Rule and cannot be caught.
3 is actually just a special case of 1.

I can’t think of others off the top of my head. I’m sure there are more. But all three of the above can be handled quite effectively by using standard defensive programming techniques (i.e. checking the type of an Item before you try to use it). And I believe both 1 and 3 can’t be caught in your Rule anyway. Your Rule will just stop.

@raulix

In my eyes OH does not have an own documentation about rule syntax, there are just some examples.

We would welcome any contributions you can make to improve them.

@raulix

So I looked in Xtend docu for language features too. And there you can find lambda functions and try-catch-constructs.

There are also language features like arrays and classes which do not exist in the Rules DSL in the Xtend docs. Just because Xtend supports it doesn’t mean Rules DSL does.

Personally, I have high hopes for the Experimental Rules Engine. Not only does it introduce concepts that will make certain types of Rules really easy to create and it gives the non-technical users a nice UI based way to create Rules, but under the covers it appears to support JSR223 Rules. By default we can enter JavaScript in the UI but 5iver has discovered that the ERE can call Jython and I would assume Groovy written Rules as well. I’m hoping that a lot of these sorts of issues will simply go away.

[raulix (R.R.)]

My solution to these problems: an external rule engine based on Python 3 and the REST API.

Features:

• connects to OpenHAB via REST API
• gets updated by change notifications about state changes of items, groups, things
• reads and caches states of items, groups and things.
• change states via COMMANDs and UPDATEs
• write your own rule classes, subscribe for state changes and cron events
• runs as daemon (command line interface: --start/–stop)
• programming language: Python 3 (use your favorite tools incl. debugger, unittests, code inspection)

Homepage: https://github.com/rosenloecher-it/prend