Lambda functions fail (not thread safe?)

Setup, Configuration and Use Scripts & Rules
1

During my upgrade OH 2.1 to 2.3 I had to change a lot of my rules. It seemed that try-catch-finally caused some problems with lock sections (ReentrantLock). I removed most of the locks and now stumble into the next problems.

When two rules invoke the lambda function at the same time, an Exception is thrown:

... [ERROR] [clipse.smarthome.model.script.rrinfo] - bugRule2 - java.lang.IllegalStateException

Do I miss something out? If no, I would file an issue.

val Functions.Function1<Number, String> bugLambda =
    [ dummyArg |
		val String logClass = "rrinfo"
        val String logKey = "bugRule.bugLambda - "
        var String result = ""
        try {
            result = String.format("%d", dummyArg)
        } catch(Throwable t) {
            result = result + t.toString
            logWarn(logClass, logKey + t.toString)
        } finally {
            result = result + "finally"
        }
        return result
    ]


rule bugRule1
when
    Time cron "12/6 * * * * ?"
then
    val String logClass = "rrinfo"
    val String logKey = "bugRule1 - "
    try {
        logDebug(logClass, logKey + "in")
        val String result = bugLambda.apply(1)
        logDebug(logClass, logKey + "out:" + result)
    } catch (Throwable t) {
        logError(logClass, logKey + t.toString)
    }
end

rule bugRule2
when
    Time cron "12/6 * * * * ?"
then
    val String logClass = "rrinfo"
    val String logKey = "bugRule2 - "
    try {
        logDebug(logClass, logKey + "in")
        val String result = bugLambda.apply(2)
        logDebug(logClass, logKey + "out:" + result)
    } catch (Throwable t) {
        logError(logClass, logKey + t.toString)
    }
end

My system:

  • Ubuntu 16 LTS, docker: openhab:2.3.0-amd64-debian
  • bindings: homematic (2x: original ccu2 + raspi-ccu), hue, modbus, http, jdbc, mapdb, weather, astro
2

Lambdas are not thread safe. Nowhere in the Rules DSL docs or Xtend docs are they claimed to be thread safe. It would be a mistake to assume that anything in the Rules DSL is thread safe.

If you must have a thread safe lambda you have two options, listed in my recommended order of preference:

  1. Look for an approach that doesn’t require the lambda in the first place. In my experience, lambdas are a tool that rarely need to be used in Rules as there is almost always a better alternative that avoids many of the pitfalls lambdas bring with them. Design Pattern: Separation of Behaviors, Design Pattern: Associated Items, and applying Design Pattern: Working with Groups in Rules were written to show how to write generic Rules or combinations of Rules that do not require lambdas in the first place,

  2. Create multiple instances of the lambda. A lambda is just an Object. So create a pool of th2m to call so each thread get’s it’s own instance of the lambda.

  3. Put a lock around the call to the lambda, not inside the lambda itself.

Please do file the issue as this might be something the developers can address, but I suspect the problem lies deep down in the base libraries/languages that the Rules DSL is built upon.

3

Do you get this every time the rules run? I copy/pasted what you posted and I’m not getting any exceptions. OH snapshot 1361.

4

I would expect that if you are on a fast enough machine the timing may work out what the two lambas end up running in serial instead of parallel. As written the lambda shouldn’t take more than a dozen msec to run. Maybe add a Thread::sleep to keep that from happening to see if that makes it happen every time.

5

I tried sleeps in the rules and lambda, but never got an exception. The OP is running Ubuntu and docker, so likely not under powered. I’m currently running Oracle Java 8, Fedora 28, and have OH on an old dual core desktop. Maybe something was resolved in the snapshots… I haven’t been able to reproduce the issue.

Before migrating to JSR223-Jython, I used lambdas extensively, and they were often called simultaneously.

6

I have seen in the distant past, when I was using a lot of lambdas, strange behavior in lambdas. Keep in mind I’m dragging this up from years ago. Things I’ve personally seen or seen reported:

  • finally not being called every time, particularly with certain exception that happen that appear to be caught inside the Rules Engine itself and the stack never unwinds back into the Rule that called the code in the first place.
  • local variables in a lambda retaining/reflecting the value from a different invocation of the lambda
  • repeatedly calling the lambda (e.g. from a Group received update triggered Rule) randomly causes errors and adding locks inside the lambda does not prevent the errors
  • calling break or return from the lambda exits the whole Rule.

I’ve managed to eliminate pretty much all of my lambdas, not because of these problems but because they don’t make sense to use for me anymore given my generic Rules, so I can’t say if these are still a problem. And because they were always intermittent I was never ever able to gather enough logs or generate code where I could reliably reproduce the error to file an issue.

7

@5iver
I got the exception immediately without waiting. I didn’t count and didn’t waited for more than 2 times.

Within the real code I got these exception over the last days between 3 and 10 times per hour. The code is optimized for not running all the same time, but I cannot influence value trigger.

As I wrote, environment was: docker: openhab:2.3.0-amd64-debian

I won’t go for snapshots. I don’t expect it more stable than a stable version.

Issue: https://github.com/eclipse/smarthome/issues/6234

8

It is usually a good idea to test in the latest snapshot before filing an issue since it might already be fixed.

9

Rich,

I have a rules file that has a fair bit of repetitive code. Wouldn’t a lambda be the best way to write the code once and reuse it in various rules versus have the same code duplicated throughout?

10

It depends. Another approach may be Design Pattern: Separation of Behaviors. Yet another could be writing your Rules in a way where all those similar Rules end up merged into a single Rule. Perhaps using OH Scripts might be a better approach.

As with most things, it depends on the details. It may indeed be the case where using a lambda is the best option. But I’ve found that to only rarely be the case.

11

It seems to me that the experts using JSR223 too.

Since JSR223 is available, the philosophical question is why the world needs another programming language (Xtend*) an why should someone maintain the integration with openhab.

For me JSR223/Javascript looks promising, but I haven’t done too much.

Some disadvantages of JSR223/Javascript for keeping in mind but not off:

  • Error checking by the OH (LSB) didn’t work for me properly in Vscode => Debugging via logs
  • no new ES6 language features (just plain vanilla Javascript) => you get used to it
  • In my opinion JSR223/Javascript is based on the Nashorn engine which won’t be continued with Java 11…

I’m not sure about Jython (seem old 2.7) or Groovy (docs not ready).

12

Just a very few do.

Because Xtend was there before JSR223?
Because it’s restricted thus easier to use for most non-programmer type of home automation users ?

13

OH LSP will only check Rules DSL. However, I’m sure there are JS extensions for VSCode that will give you basic syntax checking.

As Marcus said, it’s easier for the non-programers to use, it’s been there longer, and far fewer OH users use JSR223 than use Rules DSL. The documentation for the JSR223 libraries and syntax is also not as thorough.

Ultimately I expect that the Experimental Rules Engine will take the place of Rules DSL for the new users/non-programers. Rules DSL won’t go anywhere anytime soon since so many users, including experts, depend upon it. My hope is that JSR223 will continue as well. In fact @5iver mentioned that the ERE can already call JSR223 rules so this might be further along than I thought. But since there are so few users of JSR223, there are fewer people working to upgrade and maintain it. If you’ve the skills we sure could use the help.