Automation/Orchestration Design Patterns

Is it possible (physically) to manually switch the light (either direction) without being in range of (and triggering) the motion detector ? Or are you using a GUI on web/phone to do the “manual” switching ? If you are using a GUI to do the manual switching, you could set a latching variable in the handling of the proxy event switch from the GUI. (This could also be a second proxy which you could check at execution time in other rules.)

Another approach would be to use the manual OFF operation to start a named timer which is checked for existence during the Motion ON rule and skipping the sendCommand that manipulates the Light if the timer is still running/exists.

Do you also have a PROXY_SYNC-type rule which keeps the proxy and real device states aligned ? Such a rule would trigger on “Item Lt_Office changed from OFF to ON” (as in your above SW_1) or “ON to OFF” as in your SW_2.

rule "PXY_SYNCH: Update Lt_Office PROXY state"
    when
        Item Lt_Office changed from OFF to ON
    then
        Thread::sleep(10)
        logDebug("OFFICE","PXY_SYNCH: Updating Lt_Office Proxy to:" + Lt_Office.state)
        postUpdate(Lt_Office_PROXY, Lt_Office.state)
end

You do have to be careful to avoid open-ended updating loops between devices and proxies. By that I mean use “changed from…to…” as opposed to simply “changed”. So you may want two sync rules for both variants of “from…to…”. You can have either the device or the proxy with a simple “changed” rule, but having both with a simple “changed” will get you a potential race condition. (This can be entertaining in the “WTF is going on sense”, but definitely not what you want.)

Especially with dimmers you can easily get a race condition going between the device and proxy, so it is also important to have a Thread::sleep() in the updating clauses on syncing rules. (You may have to experiment with the argument to sleep() that works for you. ) The sleep() allows for both order of sequence and differential execution timing between sendCommand and postUpdate to be reasonably smoothed to induce the desired behavior.

For the garden lighting yes we can switch manually without being in range. The bticino binding is contunually polling the lighting system so knows whether lights are on or off so I think whether I switch via the GUI or the wall I can pick up an Item change. I think the latching variable sounds the one to try. So do I add another switch item to the proxy rule or is this a variable in which case how does this get sent to the SW_1 rule? If it is a switch perhaps then I can use this in the interface to display or even change the status of the automation.

I think I’d set the latch as another switch variant on the real device name, viz

sw_Light_Office  (the real deal)
sw_Light_Office_PROXY (for action target, interceptable at rule execution time as otherwise shown)
sw_Light_Office_LATCH (a specific state item which can be part of the interception checking routing in the PROXY, but settable by anything in your rule/script space).   

The LATCH.state is just an aggregate result of all the things where you want to override your general rules, but it lets you set/unset the LATCH.state anywhere in your rules and shorthand that into the PROXY state check at interception time rather than enumerating all the exception cases in the PROXY rule. I personally think I"d keep timer checks etc inline in the PROXY rather than having the timer toggle the LATCH, and use the LATCH to capture other exceptional conditions. (eg nobody home, your motion situations, etc)

Thanks that works for me - final question I promise! Can you combine 2 switches on one line in the sitemap - so I’d want to have Office light it’s latch state which I can use as an override (ie turn off/on automation) and also it’s real state which I can also turn on and off. I’m guessing not as I’ve not seen it anywhere.

You might be able to display something in the real item using color driven by the override state, but you’ll need to tinker with that a bit. From a UI/functionality perspective, I don’t think you’d want them fully intertwined with each other in a full control sense. I’d probably put the override “switch(es)” in a separate group entirely and “hide” them one layer down just to keep the confusion factor down.

I’ve debated whether to continue this thread or to start a new one and decided to add to this thread. Ultimately this will probably be added to the wiki and/or included in the OH 2 user’s guide.

So, it has been several months since I posted my design patterns above and as I’ve refactored things I’ve come up with some improvements which I will post here.

Time of Day Design Pattern

In the original above I use Switches to represent the current time of day state. This made sense at the time because it more closely followed how a state machine would work. However, in practice, I found that using multiple switches adds a lot of extra logic when you care about more than one state at a time (see the Group and Filter code below). So the big change is to follow the suggestion @watou made on another thread and instead of using multiple Switches use a single (two actually but more on that later) String Item to represent state.

In the Items and Rules below there are two Items to store the current time of day and the previous time of day. There are a number of other Items used to trigger the start of a new time of day based on sunrise and sunset. We use the switch to trigger a rule to transition to the new time of day state and the DateTime when openHAB starts and we need to figure out what time of day it currently is.

Items

String          TimeOfDay
String          PreviousTimeOfDay

Switch          Twilight_Event                              (Weather) { astro="planet=sun, type=set, property=start, offset=-90" }
DateTime        Twilight_Time   "Twilight [%1$tr]"  <moon>  (Weather) { astro="planet=sun, type=set, property=start, offset=-90" }
Switch          Sunset_Event                                (Weather) { astro="planet=sun, type=set, property=start" }
DateTime        Sunset_Time     "Sunset [%1$tr]"    <moon>  (Weather) { astro="planet=sun, type=set, property=start" }
Switch          Sunrise_Event                               (Weather) { astro="planet=sun, type=rise, property=start" }
DateTime        Sunrise_Time    "Sunrise [%1$tr]"   <sun>   (Weather) { astro="planet=sun, type=rise, property=start" }

String Condition_Id "Weather is [MAP(yahoo_weather_code.map):%s]" (Weather) { weather="locationId=home, type=condition, property=id" }

Rules

import org.openhab.core.library.types.*
import org.joda.time.*
import org.eclipse.xtext.xbase.lib.*

val Functions$Function3 updateTimeOfDay = [String tod, String ptod, boolean update |
        logInfo("Weather", "Setting PreviousTimeOfDay to \"" + ptod + "\" and TimeOfDay to \"" + tod + "\"")
        if(update) {
                TimeOfDay.postUpdate(tod)
                PreviousTimeOfDay.postUpdate(ptod)
        }
        else {
                TimeOfDay.sendCommand(tod)
                PreviousTimeOfDay.sendCommand(ptod)
        }
]

rule "Get time period for right now"
when
        System started
then
    val morning = now.withTimeAtStartOfDay.plusHours(6).millis
        val sunrise = new DateTime((Sunrise_Time.state as DateTimeType).calendar.timeInMillis)
        val twilight = new DateTime((Twilight_Time.state as DateTimeType).calendar.timeInMillis)
        val evening = new DateTime((Sunset_Time.state as DateTimeType).calendar.timeInMillis)
        val night = now.withTimeAtStartOfDay.plusHours(23).millis

        if(now.isAfter(morning) && now.isBefore(sunrise))       updateTimeOfDay.apply("Morning", "Night", true)
        else if(now.isAfter(sunrise) && now.isBefore(twilight)) updateTimeOfDay.apply("Day", "Morning", true)
        else if(now.isAfter(twilight) && now.isBefore(evening)) updateTimeOfDay.apply("Twilight", "Day", true)
        else if(now.isAfter(evening) && now.isBefore(night))    updateTimeOfDay.apply("Evening", "Twilight", true)
        else                                                    updateTimeOfDay.apply("Night", "Evening", true)
end

rule "Morning start"
when
        Time cron "0 0 6 * * ? *"
then
    updateTimeOfDay.apply("Morning", TimeOfDay.state.toString, false)
end

rule "Day start"
when
        Item Sunrise_Event received update ON
then
    updateTimeOfDay.apply("Day", TimeOfDay.state.toString, false)
end

rule "Twilight start"
when
        Item Twilight_Event received update ON
then
    updateTimeOfDay.apply("Twilight", TimeOfDay.state.toString, false)
end

rule "Evening start"
when
        Item Sunset_Event received update ON
then
        logInfo("Weather", "Its Evening!")
        PreviousTimeOfDay.sendCommand(TimeOfDay.state.toString)
        TimeOfDay.sendCommand("Evening")
end

rule "Night started"
when
        Time cron "0 0 23 * * ? *"
then
    updateTimeOfDay.apply("Night", TimeOfDay.state.toString, false)
end

NOTES:

  • We will need both the Times and the Events
  • Persistence is not required for this to work
  • Because rrd4j does not support Strings and mapdb does not give you the previous value, maintaining the previous time of day in a separate Item is required (if you need to know the previous state in your rules)

In a rule that may do something different based on the time of day you use an if statement similar to:

if(TimeOfDay.state.toString == "Night")

To trigger a rule when the time of day changes:

when
    Item TimeOfDay received command
then

To trigger a rule when it becomes a specific time of day:

when
    Item TimeOfDay received command "Night"
then

Group and Filter

The concept is still the same but I wanted to update my example with my current lighting setup so it illustrates my use of TimeOfDay and PreviousTimeOfDay and it illustrates some more examples of how to use Groups to organize things to make rules simpler.

The big changes you will find is the elimination of the lambda and the movement of the state that was being stored in global vars to Items. Also, by using Groups I’m able to consolidate the rules that were triggered based on time of day Switches into a single rule. A new concept illustrated here is also the use of naming conventions which we can use to programmatically construct the name of an Item of a Group and filter that Item or Group out of a higher level group.

The concept is as follows:

  • Each time of day has two groups, an ON group and an OFF group. The group names follow the pattern g<time of day>Lights<ON or OFF> (e.g. gMorningLightsON). Lights which are members of an ON group will be turned on when that time of day starts. Lights which are members of an OFF group will be turned off when that time of day ends. This allows one to turn on a light during one time of day and turn them off at a later time of day without toggling them between times of day.
  • All of the groups belong to a gTimerLights group so we can find the one we want by name when the time of day changes.
  • Each light now has a secondary Override switch and these Overrides belong to the gLightsOverride
  • The old whoCalled var is now a String state and is used to determine whether a light is turned on manually (and therefore should override the rules) or by a rule.
  • There is one rule that toggles the lights based on the weather which can be manually overridden.
  • By default V_WhoCalled is set to “MANUAL”. When a time of day causes the lights to change all overrides are removed and while the rule is processing the V_WhoCalled gets changed to “TIMER”. When the Weather Rule executes V_WhoCalled is set to “WEATHER”. When any light is toggled for any reason (manually, timer, or weather rule) the Override Lights rule gets called. If V_WhoCalled is “MANUAL” it means the light has been manually triggered so the light is marked as overridden.

Items

Group:Switch:OR(ON,OFF) gLights "All Lights"    <light>
Group gTimerLights
Group gMorningLightsON          (gTimerLights)
Group gMorningLightsOFF         (gTimerLights)
Group gDayLightsON              (gTimerLights)
Group gDayLightsOFF             (gTimerLights)
Group gTwilightLightsON         (gTimerLights)
Group gTwilightLightsOFF        (gTimerLights)
Group gEveningLightsON          (gTimerLights)
Group gEveningLightsOFF         (gTimerLights)
Group gNightLightsON            (gTimerLights)
Group gNightLightsOFF           (gTimerLights)
Group gWeatherLights

Group gLightsOverride
String V_WhoCalled

Switch  S_L_Front           "Front Room Lamp"  <light> (gLights, gWeatherLights, gMorningLightsON, gMorningLightsOFF, gTwilightLightsON, gEveningLightsOFF) {zwave="3:command=switch_binary"}
Switch  S_L_Front_Override                             (gLightsOverride)
Switch  S_L_Family          "Family Room Lamp" <light> (gLights, gWeatherLights, gTwilightLightsON, gEveningLightsOFF)                                      {zwave="10:command=switch_binary"}
Switch  S_L_Family_Override                            (gLightsOverride)
Switch  S_L_Porch           "Front Porch"      <light> (gLights, gEveningLightsON, gEveningLightsOFF)                                                       {zwave="6:command=switch_binary"}
Switch  S_L_Porch_Override                             (gLightsOverride)
Switch  S_L_All         "All Lights"           <light>

Rules

import org.openhab.core.types.*
import org.openhab.core.items.*
import org.openhab.core.library.items.*
import java.util.Set

// Yahoo cloudy weather condition IDs
val Set<String> cloudyIds = newImmutableSet("0",  "1",  "2",  "3",  "4",  "5",  "6",  "7",  "8",
                                                "9", "10", "11", "12", "13", "14", "15", "16", "17",
                                                "18", "19", "20", "26", "28", "35", "41", "43", "45",
                                                "46", "30", "38")

// Turn off the lights from the previous time of day and turn on the lights for the current time of day
rule "TimeOfDay changed"
when
        Item TimeOfDay received command
then
        // Disable overrides
        V_WhoCalled.sendCommand("TIMER")

        Thread::sleep(100) // give lastUpdate time to catch up

        // Turn off previous time of day lights
        val lastTod = PreviousTimeOfDay.state.toString
        val offGroupName = "g"+lastTod+"LightsOFF"
        logInfo("Lights", "Timer turning off " + offGroupName)
        val GroupItem offGroup = gTimerLights.members.filter[g|g.name == offGroupName].head as GroupItem
        offGroup.members.forEach[light |
                logInfo("Lights", "Timer turning OFF " + light.name)
                light.sendCommand(OFF)
        ]

        // Turn on current time of day lights
        val onGroupName = "g"+receivedCommand+"LightsON"
        logInfo("Lights", "Timer turning on " + onGroupName)
        val GroupItem onGroup = gTimerLights.members.filter[g|g.name == onGroupName].head as GroupItem
        onGroup.members.forEach[light |
                logInfo("Lights", "Timer turning ON " + light.name)
                light.sendCommand(ON)
        ]

        Thread::sleep(1000) // give all Override rules to finish running after all the switching above
        V_WhoCalled.sendCommand("MANUAL")
        gLightsOverride.members.forEach[o | o.sendCommand(OFF)]
end

// Control ALL the lights with this switch. Put a sleep between triggering each light so the "Override Lights"
// rule is guaranteed to execute correctly.
rule "All Lights Switch"
when
        Item S_L_All received command
then
        V_WhoCalled.sendCommand("MANUAL") // Using the All switch counts as an override
    gLights.members.forEach[light |
        sendCommand(light, S_L_All.state.toString)
        try {Thread::sleep(110)} catch(InterruptedException e) {} // sleep to avoid fouling up mostRecent above
    ]
end

// This rule gets called for ALL updates to ALL lights. Set the Override flag to ON for
// any light that is updated when V_WhoCalled is set to MANUAL
rule "Override Lights"
when
        Item gLights received update
then
        Thread::sleep(100) // give lastUpdate time to be populated
        val mostRecent = gLights.members.sortBy[lastUpdate].last as SwitchItem
        if(V_WhoCalled.state == "MANUAL") {
                logInfo("Lights", "Overriding " + mostRecent.name)
                gLightsOverride.members.filter[o|o.name == mostRecent.name+"_Override"].head.sendCommand(ON)
        }

        // Keep S_L_All up to date, but use postUpdate so we don't trigger rule below
        // If one or more lights is OFF leave the state as OFF so we can toggle the rest
        if(gLights.members.filter[l|l.state==OFF].size > 0) S_L_All.postUpdate(OFF)
        else S_L_All.postUpdate(ON)
end

// When it is Day, turn on or off the WeatherLights when the weather says it is cloudy.
rule "Weather Lights On"
when
        Item Condition_Id changed
then

        // Only run the rule during the day
        if(TimeOfDay.state.toString == "Day") {

                // Get the new light state
                val State state = if(cloudyIds.contains(Condition_Id.state)) ON else OFF

                // Toggle any non-overridden lights
                V_WhoCalled.sendCommand("WEATHER")
                gWeatherLights.members.forEach[ light |
                        if(gLightsOverride.members.filter[o|o.name == light.name + "_Override"].head.state == OFF &&
                           light.state.toString != state.toString){

                                logInfo("Lights", "Weather turning " + light.name + " " + state.toString)
                                light.sendCommand(state.toString)
                                try {Thread::sleep(100)} catch(InterruptedException e){} // don't overwhelm "Any light in gLight triggered" rule
                        }
                ]
                V_WhoCalled.sendCommand("MANUAL")
        }
end
5 Likes

Julian, any chance you can post your working code? I’m trying to do the same thing - kitchen switch which detects motion, but when I want light on permanently I want to override it at the manual switch. I go past my motion sensor on the way in and the way out.

Posting this here so I can easily find it later. Two more design patterns came to mind.

Separation of Behaviors

Problem Statement:
I have found in my rules and from helping others that often times certain sorts of checks, conditionals, etc. cross cut multiple functionality categories. For example, lighting and HVAC may both care about what time of day it is. Just about anything may want to send a notification. Lighting, blind/rollershutter controls/HVAC may all care whether it is cloudy.

What can often happen is the same logic gets duplicated in multiple places as the rules check for these states or generate the actions. For example, if today we are using my.openhab for notifications we might have a call to sendNotification() all over our files and if we decide we don’t like my.openhab and want to use Notify My Android instead we have to change it everywhere.

Concept:

Set up a proxy Item to represent the state or trigger the action and a rule to centralize the logic. There are two approaches depending on the direction of the logic.

The first approach is to set up a String or Number Item that your rules sendCommand some value to that triggers an action, such as sending a Notification. In this rule you can then start to take into consideration additional traits or add additional logic which would be difficult to implement across your files as lambdas such as, as illustrated below, use a different notification service depending on the time of day.

Items

String Notification_Proxy_Info
String Notification_Proxy_Alarm

Rules

val String logNameNotification = "notification"

rule "Dispatch Info Notification"
when
        Item Notification_Proxy_Info received update
then
        val String msg = Notification_Proxy_Info.state.toString
        logInfo(logNameNotification, msg)
        if(TimeOfDay.state != "Night") sendPushToDefaultDevice(msg)
end

rule "Dispatch Alarm Notification"
when
        Item Notification_Proxy_Alarm received update
then
        val String msg = Notification_Proxy_Alarm.state.toString
        logError(logNameNotification, msg)
        if(TimeOfDay.state == "Night") sendPushToDefaultDevice(msg) else sendNotification("email", msg)
end

I can send a notification with a simple Notification_Proxy_Info.sendCommand("Notification text!")

The second approach is to set up a proxy Item to hold the result of a complex calculation (e.g. a Number of a Switch) and the state of this Item is checked in other rules. For example, below I have a Weather_Cloudy Item Switch which gets set to ON when the Weather’s Condition ID says it is cloudy. By centralizing this check I can now react in rules across my environment without duplicating logic and change it if needed in one location (e.g. switch from Yahoo to Wunderground).

NOTE: This is an update to the Lighting rule int eh Group and Filter example above.

Item

Switch      Weather_Cloudy       "Conditions are Cloudy [%s]" <rain>

Rule

rule "Update Cloudy Switch"
when
        Item Condition_Id changed
then
    // Yahoo cloudy weather condition IDs
    val Set<String> yahooCloudyIds = newImmutableSet("0",   "1",  "2",  "3",  "4",  "5",  "6",  "7",  "8",
                                                         "9",  "10", "11", "12", "13", "14", "15", "16", "17",
                                                         "18", "19", "20", "26", "28", "35", "41", "43", "45",
                                                         "46", "30", "38", "29", "30", "44")
    // https://www.wunderground.com/weather/api/d/docs?d=resources/phrase-glossary
        val Set<String> wundergroundCloudyIds = newImmutableSet("2",  "3",   "4",  "6", "9", "10", "11", "13",
                                                                "14", "15", "16", "18","19", "20", "21", "22",
                                                                "23", "24")
        val isCloudy = yahooCloudyIds.contains(Condition_Id.state.toString)
        if(isCloudy && Weather_Cloudy.state != ON){
                Weather_Cloudy.sendCommand(ON)
        }
        else if(!isCloudy && Weather_Cloudy.state != OFF) {
                Weather_Cloudy.sendCommand(OFF)
        }
end

In the lighting rule I can check if it is cloudy with a simple Weather_Cloudy.state == ON.

Dead Man’s Switch

Problem Statement
This one is a bit more complex in both implementation and concept. Sometimes you are in a situation where you want to know where a certain command came from but there is no way built into the binding or the device to tell the difference between a manually initiated command at the device or a command initiated by a rule. For example, when one wants to override a rule’s behavior when a light switch is manually triggered.

Concept
Create a dead-man’s-switch which gets set to one value at all times except when a rule is running. When a rule runs it temporarily sets the dead-man’s-switch Item to something beside the default value then changes it back to the default value. A rule gets triggered for ALL changes to the device’s Item. This rule check’s the dead-man’s switch and if it is set to the default value we know the Item was manually triggered.

For example, in the Group and Filter rules above I have a dead-man’s-switch called V_WhoCalled which gets set to “MANUAL” by default, “TIMER” by the rule that changes the lights based on Time Of Day, and “WEATHER” when the weather lighting rule runs.

In a simpler and more genericexample:

Items

String DeadMansSwitch
Group gOverrideItems

// Switches, Dimmers, etc that are members of gOverrideItems which we want to know whether it was manually changed or not

Rules


// Set the Dead Man's Switch off of default during startup
rule "System Started"
when
    System started
then
    DeadMansSwitch.sendCommand("STARTUP")
    createTimer(now.plusSeconds(30), [| DeadMansSwitch.sendCommand("MANUAL")
end

rule "Timer Rule"
when
    Item TimeOfDay changed
then
    DeadMansSwitch.sendCommand("TIMER")
    Thread::sleep(50) // give it time to populate through the event bus
    // do stuff
    Thread::sleep(200) // give stuff time to complete
    DeadMansSwitch.sendCommand("MANUAL")
end

// more rules which trigger members of gOverideItems

rule "Is Manually Triggered?"
when
    Item gOverrideItems received update // we don't care if it triggers more than once per update 
then
    if(DeadMansSwitch.state.toString == "MANUAL") {
        // the Item was manually triggered
    }
end

In the rules above if the Item is updated at the device (assuming the device reports such updates) or on the sitemap DeadMansSwitch will be “MANUAL” and we will know the device was manually updated.

8 Likes

Nice !

One more design pattern to add here so I can find it again and reference is future posts.

#Group Persistence

Problem Statement
There are several reasons why a home automation enthusiast would want to use openHAB persistence: charting, using historical Item states in rules logic, restore previous values on openHAB restart, and my.openhab integration, detailed analysis, access to the data with external tools, etc. However, different databases are more or less well suited to each of these use cases.

Concept
Since no one persistence engine is best for all use cases, configure more than one and use Groups to allocate which Items get saved to which persistence engine and how.

For example, one can set up MapDB for restoreOnStartup, rrd4j for charting recent data and historical data, my.openhab for IFTTT integration, and MySQL for Items for detailed external analysis.

For each use case, create a Group and only use these groups in the .persist files.

Then allocate your Items to whatever group(s) based on how that specific Item will be used.

Example
In my current setup I restoreOnStartup on all my Items, I use rrd4j for charting and historical state, I have a couple of Items I expose to IFTTT through my.openhab, and no Items that I analyze or expose the DB outside of OH.

Items:

// Persistence Groups
Group gMyOpenhab // for Items that are exposed to IFTTT
Group gChart     // for Items to persist for charting
Group gHistory   // for Items to preserve their history
//Group gRestore // for Items to restore on startup (currently everything so commented out)

...
// Example Item that I Chart
Number          Weather_Temp_Min        "Minimum Outside Temp [%.0f] °F"        <temperature> (gChart, gWeather_Temp_Chart)

// Example Item that I use historical data
Contact       N_D_Front                        "Front Door [MAP(en.map):%s]"             <frontdoor>  (gHistory, gDoorSensors, gRemindDoorSensors, gAlarmSensors, gHydraSensors)    { mqtt="<[mosquitto:entry_sensors/main/front_door:state:default]" }

// Example Item that I expose to my.openhab NOTE: it is also a member of gHistory
Switch  T_D_Garage1     "Garage Door 1"         <garagedoor> (gHistory, gMyOpenhab, gGarageOpeners)

*mapdb.persist file: I only use MapDB for restoreOnStartup and restoreOnStartup applies to all Items

Strategies {
        default = everyUpdate
}

Items {
        // persist all items on every change and restore them from the db at startup
        * : strategy = everyChange, restoreOnStartup
}

NOTE: I don’t know if setting a default strategy is required, I included it anyway, but it is basically meaningless because it is never used in the Items section.

To make restoreOnStartup work we need to persist every change. The ‘*’ resolves to all Items.

rrd4j.persist file: I use rrd4j for both charting and historic data. Because of limitations of rrd4j data must be saved at least every minute for charting or else you end up with blank charts. I’ve also found getHistoricState and previousState do not work without saving every minute as well.

Strategies {
        // for rrd charts, we need a cron strategy
        everyMinute : "0 * * * * ?"

        default = everyChange
}

Items {
        // additionally persist weather info every minute
        gHistory* : strategy = everyUpdate, everyMinute
        gChart*   : strategy = everyUpdate, everyMinute
}

myopenhab.persist:

Strategies {
        default = everyUpdate
}

Items {
        gMyOpenhab* : strategy = everyUpdate
}

Advantages

The advantages of this approach include:

  • The persistence behavior for each Item is documented with the Item (i.e. the group membership) keeping as much information about the behavior of the Item in one place
  • Adding new Items or changing how they are persisted simply involves changing that Item’s group membership.
  • Once set up, the .persist files need never be changed unless you decide move from one persistence engine to another
  • Uses the best persist engine for the desired behavior. For example MapDB can save all types of Items but only keeps the last value do it is great for restoreOnStartup but not great for historic data whereas rrd4j is great for charting recent data but can’t save anything that isn’t a numerical value so isn’t so great for restoreOnStartup
12 Likes

Has modeling of things using an object oriented paradigm ever been discussed? I am admittedly new to making rules, but coming from OOP languages, I find it very frustrating that in modeling “things” in openHAB we have perhaps the most apt application of OOP in the history of the universe, and yet here we are stuck in procedural programming.

I’d prefer to be telling my things (objects) to turn off and let some internal logic to the thing that can be reused cleanly deal with the quirks of how that thing should be used. Like in the “someone’s still observed via motion in the kitchen, so we shouldn’t turn off the light yet even if someone tries to” case mentioned by the OP. I’d rather let that logic be baked into the thing (since that’s what it belongs to) rather than have it spilled out across many different items and rules.

I’m not familiar with recipes and templates, but I think that templates and recipes don’t really solve the root problem. They just make it someone else’s (still hard) problem. Better to use recipes to promote reuse, not as a cover up.

It’s probably a contentious topic, but am I off base here? What am I missing? Flame away…

I’ve posted on this before. I’ve not much time so I apologize for being terse.

  1. Rules deal with Items. Not Things.

  2. The rules dsl is not procedural. It is event based (think languages like Erlang). There is a reason why OO languages are rarely chosen in similar event driven systems (asterisk, freeswitch, tasker, telecom systems, etc)

  3. Given the architecture of oh it is debatable whether OO would be a superior approach. I’ve been a professional programmer using OO languages for close to two decades now so I do have experience to back up my opinion.

  4. Home automation behaviors rarely involve just one item or just one class of items. It’s usually all about reacting to events from one Item and generating events sent to other Items. For example, talking a motion sensor event and generating events to turn on the light, but only at night on week days when someone is home. That is a lot of items. It often is not clear on what items such logic would be appropriate to implement it on.

  5. I don’t understand what you mean about being spilled across Items and Rules. I see nothing that an OO approach would do to change this. Nor do I see anything sipping you from organizing your rules in a way too avoid this.

  6. I don’t know what you are referring to by “recipes and templates”. Is this something from the experimental rules engine? If so realize the primary purpose of the experimental rules engine IS to promote reuse. If not realize that recipes and templates are not terms or concepts used in the current rules dsl.

  7. The current rules dsl does not support reusable code. Hence the examples and design patterns to help provide known to work solutions to common problems until the experimental engine gets developed.

  8. If you don’t like the rules dsl, you can use the jsr233 binding and code your rules in jython or javascript, I think go is also supported. I don’t know if it works in oh 2 yet.

1 Like

I definitely had you in mind while writing this post since all of your posts about patterns are what got me thinking about this and because I see many of your posts have an academic mind set (I mean that in a good way).

Here’s an example of what motivated my post. It seems that when manually locked or unlocked, Z-Wave locks send an alarm message instead of updating the status of their lock “switch” (I don’t know the proper term).

So now it gets pushed onto me to handle that in my rules. I’d rather model my lock outside of my rules and interpret a “manually unlocked” alarm message as updating the status of the lock switch to unlocked. Maybe that “outside of my rules” place would be event-driven, but it seems like a good place to use OO.

So how would you handle this kind of scenario? I think you’d say that you’d create a proxy for the lock switch and change it’s status to “unlocked” when the manual unlock alarm message comes in. Maybe I just need to get over it, but I’d prefer to see the ability to replace the thing’s actual item with the proxy item and never have to look at the messy actual thing item ever again. If that was possible, then there wouldn’t be the clutter of non-ideal Thing items hanging around and the complexities of the Thing would effectively be hidden/abstracted from the rule design process. This kind of separation of concerns is what led me to my OO thinking.

Do you agree that there is room for improvement related to these kinds of scenarios?

To respond directly to some of your points:

I am referring to the way that rules do not really encourage proper separation of concerns and isolation from change. I have no doubt that rules can be used to do many things, but that’s kind of the problem. Rules and Items do nothing to encourage proper modeling of the problem they do not acknowledge the patterns that we know exist in this space. You are even perhaps implicitly acknowledging this point by posting patterns to stand in place of the lack of structure to the current rules.

It’s kind of backwards from what I’d like to see. I’d like to see the default rules paradigm be tailored and structured to make it easy to separate concerns and to isolate change, with the option to go to a less constrained rules language like the current one if your needs somehow can’t fit that common mold. Today, the user is pushed into the deep end right off the bat, free to hang themselves in complexity if they are not careful or don’t understand certain programming principles. I can’t see that standing up well in the long term. But I can see why the strategy of a project would start with the powerful language and then see what patterns develop to know how to design the eventual tailored, default language. To be clear, I’m not knocking the project, just trying to plant a seed on the idea of making things better.

I’m referring to Kai’s post here.

I probably should have started a new thread for this.

Overall, I think I need to gain more experience with designing complex rules before I go totally second-guessing everything. Then my ideas would be tested and I would at least be able to put a finer point on the problems I perceive.

1 Like

The rules dsl is not procedural. It is event based (think languages like Erlang). There is a reason why OO languages are rarely chosen in similar event driven systems (asterisk, freeswitch, tasker, telecom systems, etc)

Just curious, what is that reason from your perspective? I’m not wanting to be an apologist for event-driven OO but there are many, many event driven programs written with OO (e.g., event-driven user interfaces). Even openHAB itself, which is clearly event-driven, is written using OO techniques.

Given the architecture of oh it is debatable whether OO would be a superior approach.

But, again, the openHAB architecture is itself written using OO techniques. If, by architecture, you mean the item-based approach to integrating Things/Channels with the user interfaces (and rules) then that would be an interesting debate.

I don’t understand what you mean about being spilled across Items and Rules.
I see nothing that an OO approach would do to change this.

Imagine you have some automation functionality that operates on a set of closely-related data and has nontrivial functionality that operates on that data when an event occurs. An OO approach provides encapsulation of the data and the operations on that data. Using inheritance, you could potentially have a basic implementation of something like a motion-driven light/actuator that has extension points (polymorphic methods) to override behavior in subclasses for new types of sensors and/or actuators.

If you don’t like the rules dsl, you can use the jsr233 binding and code your rules in jython or javascript, I think go is also supported. I don’t know if it works in oh 2 yet.

@scurrier03 I’ve been very happy with JSR223 Jython (Python) in OH1. The OH2 JSR223 support is being actively developed and should be ready soon. The rules must still operate on items but I’ve been experimenting with techniques to hide some of that interaction.

@rlkoshak Are you thinking of Groovy rather than Go? I haven’t heard of any JSR223 support for Go.

Yeah, I think he must be thinking of Groovy (or something else). My heart jumped a happy beat at the thought of being able to use Go (my current favorite language) for OpenHAB, but… yeah I don’t think it’ll be available as a JSR223 language anytime soon.

I’m in OH2. I’d be interested to see an example of what this looks like once the OH2 support is complete and ready for the average user’s consideration.

You would still have to write the logic to deal with the inadequacies of the underlying technology. So what does OO really buy you? What benefit is gained by adding yet more complexity to OH overall and adding another place where one can write behavior logic? I just don’t see the benefits. All I see is “I like to solve problems using OO.”

Solving these sorts of inadequacies with the underlying technologies is one of the main purposes of the Rules layer. I really do not see the benefit of having a two layer logic where solving one type of problem requires using one type of coding and other types of logic require a different type of logic.

I guess it comes down to, why do you think solving these sorts of problems are not Rules?

In an ideal world, the lock would report appropriately. But this is home automation, about as far as we can get from ideal.

Boy, you would have hated OH 1.x. Things are so much better abstracted now that we have Things and Channels. Imagine the complexities we faced when ALL we had were Items and everything had to be declared and configured on the Item.

So let’s go back to my point 2 above and elaborate on what I mean by OH being an event based system. In OH we have Items. Items change state based on external stimulus (update from a Channel, 1.x binding, or from a Rule). State changes and updates are events.

Rules get triggered by events, either from Items or Time or system events (e.g. system started). The body of a Rule is to perform some logic in response to the event. Often that logic is to change the state of one or more other Items.

So one can view as defining an event stream. I think this is one of the reasons why systems like NodeRed are so successful because they make this event stream flow much more apparent given their graphical lines and nodes presentation.

Given this, writing a Rule to handle linking the ALARM events t the lock’s Switch seems perfectly natural. In fact, it is probably one line of code (minus the rule boilerplate) and there is no need for a proxy Item.

rule "Lock manually opened"
when
    Item MyLockAlarm received update // don't know this device so don't know if you get a command or just an update
then
    MyLock.postUpdate(ON) // I'm assuming the Lock is represented as a Switch
end

Of course, this is an aberrant case. Chris already has an Issue to correct this in the Binding.

BUT, now that I read more closely what you are really objecting to is the extra Things and Items, not the Rule themselves.

I really don’t see how adding an OO logic layer would help you in this case either. Things are the abstraction layer to the outside world. Each Thing has one or more Channels or control and/or information. This is how devices are abstracted to OH. So even if you had some OO logic on your Lock Thing, you would still need to write the code like the above. You would still need to deal with both Channels.

So is it really worth completely reworking the entire OH architecture to solve a 6 LOC problem? (one for the Item linked to the Alarm Channel and five for the complete Rule). It could even be 1 if you don’t mind run-on lines of code.

Unfortunatelty, every system I have dealt with like OH across multiple domains will have one or more places where an abstraction layer is incapable of completely hiding the complexity of the layer below. In OH2’s case that is the Things layer. It is sooooo much better now but the complexity is still there. But the Things and Items layers are declarative by nature. So the complexity really comes from the proliferation of them, not inherent logic. So personally I’d rather have lots of the Things and Items if I can make all my logic concise and easy to write and read. I think OH 2 is successful in this regard.

“I can program Fortran in any language.” :slight_smile:

Here’s the point I’m trying to make when I say Rules are event based. There are no separations of concerns or isolation from change in an event-based system, at least not in the way you think about from an OO perspective. Instead of Objects, you have Actors. Actors are atomic isolated chunks of logic that process an event in an isolated fashion and which cause change through well defined and constrained interfaces. In a way, an Actor is like an Object that has no state, only methods. Actors are great because they scale really well because they process each event in isolation. They also work great because their ability to modify state is tightly constrained.

In the Rules DLS’s case, the Actors are the Rules. The system state is stored in Items (and sometimes in global vals/vars which are a whole other philosophical discussion; suffice to say global vals and vars are kind of like primitives in Java, incredibly useful but totally antithetical to the purity of the programming paradigm). The interface for changing state is sendCommand and postUpdate.

The language isn’t a wild free for all where anything goes. It is in fact very constrained with very strong isolation. But it is following an Actor based model, not an OO model. I’ll also add that as an Actor based language, the Rule’

I completely agree and that is one of the major accusations against the DSL that I wholly agree with. It is not based on a popular language (I’d never heard of Xtext before using OH), the documentation leaves much to be desired, and even then there have been so many modifications to create the Rules DSL that one has to be careful with the documentation that does exist for Xtext (e.g. there are no classes or arrays in the Rules DSL but there are in Xtext).

I’ve found in my years helping newcomers on this and the old forum that the only users who end up hanging themselves in complexity are the programmers who come to the Rules DSL expecting it to work like an OO or procedural language. The non-programmers seem to do pretty well with it largely because they don’t come to it with any preconceived notion of how it ought to work.

I too struggled with the language for well over a year. My rules were lengthy, complex, brittle and ugly. I felt like the language was fighting me every step of the way. “Come on, I can only pass 7 arguments to a Lambda?!” “Really, there are not arrays?! How am I supposed to save this data structure?” So I started to study the architecture a little more closely and I realized:

  1. The language is best thought of as an Actor based language with the Rules being the Actors.
  2. State really belongs in Items, not global vars and vals
  3. Use Groups to create “data structures” out of your Items

With these realizations, the light bulb went off and I’ve really grown to appreciate how well suited this style of programming is to this domain. As I applied them across my rules I saw a drop from

It’s not perfect. @Spaceman_Spiff will be sure to tell you that JSR233 with Jython runs much faster (does it work in OH 2 yet?). There is no mechanism for reusable libraries. It isn’t a well-known language and is relatively poorly documented. While it will let you code in a more OO style should you choose, realize it won’t address your problems with the proliferation of Things and Items. That layer remains unchanged.

Nor do the developers of OH 2 which is why they are building a replacement for the Rules DSL (more on that below).

Just realize you are not the first to bring this up. There are dozens of similar threads on this and the old Google Groups forum discussing this exact issue. There is a whole binding written to allow users who simply do not want to use the Rules DSL to program using more OO style languages like Jython or JavaScript.

Then you are referring to the Experimental Rules Engine. This is a complete rewrite of the Rules Engine into a new language which is being positioned I think to better support graphically writing rules (a la Scratch and NodeRed) and code reuse. So I could, for example, write a State Engine library and share the library, not just post an article with the code on how to write your own. That is what he means by Recipies and Templates.

I don’t mean to say OO can’t solve the problem nor that it isn’t a good choice in many cases. But given the interface, OH presents to the Rules I’ve found in my experience (having had to code to similar interfaces both ways in the past) that an Actor based approach results in shorter, simpler, easier to reason about and model, and easier to debug code than the OO approaches.

It largely depends on the amount and types of information that needs to be processed through the events. There are relatively few event types and relatively few data types one has to contend with in the Rules DSL. I’ve found in those situations an Actor based approach works better.

So, just to be clear because I do see that some of the things I say do seem like generalizations, I’m primarily referencing event driven systems with relatively few event types and states.

That is mostly what I mean. I’m mainly talking about the Item based approach to integrating events and state to the rules. I think my thoughts would hold equally well under OH 1.x.

The Rules DSL is really like an Actor based language. I don’t think it is strict enough to truly be considered one. But thinking about it as one I find really helps one reason about and write their rules.

Clearly, I’m not one to back away from such a debate. And I’m even willing to have my mind changed. :slight_smile:

By closely-related data, I assume you mean Items?

So I have my closely-related data in the same Group and process events on those Items through the same Rule(s). I have lambdas as extension points to override behaviors (see this). And I don’t have to teach non-programmers the surprisingly difficult to grasp the concept of OO in order for them to start coding rules.

And I still don’t understand the “spilled across Items and Rules”. OO is a different way to slice and dice the problem but I don’t see it actually addressing the problem that you will have multiple implementations to handle multiple different cases.

I am, and that makes much more sense. Even when I wrote that a little voice in the back of my mind said “really, does Go run on the JVM?” but I was tired so didn’t go check.

Sorry

It should be pretty much the same as documented here.

4 Likes

@rlkoshak: I really admire the patience you have explaining stuff to people!
Keep it up - I really like it! :thumbsup: :slight_smile:
The community lives from people like you.

3 Likes

@rlkoshak, I really appreciate the effort you invest in supporting the user community. You are a valuable asset to the project and I have great respect for you. However, this is one topic where we have very different opinions.

The language is best thought of as an Actor based language with the Rules being the Actors.

It seems to me that OH Rules are quite different than Actors in the Actor model of computation. For example:

  • Actors can have private state. Rules can’t.
  • Actors process messages sequentially. Rules don’t (mutexes may be required).
  • Actors can create other Actors. Rules can’t create other Rules.
  • Actors have an address for receiving messages. Rules don’t.
  • Actors can send messages to an actor address. Rules can’t send messages directly to other Rules.

The purposes are different too. The Actor model of computation is addressing asynchrony, concurrency, and fault tolerance. My understanding is that the OH Rule Engine was a replacement for the Drools engine they previously used. Drools is a Rete-based forward-chaining inference engine (production rule system) and is not well-suited for home automation applications. The OH Rule Engine is an improvement in several ways, but I wonder if the automation support would have evolved in a different direction without the Drools influence.

Here’s another way to look at it. Someone can create simple rules in Jython or Javascript as easily as with the Rule DSL (and with well-documented languages, third-party library support, and a variety of IDEs). However, they can do much more with the JSR223 languages. You mentioned several benefits already. Having the option (not requirement) to use OO techniques is just one of the benefits. You could use functional programming or procedural techniques, for example. If someone is interested in abstractions (very useful for modeling real-world objects like devices and services), polymorphism, encapsulation and all the other benefits of OO, they at least have the option to use it. I know you say the limitations of the Rule DSL are its strength for new users, but if they can effectively program the lambda/Group-based workarounds you promote with the DSL, I believe they can write programs in any of the supported JSR223 languages.

State really belongs in Items, not global vars and vals

Items are just a different type of global variable that generates update/changed events.

Use Groups to create “data structures” out of your Items

This is limited and inflexible, but I do understand the issue of “when you only have a hammer…”. You can create a Group of Items for a specific purpose (like a motion/light/presence-controls lighting actuator) and use nested lambda functions to extract information from the items. But what about the second instance? You could create a Group of Groups and another level of lambda functions and the code becomes even more complex. Or one could create a copy of the rule implementation. With an OO approach, you just create another instance of the class (same implementation, different state).

For example, I’ve created a ClockDrivenCommand class. Instance can be constructed with a cron spec or with an absolute time of day (which is parsed and converted into a cron spec in the implementation). Now, my associated “rule” instance declarations look like:

...
ClockDrivenCommand("17:00", CarportLight, ON),
ClockDrivenCommand("23:00", CarportLight, OFF),
...

It would be easy to make this even more terse and have something like:

OnBetween(CarportLight, "17:00", "23:00")

Sure you can write DSL Rules for each of these, but if you have 20 or 30 or more of them it becomes messy. If you want to make a small change to the implementation, then every rule must be edited.

For example, if I want to add additional behavior to only enable some schedules when the house is in “vacation mode”, then I just subclass ClockDrivenCommand and create a VacationCommand class that uses an Item (so it can be controlled from the UI) to determine the vacation mode. All the rest of the functionality is inherited from ClockDrivenCommand.

As another example, I have several IP surveillance cameras. I have created a class that represents a camera and manages all the related Items and external behavior (API calls to take snapshots, turn on/off motion control, parse the network logs to look for suspicious IP addressed, send security notifications, etc.). When I buy a new camera I just create a new instance of that class and I’m done. Almost. Of course, I still have to define the Items separately. However, in OH2 I’ll be able to create the “private” items automatically from JSR223 code. Even for the UI items, I’m going to experiment with generating those automatically and using a site map template to automatically include new aggregates of items related to a specific function.

I do something similar for motion-controlled lights, where the class implements relatively complex functionality (daytime determination, light level detection, stale data detection, etc.) and I just create a new instance when I get a new motion detector and I want to control one or more lights or other on/off actuators.

By closely-related data, I assume you mean Items?

No, I don’t mean that. Items are one way to represent data and data structures. Other languages, like Python have far more powerful support for data structures. With the Rule DSL (AFAIK) you are limited to Items or file-scoped variables. If I have private state, I’d prefer to keep it private (encapsulated), rather than in a global Item. It’s even worse that I have to edit a separate file to “declare” my private data Item (some people call these “virtual items”).

I’m mainly talking about the Item based approach to integrating events and state to the rules. I think my thoughts would hold equally well under OH 1.x.

There are triggers that are not Item based. In OH1 there is the “startup” trigger and time triggers. OH2 will be more flexible and there will be the possibility to define other trigger types that are not Item-based. As examples, there could be a file change trigger, a JMX MBean notification trigger, a process shutdown trigger, or a raw MQTT message trigger that bypasses the Things/Channels/Items. My point is that although Items are a key architectural component of OH, there are other possibilities in some cases.

I too struggled with the language for well over a year. My rules were lengthy, complex, brittle and ugly. I felt like the language was fighting me every step of the way. “Come on, I can only pass 7 arguments to a Lambda?!” “Really, there are not arrays?! How am I supposed to save this data structure?”

Dude, you are one persistent guy! I spent about 30 minutes with the Rule DSL and could immediately see the issues with it. I switched to JSR223 and never looked back. I’ve never felt like the language (Jython) was fighting me.

1 Like

I’m not ignoring you. I’m waiting to reply when I can sit at a real keyboard.

My initial thought is perhaps I’ve been a little too literal when I call the rules DSL an actor based language. I mean it to mean it helps, at least it helps me, to approach rules from that mental framework. But I clearly see that is not what my actual words that I’ve written say.

More discussion to follow…