Design Pattern: Working with Groups in Rules

Edit: Updates for OH 4, inclusion of JS Scripting and Blockly examples, removed the doors example as it’s superfluous.

Please see Design Pattern: What is a Design Pattern and How Do I Use Them for how to read and use DPs.

Problem Statement

One powerful way to consolidate code in rules is to use array/list manipulation operations on the members of a Group. For example, filter down to just those members that are ON, get a list of the labels of those Items that are NULL, etc.

Concept

This DP is not a standard DP in that it provides a single concrete template to follow to solve a specific problem. Instead this DP is more of a grand exploration of some of the things one can do with Groups. See the bottom for a list of some related DPs that utilize some of the techniques discussed here in a more concrete way.

This DP presents a series of examples for how to access and manipulate the members of Groups which is a common requirement in many use cases.

Important GroupItem Methods

Within Rules, a Group is represented using a GroupItem. The GroupItem has a method named members which provides access to all the Items that are members of that Group as a Set. In some languages, that Java Set is converted to a native type (e.g. a JavaScript Array). Some languages also has a method to get the members of sub Groups.

Blockly

Unfortunately Blockly does not support list streaming operations like other languages do so most operations involve looping through the lists and creating a new list. Any of the JS Scripting examples below can be uses in an inline script block where necessary. This section will show the blocks to work with the members of a Group.

  • image: Get the members of a DoorStatus

  • image: Loop through all members of DoorStatus and execute <code> on each.

  • : Puts a list of all the members of MyGroup that have the state ‘ON’ into the variable filteredList.

  • image: Puts the first member of the DoorStatus whose state is ‘ON’ into the variable found. (Note the “in list X find first” block cannot do this as it only works with whole Object matching, not matching one member of an Object)

  • : Sorts the list of Items based on the Item’s numeric state (the sharp eyed among you may recognize the bubble sort algorithm). Note this is an in place sort and does not create a new list. (Note the existing sort block only works on numbers and strings, not Objects)


  • : Sets slice to a list containing the first six members of MyGroup.

  • : Sets mapped to a list containing a string for each Item that is the “Item’s label is state” (e.g. “MyItem is ON”)

  • : Sets mapped to a list containing the Item’s states.

  • image: Sums up the states of all members of MyGroup and stores it in the variable sum.

Combining some of the above, let’s build a String showing all the doors that are open:

image

JS Scripting (aka GraalVM JavaScript)

  • items.MyGroup.members: get a list of all the direct members of the Group, GroupItems that are also members will be included but not their members.

  • items.MyGroup.descendents: get a list of all members of the Group and all members of the Groups that are members of this Group. The GroupItems are not included.

  • items.MyGroup.forEach( item => { <code> }) : Loop through all members of MyGroup and execute <code> on each.

  • items.MyGroup.members.filter( item => item.state == 'ON' ): returns a list of all the members of MyGroup that have the state ON

  • items.MyGroup.members.find( item => item,.state == 'ON' ): Returns the first member of the Group whose state is ‘ON’.

  • items.MyGroup.members.sort(a, b => { if(a.state > b.state) return 1; else if(a.state < b.state) return -1; else return 0; }: Return the members sorted by their state as a number

  • items.MyGroup.members.sort(a, b => a.numeriucState - b.numericState): a more concise way to sort by numericState but which works with numbers only

  • items.MyGroup.members.sort(a, b => a.quantityState.minus(b).float): a concise way to sort by Quantity states

  • items.MyGroup.members.slice(0, 4): Returns an array containing the first five members of MyGroup.

  • items.MyGroup.members.map(item => item.label + ' is ' + item.state): Returns an array containing a string for each Item that is the “Item’s label is state” (e.g. “MyItem is ON”)

  • items.MyGroup.members.map(item => item.state): Returns an array containing the Item’s states.

  • item.MyGroup.members.map.(item => item.numericState).reduce(result, currVal => result + currValue, 0): Sums up the states of all members of MyGroup.

Map and Reduce are great ways to create summaries on the members of the Group. For example, to build a String showing all the doors that are open:

var msg = item.MyGroup.members
              .filter( door => door.state == 'OPEN' ) // Get the OPEN doors
              .map( door => door.name ) // Get the door names
              .reduce( result, doorName => result + ', ' + doorName, '' ); // Build a comma separated string of the door names

// Using join instead of reduce
var msg = item.MyGroup.members
              .filter( door => door.state == 'OPEN' ) // Get the OPEN doors
              .map( door => door.name ) // Get the door names
              .join( ', ' ); // Build a comma separated string of the door names

Nashorn JavaScript

Nashorn uses the raw Java Classes. See Stream (Java SE 17 & JDK 17)

JSR223 Python

  • ir.getItem("MyGroup").members: get a list of all the members of the Group, GroupItems that are also members are of the list.

  • ir.getItem("MyGroup").allMembers: get a list of all members of the Group and members of subgroups; the subgroup GroupItems are not included, only their members.

  • for i in ir.getItem("MyGroup").members: <code>: Loop through all members of MyGroup and execute <code> on each.

  • filter(lambda item: item.state == ON, ir.getItem("MyGroup").members): returns a list of all the members of MyGroup that have the state ON

  • [item for item in ir.getItem("MyGroup").members if item.state == ON]: alternative to filter using Python list comprehension

  • filter(lambda item: item.state == ON, ir.getItem("MyGroup").members)[0]: Returns the first member of the Group that meets the condition defined in the lambda.

  • [ item for item in ir.getItem("MyGroup").members if item.state == ON][0]: Returns the first member of the Group that meets the condition defined in the lambda using Python list comprehension.

  • sorted(item for item in ir.getItem("MyGroup").members , key = lambda item: item.state): Returns a list containing the members of MyGroup sorted by their state.

  • ir.getItem("MyGroup").members[0:5]: Returns a list containing the first five members of MyGroup.

  • map(lambda item: item.state, ir.getItem("MyGoup")): Returns a list of the states of all the Items that are a member of MyGroup. The map lambda can be more involved (e.g. format a String for building a summary message).

  • [item.state for item in ir.getItem("MyGroup")]: Returns a list of the states of the Items in MyGroup using Python list comprehension.

  • reduce(lambda: result, value: result.add(x), map(lambda item: item.state, ir.getItem("MyGroup").members)): Generates a single sum of all the states of all the Items that are members of MyGroup.

Map and Reduce are great ways to create summaries on the members of the Group. For example, to build a String showing all the doors that are open:

msg = reduce(lambda: msg, name: "{}, {}".format(msg, name), # build the string
             map(lambda: door.name,                         # extract the Item name
             filter(lambda door: door.state == OPEN,        # get the OPEN Items
             ir.getItem("MyGroup").members)))               #  get the door Items from the Group

# Using list comprehensions
msg = reduce(lambda: msg, name: "{}, {}".format(msg, name),
             [door.name for door in ir.getItem("MyGroup").members if door.state == OPEN])

Rules DSL

  • MyGroup.members: get a Set of all the members of the Group, GroupItems that are also members are part of the set

  • MyGroup.allMembers: get a Set of all the members of the Group and all members of subgroups; the subgroup GroupItems are not included, only their members

  • MyGroup.members.forEach[ i | <code> ]: Execute <code> on each member of MyGroup, e.g. MyGroup.members.forEach[ i | i.postUpdate(CLOSED) ]

  • MyGroup.members.filter[ i | <condition> ]: Return a Set of all the members that meet the provided <condition>

  • MyGroup.members.findFirst [ i | <confition> ]: Returns the first member that matches the provided <condition>

  • MyGroup.members.sortBy[ <method> ]: Return a List of the members sorted by the value returned by <method> (e.g. using name will sort the Items alphabetically by Item name, lastUpdate by the last time they received an update which requires persistence)

  • MyGroup.members.take(i): where i is an integer, returns a set containing the first i Items in members

  • MyGroup.members.map[ <method> ]: return a Set containing the result of calling <method> on each member, for example MyGroup.members.map[ name ] returns a Set of Strings containing the names of all the Items

  • MyGroup.members.reduce[ <result>, <value> | <result> <operation> <value> ]: returns the reduction of all the values in the Set by the <operation>, for example MyGroup.members.reduce[ sum, v | sum + v ]will return the sum of all members of MyGroup

A note on the use of ?: a number of examples found on the wiki and on this forum will use a ? between the Group name and .members as in MyGroup?.members. This means that if MyGroup.members === null ignore the line. Put another way, it will not generate an error if MyGroup has no members. I do not recommend using this syntax as it can hide some types of errors.

A note on ( ) verses [ ]. The Rules DSL has a concept of a lambda. A lambda is a function that you can treat as if it were like any other var or val. Lambdas are usually defined by surrounding the function in [ ] (note there must be a space after the [ and before the ]). However, the Rules DSL allows parens to be used instead in some circumstances. I prefer to stick to [ ] and will do so in the examples below. Do not be confused however if you ever see parens instead as that is also allowed. For example MyGroup.filter[ i|i.state==OPEN ] and MyUpdate.filter(i|i.state==OPEN) are both valid.

A note on var and vals. In the lambdas passed to these functions, only vals can be accessed. Unfortunately, that means it is impossible, for example, to calculate the sum of all the NumberItems in a Group using forEach. That is where map and reduce come in. To get the sum of NumberItems one can use:

MyGroup.members.map[ state as Number ].reduce[ sum, v | sum + v ]

To build a String containing all the names of Items that are OPEN there are two approaches: use map/reduce or use a val StringBuilder.

// StringBuilder
val StringBuilder sb = new StringBuilder
MyGroup.members.filter[ i | i.state == OPEN].forEach[i | sb.append(", " + i.name) ]

// Map/Reduce
val str = MyGroup.members.filter[ i | i.state == OPEN ].map[ name ].reduce[ s, name | s + ", " + name ]

The StringBuild approach works because we can declare it as a val but still can call methods on it to change it.

Related Design Patterns

Design Pattern How It’s Used
[Deprecated] Design Pattern: Time Of Day For determining when it is NIGHT or BED time
Design Pattern: Associated Items For an explanation on how to pull out the Timer and DateTime Items associated with a given Contact
Design Pattern: Group Based Persistence For setting up persistence
Design Pattern: Sensor Aggregation For detecting presence
Design Pattern: Expire Binding Based Timers The Timers used
Design Pattern: Human Readable Names in Messages Converting Item names to a more friendly name for messages and alerts
Design Pattern: Cascading Timers Using Groups to create Timers that trigger other Timers
Design Pattern: Group Based Persistence Using Groups to specify how Items are persisted
[Deprecated] Design Patterns: Generic Is Alive Using Groups to monitor whether a sensor or device is alive or hasn’t reported in awhile

Edit: Updated to use triggeringItem and Member of Rule trigger instead of the persistence hack.

82 Likes

Thank you @rlkoshak so very much. As you almost always do in your posts you have managed to address several of my questions at once and answered a few more that I wasn’t even ready to formulate yet.

4 Likes

Another very useful and well written post @rlkoshak - well done!

I recently just added a similar set of rules for my external doors/windows but went down the lambda route.

Would be happy to share here if anyone is interested, but don’t want to derail this thread otherwise.

Keep up these awesome posts, so much benefit to all!

4 Likes

I do not think this would be a bad place to show it. It is good to have alternative examples. If you do post it in another thread, at least link it back here so future readers can find it easily.

I used to use lambdas for this sort of thing everywhere, and still do in a couple of places. But a few months back I had a revelation as I was editing a file. In a lot of cases the only reason I needed the lambda was to pass the calling Item and one or two “associated items” to the lambda. If I could collapse everything down to one rule I don’t need the lambda any longer nor do I need all the one liner rules to call the lambda.

It can’t always be done cleanly but when it can I think it results in clearer code. Some cases where it can’t is if the events can occur closer together than the initial Thread::sleep or if you have more interesting things to do for each Item than just call the same lambda with different arguments. For example, if I wanted to do something different for the Garage Door Openers than for the rest of the doors I would probably go down the lambda route.

Ok - here is my lambda;

val Map<String, Timer> externalDoorTimers = newHashMap

val org.eclipse.xtext.xbase.lib.Functions$Function3 checkExternalDoor = [
    SwitchItem doorItem,
    Number timerMins,
    Map<String, Timer> timers |
        var itemName = doorItem.name
        var displayName = ""
        var index = 0
        for (String part : itemName.lowerCase.split("_")) {
            if (index > 0)
                displayName = displayName + part + " "
            index = index + 1
        }        
            
        // if there is already a timer, cancel it
        timers.get(itemName)?.cancel

        if (doorItem.state == OPEN) {
            // notify the event (and play a barking dog if no one home)
            if (GP_Presence_Secure.state == OFF) {
                VT_Notify_Alert.postUpdate("The " + displayName + "opened and no one is home!")
                callScript("play_dog_barking")
            } else {
                VT_Notify_Trace.postUpdate("The " + displayName + "opened")
            }

            // create a timer to announce an alert if we are still OPEN after 'n' minutes
            timers.put(itemName, createTimer(now.plusMinutes(timerMins)) [|
                VT_Announce_Alert.postUpdate("The " + displayName + "is still open!")
                timers.remove(itemName)
            ])
        } else {
            VT_Notify_Trace.postUpdate("The " + displayName + "closed")
        }
]

This allows each door/window to decide how long to wait before notifying and even allows this time to vary based on other state (see the back door rule below).

Then for each door/window I have a simple rule;

rule "Front door"
when
    Item GF_Front_Door changed
then
    checkExternalDoor.apply(GF_Front_Door, 10, externalDoorTimers)
end

rule "Back door"
when
    Item GF_Back_Door changed
then
    // long *open* timer if during the day (since we often leave the back door open)
    if (VT_Day_Outdoor.state == OFF) {
        checkExternalDoor.apply(GF_Back_Door, 10, externalDoorTimers)
    } else {
        checkExternalDoor.apply(GF_Back_Door, 360, externalDoorTimers)
    }
end

This relies on a sensible and consistent naming scheme but I like this method since I only need the door item, i.e. no timer or datetime items (I don’t time these events but I do this in a similar lambda for appliances which stores the turn-on dttm in another internal dictionary).

Definitely a place for both approaches however.

4 Likes

A couple of ideas, many of which I just had in the past couple of week of rebuilding my environment:

  • You can eliminate externalDoorTimers and all the associated Timer management logic if you use the Expire binding and Associated Items, though you can’t vary the timer period at runtime like you are now which is really nice

  • You can eliminate the String parsing to get the displayName by creating a .map file that maps Item names to display names and call transform("MAP", "doors.map", doorItem.name)

The dog barking is a nice touch. :slight_smile: It might make it worth setting up audio in my HA.

That is an excellent use of ? as well. I never thought of using it like that (sometimes the obvious escapes me). Now that I see it, that is clearly why ? exists. If I had any Timers left I’d be running to add ? all over the place. I’ve pretty much moved everything over to Expire for now though. I find the resultant code to be a little cleaner and I hate bookkeeping.

I’m really glad you shared. Its given me some ideas.

Yes I have been keeping an eye on the expire binding - and I can see a multitude of places I could use it throughtout my code, I just haven’t gotten around to implementing it yet.

And I really like that idea of using a map for item names - that is very handy. I went through a massive task of renaming my entire config last year to make generating these sorts of rules easier. But there are always exceptions and edge cases, so having something like a lookup map solves all those issues.

Very useful thread indeed!

Hello @rlkoshak!

Great tutorial! I see you are using Group:DateTime:MAX group item to get date and time of the last door event. I have tried it to get date and time of the last air quality index update, but it doesn’t work. I have created following items:

Group:DateTime:MAX Weather_AirQuality_Average_Index_DateTime "Date and time [%1$td.%1$tm.%1$tY %1$tH:%1$tM]" <clock>

DateTime Weather_AirQuality_Bjelave_Index_DateTime "Date and time [%1$td.%1$tm.%1$tY %1$tH:%1$tM]" <clock> (Weather_AirQuality_Average_Index_DateTime) {channel="airquality:aqi:bjelave:observationTime"}
DateTime Weather_AirQuality_Ilidza_Index_DateTime "Date and time [%1$td.%1$tm.%1$tY %1$tH:%1$tM]" <clock> (Weather_AirQuality_Average_Index_DateTime) {channel="airquality:aqi:ilidza:observationTime"}
DateTime Weather_AirQuality_Otoka_Index_DateTime "Date and time [%1$td.%1$tm.%1$tY %1$tH:%1$tM]" <clock> (Weather_AirQuality_Average_Index_DateTime) {channel="airquality:aqi:otoka:observationTime"}

All separate items have date and time of the last update, but the group item just shows -.-.- -:- . I’ve waited for some of the DateTime items to change, but it still doesn’t work. What am I doing wrong here?

Best regards,
Davor

Hmmmm. I’m not sure. Now that I’ve gone back to my sitemap to check I’m finding that Item to be undefined as well (that is what the -'s mean).

What version of OH are you using?

Hello!

I’m using OH2 nightly. I’ve solved it using a rule. Not the best solution, but it works. Thank you for your help.

Best regards,
Davor

Hi Rich,
I’m looking at your example code of groups and trying to incorporate something similar but i’m not sure its possible (although it looks likeit could be).

Basically instead of storing a value in a Item, is it possible to create a variable and store it there instead when it loops though a group of items.

For example here’s what i’m trying to accomplish:

Item:
vGarageLight (gSoffitLights)
vHouseLight (gSoffitLights)

  1. Door Opens, check gSoffitLights (gSoffitLights.members.filter[i|i.state != 0].forEach [ i | ???]) group for all lights currently ON and the dimmer value.

  2. Create a variable called “item.name + __restore” and store the dimmer value of the light in that. Eg. vGrageLight_restore, vHouseLight_restore

How would code it to store the variable in a variable, is that possible?

First I’ll say I usually recommend storing such things in Items because you then can take advantage of restoreOnStartup which will allow your Rules to behave more consistently during OH startup or .rules files reloading. In those cases all your vars get wiped out and reinitialized.

To answer your specific question:

Whenever you see [ ] in the code above you are seeing a lambda. These lambdas have a limitation that they can only reference and use vals (i.e. constants). So, you have a problem because once you set a val you cannot change it later which I think would eliminate the possibility to achieve what you are trying to achieve.

So you have three approaches you can use to solve this problem (in my preferred order):

  1. Set up persistence on the Lights (not MapDB because we need the previous value) and get the last Dimmer values from persistence using item.previousState(true).state as PercentType in your 2. restore loop.

  2. Add new Items with findable names (like above) and store the current Dimmer values in those Items. The references to Items are constant so you can access and postUpdate to them from inside the lambdas.

  3. Create a global val Map<String, PercentType> prevDimmerVals = newHashMap and in your 1. forEach put the current Dimmer value into prevDimmerVals using the i.name as the key. In 2. you can get the old Dimmer value back out to apply to the lights. This works because the Map is a val but provides methods that lets us change it from inside the lambda.

1 Like

As I understand it, there is no guarantee that this gives the item that triggered the rule, because
a) 100ms may be to short to wait for the update during high load
b) 100ms may be to long to wait for the update in case several items are updated at the same time.

So, is there any better way to do it?

Anyhow, please don’t call such a dirty workaround a design pattern… it’s more an anti-pattern…

2 Likes

Of course, simply have individual rules triggered on individual Items, so that you will have access to exactly what the trigger was.
See Ben Jones lambda example above for one way to re-use common code across many rules.

  1. One must work within the limitations the Rules DSL imposes if one is to work within the Rules DSL. If you trigger a rule by a group or by multiple items, this is the ONLY way to get the triggering item.

  2. If you want guaranteed don’t use this approach. If you want something that works 99% of the time in the home automation context this is an excellent way to simplify your errors and reduce lines of code.

  3. That is one example line among eight demonstrated ways to use groups in rules. Would you there the baby it with the bath water?

So, if the only way to achieve something that works 99% of the time in the given problem domain and which makes up a small fraction of the article is an antipattern, that is your option and one I wholly disagree with.

Hello Rich,

Thanks for the very detailed rule guidance.

I have a problem with the “lastUpdate” property.

I have persistence (mapdb, and influxdb) both are set to “everyChange”

When i try to use the mapdb within the rule for a group of two items, only one is getting updating correctly with the lastUpdate property, while the other is somehow shows the last update is from 2 weeks ago which is wrong.

When i try with influx, it shows both are updated exactly at the same time (which is also wrong).

Any idea what can cause this behaviour?

The MapDB problem looks like for some reason the Item is not being persisted to MapDB. Double check your .persist file and make sure the Item is configured to be saved.

Add a sleep before calling lastUpdate to make sure persitence has a chance to catch up. Sometimes a rule can run too fast and execute before the value has been saved.

Add a chart for InfluxDB to see what is actually in the database. Or do some manual queries.

Compare the values in the DB with what you are seeing in events.log. The binding or one of your rules might be updating these Items behind the scenes.

Keep in mind that when using everyChange, lastUpdate will only change the timestamp if the new value posted to the Item is a change. If it is just an update with the same value the timestamp will not be updated. If this is the case I recommend using everyUpdate instead of everyChange.

Hi @rlkoshak - great pattern, thank you.
Can i use:

rule "A Door Sensor Changed"
when
    Item gDoorSensors changed 
then

to shorten my code, instead of list all items in group?

You can use any trigger on a Group that you can on an Item. However, it may not work like you need it to.

In this case, it would not work as you want because gDoorSensors will only change when all the doors are closed and one opens or all the doors become closed.

Imagine this scenario.

Step Doors State gSensors.state Rule Triggers?
1 All closed CLOSED No
2 One door opens OPEN Yes
3 Second door opens OPEN No
4 Third door opens OPEN No
5 Third door closes OPEN No
6 Second door closes OPEN No
7 First door closes CLOSED Yes

As you can see, the Rule would trigger when the first door opens but will not trigger again until ALL the doors are closed again. You would miss all the openings and closings of all the other doors after the first one opens. That is almost certainly not the behavior you want.

The only way to trigger a Rule for every change to a Group’s members is to use received update, though then your Rule has to deal with the fact that the Rule will be triggered multiple times for one change to one of its members. Often this doesn’t matter but in cases like the rule above, it does matter and it was far easier to write the rule with one trigger per door and know the Rule will only trigger once per change than it was to manage filtering, reentrant locks, and all the other code necessary to deal with the fact that the Rule would trigger multiple times per change to a door, and that those multiple Rules will be running at the same time.

1 Like

Is any right way to use something like:

Group:Number:SUM  gDoorSensors "[%d]"

to manage 10+ items?