Design Pattern: Gate Keeper

designpattern
Tags: #<Tag:0x00007f1e5652f550>
(Rich Koshak) #1

Edit: Added an implementation that doesn’t require a lock and is therefore safer to use over all.

Please see Design Pattern: What is a Design Pattern and How Do I Use Them for a desciption of DPs.

image

Problem Statement

There are some technologies that are sensitive to receiving multiple commands too close together. When they occur too close together commands get dropped or other problems may occur. Insteon and 433MHz are two such technologies with this problem.

Concept

Centralize all the communication with these technologies into a single gatekeeper rule and add delays between each command sent to the devices there so no two commands, regardless of their target device, that ever get sent too close together.

Simple Example

This example uses a single String Item which gets commanded with a command line script that gets called using executCommandLine to trigger the device. This may not be possible in all cases so please post questions if you need help dealing with multiple Items to be commanded. The approach would be the same only we would parse the command into an Item name and new state.

Items:

String WirelessController
Switch Outlet_A 

Rules:

import java.util.concurrent.locks.ReentrantLock
var lock = new ReentrantLock

rule "433MHz Controller"
when
    Item WirelessController received command
then
    lock.lock // Ensures only one instance of the Rule can run at a time
    try {
        val results = executeCommandLine(WirelessController.state.toString, 5000)
        logDebug("433", results)
        Thread::sleep(100) // experiment to find the minimum sleep to obtain reliable switching
    }
    catch(Exception e) {
        logError("433", "Error handling 433MHz command: " + e)
    }
    finally {
        lock.unlock
    }
end

rule "Outlet A"
when
    Item Outlet_A received command
then
    if(receivedCommand == ON) WirelessController.sendCommand(“433-send xxxxx 1 1”)
    else WirelessController.sendCommand(“433-send xxxxx 1 0”)
end

Theory of Operation:

There is a script called 433-send which is called with three arguments, a controller ID (I’m basing this off of someone else’s code and do not fully understand the script), a device ID, and the command as ON=1 and OFF=0.

Rather than providing an executeCommandLine in a Rule for each device or binding the Items to the Exec binding, we use a Design Pattern: Proxy Item to represent each device. The Proxy Item triggers a Rule and in the Rule we construct the command to execute to command the device and send that to WirelessController as a command.

The WirelessController command is handled by a Rule. A ReentrantLock prevents more than one instance of the Rule from executing at the same time. This rule then executes the command and sleeps for a tenth of a second before exiting. The lock and the sleep will prevent any two commands to the 433 controller from occuring closer together than 100 msec and therefore avoiding collisions.

Complex Example

We can take advantage of Design Pattern: Associated Items and Design Pattern: Encoding and Accessing Values in Rules to make the above solution a little more generic and flexible.

Note: The code below depends on the the WirelessDevice Items to be persisted.
Items:

Group WirelessDevices
Switch WirelessDevice_xxxxx_1 (WirelessDevices)
Switch WirelessDevice_xxxxx_2 (WirelessDevices)

Rules:

import java.util.concurrent.locks.ReentrantLock
var lock = new ReentrantLock
var lastCommand = now.millis
val commandDelay = 100 // experiment to find the lowest number that works

Rule "A WirelessDevice received a command"
when
    // We can't trigger the rule using WirelessDevices received update because there is no good way to handle the multiple rule triggers
    Item WirelessDevice_xxxxx_1 received command or
    Item WirelessDevice_xxxxx_2 received command
then

    // Get the controller ID and device ID
    val split = triggeringItem.name.split("_")
    val controller = split.get(1)
    val device = split.get(2)
    val command = if(receivedCommand == ON) "1" else "0"

    // Ensures only one instance of the Rule can run at a time
    // We do this after the lines above so the delay below does not interfear with the Rule's ability to 
    // determine which Item triggered the Rule
    lock.lock
    try {
    
        // Sleep if the last command happened too close to now, but only sleep just long enough
        val deltaTime = now.millis - lastCommand // how long since the last call to executeCommandLine
        if(deltaTime <= commandDelay) Thread::sleep(commandDelay-deltaTime)

        val results = executeCommandLine("433-send " + controller + " " + device + " " + command, 5000)
        lastCommand = now.millis

        logDebug("433", results)
    }
    catch(Exception e) {
        logError("433", "Error handling 433MHz command: " + e)
    }
    finally {
        lock.unlock
    }
    
end

Theory of Operation:
A proxy Item is created for each device and the name of the device includes the controller ID and the device ID. All of these proxy Items are members of the WirelessDevices Group.

A Rule gets triggered by any one of these proxy Items receiving a command. We cannot use the Group to trigger the Rule because there is no clear way to manage the fact that the Rule gets triggered multiple times per command.

In the rule we use the lastUpdate hack to identify the Item that triggered the Rule and then parse out the controller and device IDs out of the Item’s name. Thus, adding a new device only requires adding a new Item and adding that Item as a trigger to this Rule.

We then wait to acquire the lock, check to see if we need to sleep or not, and execute the command line using the values parsed out of the Item name and the received command.

Complex Example using Queues

As documented elsewhere, locks can be dangerous to use. So we should make every effort to keep the locked portion of code as fast and error free as possible. A call to executeCommandLine does not meet that criteria.

This example shows how to create a queue of commands that get worked off in a separate Timer thread which makes a better use of the Rule’s threads.

Items:
Same as Simple Example.

Rules:

import java.util.concurrent.ConcurrentLinkedQueue

val Queue<String> commands = new ConcurrentLinkedQueue()
var Timer timer = null
var lastCommand = now.minusSeconds(1).millis

rule "433MHz Controller"
when
    Item WirelessController received command
then
    commands.add(receivedCommand.toString)

    if(timer === null) {
        timer = createTimer(now, [ |
            if(commands.peek !== null) {
                val cmd = commands.poll
                val results = executeCommandLine(cmd, 5000)
                logDebug("433", results)
                lastCommand = now.millis
            }

            val deltaTime = now.millis - lastCommand
            timer.reschedule(now.plusMillis(if(deltaTime<100) 100-deltaTime else 0) // 0 will reschedule the timer to run immediately
        ])
    }
end

rule "Outlet A"
when
    Item Outlet_A received command
then
    if(receivedCommand == ON) WirelessController.sendCommand(“433-send xxxxx 1 1”)
    else WirelessController.sendCommand(“433-send xxxxx 1 0”)
end

Theory of Operation:
This example differs significantly from the two above. At a high level, rather than forcing the Rules to queue up and consume a runtime thread while previous calls to the Rule are sleeping or running we create a queue of the commands and spin up a looping timer that works off the queue.

First in the Rule we add the command to the queue. The queue is thread safe and non-blocking so there is no waiting around for a lock. Next if the looping timer doesn’t exist we create it. This timer runs forever, checking every 100 milliseconds for a new command. If one is on the queue we pull it off the queue and call executeCommandLine. Then we reschedule the timer to run again in the lesser of 100 milliseconds or 100 milliseconds minus the amount of time that it too executeCommandLine to run.

Advantages and Disadvantages

Advantages

  • Lets you solve a technical problem with certain technologies in OH Rules
  • The complex example provides the flexibility to add new devices simply by adding new Items and Rule triggers
  • The queue example does not tie up a Rule runtime thread sleeping

Disadvantages

  • Ideally this should be implemented in the binding for the technology, though that isn’t possible when you are using Exec binding or the like
  • The first two examples can be dangerous because of the use of the lock.

Related Design Patterns

Design Pattern How It’s Used
Design Pattern: Separation of Behaviors Gate Keeper is a specific implemenation of Separation of Behaviors
Design Pattern: Proxy Item The WirelessController and Device Items
Design Pattern: Working with Groups in Rules The lastUpdate hack
Design Pattern: Associated Items Mentioned in the description but not used above
Design Pattern: Encoding and Accessing Values in Rules The naming for the device Items in the complex example
Design Pattern: Looping Timers The implementation of the loop that works off the queue in the queue example
7 Likes

How to wait in a rule?
[SOLVED] Error: Rule '<rulename>': null
Serial protocol which acknowledges by reception via rules
How to setup a FIFO queue to send commands
Unreliable rule triggering (cron) with openhab 2.3
Openhab works slow, CPU of the RPI is high
[SOLVED] Presence with Android after using Iphone
How to send IR commands synchroneously
ReentrantLock - correct use or not?
Openhab 2 Squeezebox Text to speech?
Why have my Rules stopped running? Why Thread::sleep is a bad idea
Timeout notifications
Rules / functions / scripts: best approach
[SOLVED] Error: Rule '<rulename>': null
Rules stop executing after a while
Squeezebox binding: notifications and concurrency issue
Stop timer and reset to null. Structure of rule okay?
Rollershutter Group - Display open shutters
Monoprice 6-zone Audio amp items, sitemap & rules
433 MHz sendCommand problem
Callback / wait for other rule
[SOLVED] Rollershutter group & rule
Locks in Openhab 2.4+ (Rules stopped working)
OpenSprinkler Irrigation and Cascading timers rule
Why have my Rules stopped running? Why Thread::sleep is a bad idea
RfLink binding - Sending to multiple devices unreliable
[SOLVED] Multiple ON and OFF's for crappy 433MHz-devices?
[SOLVED] Multiple ON and OFF's for crappy 433MHz-devices?
What's the meaning of the duty cycle value?
My diy HVAC zoning setup
[SOLVED] Rollershutter group & rule
Multiply Triggered Rule
(Dan) #2

Hadn’t seen that approach to locking rules before - very neat (I’d always used a variable to achieve that indirectly).

0 Likes

(Markus S.) #3

Hi Rich.
Want to use it for my hue lights.

Group:Switch:OR(ON, OFF) all_lights  "Wohnzimmer_Sofa"
Switch Licht_EG_Sofa1_Switch         "Licht Sofa1_Switch"     (all_lights)         {channel="hue:0210::8:color"}
Switch Licht_EG_Sofa2_Switch         "Licht Sofa2_Switch"     (all_lights)         {channel="hue:0210::4:color"}
Switch Licht_EG_Sofa3_Switch         "Licht Sofa3_Switch"     (all_lights)         {channel="hue:0210::5:color"}

Can you give me an example how i can use your rule?
Thank you very much.
Greetings;
Markus

0 Likes

(Rich Koshak) #4

The complex example is all that you need. If you have specific problems I’d love to help but I’m not inclined to write someone else’s rules if they haven’t even attempted writing their own.

0 Likes

(Markus S.) #5

Yes, of course. I understand.
The problem is that i don’t understand the thing with device ID and controller ID.

2018-07-16 02:16:27.150 [WARN ] [lipse.smarthome.io.net.exec.ExecUtil] - Execution failed (Exit value: -559038737. Caused by java.io.IOException: Cannot run program "zigbee-send" (in directory "."): error=2, No such file or directory)
import java.util.concurrent.locks.ReentrantLock
var lock = new ReentrantLock
var lastCommand = now.millis
val commandDelay = 100 // experiment to find the lowest number that works

rule "A WirelessDevice received a command"
when
    // We can't trigger the rule using WirelessDevices received update because there is no good way to handle the multiple rule triggers
    Item Licht_EG_Sofa1_Switch  received command or
    Item Licht_EG_Sofa2_Switch  received command or
    Item Licht_EG_Sofa3_Switch  received command
then

    // Get the controller ID and device ID
    val split = triggeringItem.name.split("_")
    val controller = split.get(1)
    val device = split.get(2)
    val command = if(receivedCommand == ON) "1" else "0"

    // Ensures only one instance of the Rule can run at a time
    // We do this after the lines above so the delay below does not interfear with the Rule's ability to 
    // determine which Item triggered the Rule
    lock.lock
    try {
    
        // Sleep if the last command happened too close to now, but only sleep just long enough
        val deltaTime = now.millis - lastCommand // how long since the last call to executeCommandLine
        if(deltaTime <= commandDelay) Thread::sleep(deltaTime)

        val results = executeCommandLine("zigbee-send " + controller + " " + device + " " + command, 5000)
        lastCommand = now.millis

        logDebug("zigbee", results)
    }
    catch(Exception e) {
        logError("zigbee", "Error handling zigbee command: " + e)
    }
    finally {
        lock.unlock
    }
    
end
0 Likes

(Rich Koshak) #6

That warning is coming from the program you are running (zigbee-send) not the rule. You probably have to provide the full path to the command.

Look at How to solve Exec binding problems

0 Likes

(Markus S.) #7

Oh, now i understand the “executeCommandLine” part. :joy:
Thanks and greetings,
Markus

0 Likes

(Petr Tureček) #8

Shouldn’t be there:
sleep(commandDelay - deltaTime) ?

If the deltaTime since lastCommand is only 20 ms, then we wait only another 20 ms, which is 40 ms total and this does not help us to reach the desired 100 ms delay.

0 Likes

(Rich Koshak) #9

That is correct. We need to wait the full commandDelay time at a minimum.

0 Likes

(Wolfram) #10

Yeah, first post here! Just wondering, I already had such a pattern with a queue in mind as I experience pretty much the same problem with my 433Mhz RFLink bridge.
However, I would prefer this kind of message handling would be realized correctly inside the bindings. Eventually these are the controller of the devices and have to take care of the message flow, not the rules. Why can’t we just state such a behavioral requirement for the bindings, specifically those with such problematic protocols? The delay could be exposed as a configurable parameter then. (I plan to try this way with a fork of the RFLink binding).

0 Likes

(Rich Koshak) #11

All I can say is it’s because the binding developers didn’t implement it that way.

0 Likes

(Wolfram) #12

Well, that’s unfortunate. However, when it comes to the RFLink binding there seem to exist some tiny delay within the serial communication. But it’s not configurable. Maybe worth to expose it to skip the hassle inside the rules…

0 Likes

(Bastian Baumeister) #13

Hello all,
I hope a comment on this pattern is a good place to put my question - if it is not, please let me know. I will open up a new thread then.

My goal

I want to send out notifications via telegram from rules

My approach

  • proxy string TNotificationViaTelegram for the message to be sent
  • any rule may set it, e.g. TNotificationViaTelegram.sendCommand(tMsg)
  • a central rule which reacts to changes in this item:
rule "Send Notification over Telegram"
  //notify via Telegram
  when
    Item TNotificationViaTelegram received update
  then
    if (SNotificationViaTelegram.state != ON) {
      return;
    }

    val String msg = TNotificationViaTelegram.state.toString
    
    sendTelegram("HomeInfo", msg)
  end

Problem

In the Telegram message stream, I sometimes miss a message and instead get the same message twice in quick succession.
I am pretty certain, that this is what is going wrong:

  1. rule A updates TNotificationViaTelegram
  2. rule B updates TNotificationViaTelegram (in quick succession - this e.g. happens by rules which react to presence)
  3. rule Send Notification over Telegram gets triggered by the rule A update, but the actual sending happens, rule B has already changed the content - ergo the rule B message gets sent
  4. rule Send Notification over Telegram gets triggered by the rule B update, the rule B message gets sent

Solution approaches

Note: these are the ones I can think of, I have not tried out any of them yet - still theorizing :wink:

  1. Add a Thread::sleep after every instance of TNotificationViaTelegram.sendCommand(tMsg)

    • Unsexy approach, as it is not centralized and I have to think of it whenever I sent a message
    • I don’t think it will work 100%, as when a certain event (presence) triggers two rules, these two rules might still update TNotificationViaTelegram at nearly the same point of time. This would be less likely to happen, if I use randomized sleep-times, which makes working with this approach even more complicated.
  2. Approach as suggested in the “Simple Example”

    • If I understand correctly, this would not solve my problem (or would it??).
      I believe the approach makes a rule a singleton. But my problem is not the rule running in parallel, but that TNotificationViaTelegram is changed while the rule is running.
  3. Approach as suggested in the “Complex Example using Queues”

    • In essence, a FIFO queue is what would solve my problem. The way the example is set up though, the queue does not span across mutiple rules-files - correct?

So what is your view on this? Can anyone give me some pointers how to get this done?
Maybe there is a simple mechanism which I am use but don’t know or fail to recognize?

By the way: @rlkoshak - I have been using OpenHab for ca. 2 years now and your posts in this forum have been tremendously helpful for my understanding. I am amazed how often you find the time to take part in a discussion and help out with your knowledge and views.
Thank you very much for that!

0 Likes

(Rich Koshak) #14

This is probably not a Gate Keeper problem, though it might be solvable with it, though I don’t think that’s necessary.

Use commands instead of updates to trigger your Rule. Command makes sense here as you are activating an external service, not just updating OH’s internal state. Then, when you use received command, you can use the receivedCommand implicit variable which will always be what was commanded regardless of timing.

Option won’t do anything because that Thread::sleep won’t affect the other Rules that call the sendCommand. It will just delay when that one instance of the Rule will exit.

Option two and three won’t really solve your problem because your actual problem, if I understand correctly, is that when two updates occur too close together one of them get’s overwritten before it can be processed by the Rule. Using a lock or a timer or the queue will only add delays between calls to telegram so you don’t bombard the service with too many calls all at the same time. But the data problem you are seeing would still be a problem.

You might still want to use a Gate Keeper to spread out the messages sent to Telegram, but you need to use commands in order to keep the states from overwriting each other.

You are most welcome! I learn by helping and contributing to the forum is my way of giving back to the community.

0 Likes

(Bastian Baumeister) #15

@rlkoshak: so you offer real-time problem solving services - cool :wink: ! In my job, I have access to some paid “premium support services” which are nowhere as good (or quick) as you are :smiley:

Thanks - I fully understand what you say. I even remember reading about the receivedCommand variable at some point. But I never used it and forgot about it.
I’ll go that way - I am very confident it’ll work!

0 Likes

(Rich Koshak) #16

You just happened to catch me when I was online. Though when I’m tagged I get an email and can often reply from my phone, which was the case here.

0 Likes

(rud) #17

Edit: @5iver solved my problem and I have corrected the code so that one can copy&paste items and code if one have a similar use case. @rlkoshak thank you for this great design pattern!

Hi @rlkoshak , this is the case with my sonos speakers, when there are multiple notifications to be played.
That’s why I try to use the complex example of your design pattern to manage playing notifications.
But there are errors in my openhab.log that I cannot solve and I couldn’t find a solution in the forum.

My items in file GateKeeperExamples.items

Group GrpMessages
Switch CreateMessage    "CreateMessage [%s]"    (GrpMessages)
String TextQueued       "TextQueued [%s]"       (GrpMessages)
String TextOutput       "TextOutput [%s]"       (GrpMessages)

My code in file GateKeeperExamples.rules

// IMPORTS **************************************************************************  
    import java.util.concurrent.ConcurrentLinkedQueue

// GLOBAL VARIABLES ************************************************************
    val MyQueue = new ConcurrentLinkedQueue()    // Edited this line. The code causing the errror was: val Queue<String> MyQueue = new ConcurrentLinkedQueue()
    var Timer timer = null
    var lastCommand = now.minusSeconds(1).millis

    var MyMessageCounter = 0

// RULES ****************************************************************************

rule "rGateKeeperExample_CreateMessage"
 when
    Item CreateMessage received command
 then
    MyMessageCounter = MyMessageCounter + 1     // MyMessageCounter see global variables section
    val MyMessage = "Message no. " + MyMessageCounter.toString + " , date = " + now.toString
    TextQueued.sendCommand( MyMessage )
    logInfo("openhab", "New Message = " + MyMessage )
end // EndOfRule rGateKeeperExample_CreateMessage

rule "rGateKeeperExample_MessageOutput"
 when
    Item TextQueued received command
 then
    val String          MyLogFile = "MySeparateLogFile"   // if 'MySeparateLogFile' does not exist, logging goes into openhab.log (to create a separate logfile see https://www.openhab.org/docs/administration/logging.html#logging-into-separate-file)
    val DateTime        MyLogDate = now  
    val StringBuilder   MyLogMessage = new StringBuilder
    MyLogMessage.append( "\n" + "\n" + " ----------   rGateKeeperExample_MessageOutput v001   ----------" + "\n" + "\n" )
    MyLogMessage.append( "    MyLogDate = " + MyLogDate.toString + "\n" + "\n" ) // ("yyyy-MM-dd'T'HH:mm:ss.SSSZ")

    MyQueue.add(receivedCommand.toString)  // https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html#add(E)
    MyLogMessage.append( "    TextQueued = " + receivedCommand.toString + "\n" ) 

    if(timer === null) {
        timer = createTimer(now, [ |
            if(MyQueue.peek !== null) {    // https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html#peek()
                val TmpText = MyQueue.poll     // https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html#poll()
                TextOutput.sendCommand( TmpText.toString )
                // say(TextOutput, "voicerss:deDE", "sonos:PLAY1:SonosLiving", new PercentType(25))
                MyLogMessage.append( "       TextOutput = " + TmpText.toString + "\n" ) 
                lastCommand = now.millis
            }
            timer.reschedule(now.plusMillis(60000))  // output of next message after 60s
        ] )
    } 
    logInfo( MyLogFile, MyLogMessage.toString )
end // EndOfRule rGateKeeperExample_MessageOutput

My openhab.log

2019-03-28 20:58:18.129 [INFO ] [el.core.internal.ModelRepositoryImpl] - Refreshing model 'GateKeeperExample.rules'
2019-03-28 21:25:55.014 [INFO ] [el.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model 'GateKeeperExample.rules', using it anyway:
The field Tmp_GateKeeperExampleRules.MyQueue refers to the missing type Object
The field Tmp_GateKeeperExampleRules.MyQueue refers to the missing type Object
The field Tmp_GateKeeperExampleRules.MyQueue refers to the missing type Object
2019-03-28 21:25:55.209 [INFO ] [el.core.internal.ModelRepositoryImpl] - Refreshing model 'GateKeeperExample.rules'
2019-03-28 21:26:26.329 [INFO ] [lipse.smarthome.model.script.openhab] - New Message = Message no. 1 , date = 2019-03-28T21:26:26.320+01:00
2019-03-28 21:26:26.359 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule 'rGateKeeperExample_MessageOutput': 'add' is not a member of 'Object'; line 38, column 5, length 37
2019-03-28 21:26:28.888 [INFO ] [lipse.smarthome.model.script.openhab] - New Message = Message no. 2 , date = 2019-03-28T21:26:28.875+01:00
2019-03-28 21:26:28.915 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule 'rGateKeeperExample_MessageOutput': 'add' is not a member of 'Object'; line 38, column 5, length 37
2019-03-28 21:26:29.725 [INFO ] [lipse.smarthome.model.script.openhab] - New Message = Message no. 3 , date = 2019-03-28T21:26:29.719+01:00
2019-03-28 21:26:29.757 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule 'rGateKeeperExample_MessageOutput': 'add' is not a member of 'Object'; line 38, column 5, length 37

Problem
I think the info in openhab.log
The field Tmp_GateKeeperExampleRules.MyQueue refers to the missing type Object
indicates, that there is something wrong with the declaration of MyQueue in this line of my code
val Queue<String> MyQueue = new ConcurrentLinkedQueue()

Every time the rule is triggered I get this error
Rule 'rGateKeeperExample_MessageOutput': 'add' is not a member of 'Object'; line 38, column 5, length 37
which points to line 38 with this code MyQueue.add(receivedCommand.toString)

I couldn’t find a solution yet, so every help is appreciated.

My system is openhabian installed on raspberry pi:

  • Release = Raspbian GNU/Linux 9 (stretch)*
  • Kernel = Linux 4.14.79-v7+*
  • Platform = Raspberry Pi 3 Model B Rev 1.2*
  • openHAB 2.4.0-1 (Release Build)*
0 Likes

(Scott Rushworth) #18

In DSL rules, only be as specific as you need to be.You shouldn’t get the error if you use this…

val MyQueue = new ConcurrentLinkedQueue()
1 Like

(rud) #19

Many thanks, that is the solution.
Small cause, big effect!

0 Likes