Proposal: ActivityManager for Consistent Handling of Motion/Triggers and Their Timers

Setup, Configuration and Use Scripts & Rules
1

I would love to get feedback from the group whether the ActivityManager I am proposing here does not yet exist and solves the problem of activity management.

@rlkoshak perfectly summarized the design pattern to handle basic motion in Design Pattern: Motion Sensor Timer with the central idea of having an activity item that is ON while there is motion and OFF if there is no motion. This idea extends nicely to a large set of triggers well beyond motion. Rich and many follow up contributions demonstrate how rules are powerful to implement broad functionality.
However, in my own rules implementation I found the code hard to debug as so many variations and timers are involved. I therefore implemented a proof of concept ActivityManager on top of the JSR223 engine that can be tested through comprehensive unit tests. The unit tests allow me to modify the manager quickly and with confidence.

The ActivityManager consistently manages the ON/OFF state of activity items according to the activity item’s configuration. The ON state can be triggered by item state changes such as motion, door contact open or manually (e.g.via a light switch for manual on/auto-off). The OFF state is set based on expired time (optionally reset if re-triggered), a set time of day or a manual trigger. As any item state change can trigger an activity item we can use an activity item that turns off as the trigger for another activity item (e.g. if motion expires in my kitchen I don’t turn off all lights, but instead trigger a second activity item that leaves some standby lights on). All activity items can be overridden manually by other items - optionally for a limited amount of time (e.g. if a user turns off a motion controlled light then motion sensing will be disabled for a certain amount of time). The ActivityManager can create a report of its state (in particular reporting expiration timers and override timers).

With the help of the ActivityManager I can condense complex activity scenarios with diverse underlying triggers into simple and consistent activity items that are straight forward to reason about. In simple cases the activity item itself can represent a physical item such as a light (light is on when the activity item is on and off when the activity item is off). But more often than not you will want a more sophisticated mechanism to manage resulting actions based on time of day, luminance and other things. For clear separation of duty ActivityManager does NOT offer those services. Instead use any complementary solution such as Area Triggers and Actions, which groups activity items, leverages Mode (Time of Day) and supports actions defined by mode. Note that ActivityManager does optionally support trigger configuration by mode (time of day), further aligning with “Area Triggers and Actions”. Additionally ActivityManager’s support for manual overrides will eliminate the need for using a dedicated intensity level such as 99 to override automation in “Area Triggers and Actions”. Finally note that “Area Triggers and Actions” offers basic timer support for delayed actions. These timers operate on the actions, but not on the underlying items and thus can’t participate in the underlying activity items and groups. If this functionality is used to enhance activity items then ActivityManager offers a cleaner solution so there is a slight overlap of functionality.

Obviously these ideas are not new and I have found various bits and pieces implemented, but I haven’t found a comprehensive implementation of the above functionality. I noticed @rlkoshak’s PR for a TimerManager and checked if I could implement ActivityManager on top of it, but didn’t see a fit as TimerManager deals with adding timers across items.
Would love to hear your feedback. Should I clean up my prototype and add it as a Community Project under the openHAB helper libraries for JSR223?

1 Like
2

This topic doesn’t fit the category of Solutions, since it does not even contain a tutorial, so I’ve moved it to Scripts & Rules.

What does the ActivityManager do that Area Triggers and Actions doesn’t?

FYI, there are other options for disabling the automations.

3

@5iver, thanks for the guidance on the post category.
Area Triggers and Actions doesn’t manage items for you so that they are on when there is activity (whatever activity means for your application) and off otherwise. Area Trigger and Actions instead assumes you already have such activity items and then let’s you group them and act on state changes of the activity items in a highly configurable way (e.g. by time of day, luminance or any other custom factors). In some cases managing an activity item is straight forward. e.g. if you have a motion sensor that can be configured out of the box to be on while triggered and off after expiration then you have an activity item out of the box. Or a contact sensor that sets an item to open/on while open and off otherwise also gives you an activity item out of the box. But for all the more complex scenarios I described the ActionManager can manage the activity item as needed.
Area Triggers and Actions offers timers - not for the activity items, but for the resulting actions. For example if you have a motion sensor that only triggers motion but doesn’t support expiration you can start an action with the trigger and delay the turn off effectively simulating expired motion. However, you don’t have an actual activity item that represents the motion sensor with expiration. Thus you can’t include your logic in groups, can’t use the activity item to trigger other activity items, can’t get override support on the activity item and so on. ActionManager implements all functionality around the concept of activity items allowing for consistent activity functionality across and making all of it then available to Area Triggers and Actions for action handling.

The specific challenge with overriding an activity is that the system needs to track if an ON/OFF event from an override item was user initiated (e.g. user turned light switch off) or was system initiated (light turned off because motion expired). There is no deterministic general way to do this as the response to a system sent command carries no id matching it to the originating command. The ActionManager handles this by tallying what it sends and what it receives and thus detecting additional override events. And since overrides are commonly meant to be temporary (“I am turning this light off. I want it to stay off for the next 30 minutes. Then activity sensing should resume”…) ActionManager optionally makes overrides temporary. In other words overriding automation is more than just specifying a toggle. That’s why Area Triggers and Actions chose to use a designated intensity value that the actions then ignore to simulate overrides.

Hope this helps a bit.

4

TimerManager along with pretty much all of my submissions to the Helper Libraries and the Design Patterns are always going to be at a lower lever than what is proposed here. I focus on building blocks, not complete solutions. So I’m not surprised TimerManager doesn’t do what you are after. All it does it what’s in the name, manage separate Timers for individual Items, a common activity Rules developers need to achieve. What those Timers are for and what happens when they expire is outside the scope.

Without the code it’s really hard to provide any sort of feedback. I honestly couldn’t explain back to you what you are trying to accomplish and how it is distinct from Scott’s Mode library.

In general, I find that a lot of frameworks like these end up making it easier to reason about and use for the original developer but often do not make it easier for someone else to come along and use. It feels easier to reason about for you because you fully understand it and have already built up a model for how it works in your mind. New users will need to go through the effort of learning how it works from scratch and building up that mental model from scratch. Often that effort is a challenging as just coding it again themselves.

Whether that’s the case here or not I couldn’t say without looking at the code and figuring out how to use it.

See Design Pattern: Manual Trigger Detection for three approaches. Probably the most generic/fool proof approach is the No. 3, Proxy Items.

1 Like
5

Thank you for the detailed feedback Rich.
Fully understood that your TimerManager is meant to be a building block. I didn’t expect it to handle the ActivityManager requirements - I just considered leveraging it to build ActivityManager on top of it. However, from what I have seen so far it doesn’t seem to be a good fit for that.
Your comment is really interesting that frameworks often have a steep learning curve and it might be faster to just re-implement them. That definitely tells me that I need to do a better job in explaining the ideas :smile: - one thing I can think of is to center documentation around concrete examples rather than abstract ideas. Often it is indeed much more fun to do something myself rather than understanding somebody else’s work (“invented here…”). But I am finding myself more and more dealing with details in my solutions that I would much rather have somebody else solve. With this post I wanted to make sure I am not missing anything very obvious. Makes sense as a next step to publish actual code for more detailed feedback.
Thank you for the pointer on manual trigger detection. I will review and see how I can leverage that.

6

ughhh… right?

yes… agreed and just coding it again themselves is also more fun!

7

I have done something similar in Jython. I use it in my home and cabin. Works great for setting up behaviors without having to hard code.

8

@mjcumming, thank you very much for your pointer. OccupancyManager is a very close match to what I am looking for. I really like the leverage of groups that enables many use cases that I hadn’t yet had in mind. In particular it enables very elegantly the chaining of occupancy items as I need in my kitchen example (where an area item becoming vacant triggers another item to become occupied). It’s interesting how we all have a different view on very similar problems given our individual use cases - as reflected in our manager names: OccupancyManager vs. ActivityManager. Your focus has obviously been occupancy with strong support for group functionality, whereas my focus has been on activities with strong support for activity management - both can be addressed with the same framework.

I think OccupancyManager nicely extends into the features that are missing for my use cases such as override support (with optional timers), mode/time of day support for variable timer durations based on mode, occupancy expiration based on mode changes rather than predetermined durations etc. (and of course much of that might already be supported and I just haven’t understood it yet).
My use cases need more flexible functionality for actions than provided by OccupancyManager, but since OccupancyManager manages areas it seamlessly integrates with Area Triggers and Actions for advanced action handling (in particular action handling by time of day). The combination of the two frameworks is very powerful and addresses a wide range of use cases.

Would you be open to me trying to add my missing features to OccupancyManager? I think it would be a straightforward extension (though the devil is of course always in the detail). Would probably be most effective for me to propose a PR to make it concrete. Goal would be to add OccupancyManager as a community project (if you don’t already have that in flight) side by side with Area Triggers and Actions.

Last but not least: Where do I find the source code?

9

Hi,

I cleaned up the code and added to a github repository

Let me know what I can do to help get it up and running

Mike

10

Thank you Michael. I’ll give it a try and come back with a proposal. Though it will probably take me a couple of months as a number of other things are going on in my non-OpenHAB life…

11

Let me know when you have sometime. Looking forward to seeing what you think.

12

Hi @mjcumming, I got my POC up and running - wohoo! Based on Occupancy Manager and Area Actions & Triggers I have my bedroom light responding to the bedroom motion sensor with an expiration timer, turning on only when bedroom luminescence is low and functioning as a nightlight during the night. Occupancy Manager manages the occupancy aspects (motion trigger sets occupancy, timer turns it off). Area Actions & Triggers manages the actions for the light (choosing the intensity based on mode/time of day and the luminescence sensor). It works very well - no surprises! Your detailed documentation proved extremely helpful.

If you are still up for it I would like to move on to adding support to Occupancy Manager for the additional use cases I had proposed above. An important use case for me is to support overrides to occupancy directly from a light switch. If a light is turned on as a result of an occupancy event and a user then goes to the light switch and turns the light off I would like occupancy for the occupancy item to turn off (along with any running timers). I believe with the existing implementation if a user turned off a light it would obviously go off because of the user action, but the underlying occupancy item would remain on and the timer would keep running. If the user then manually turned the light on it would be turned on at level 100 and thus excluded from future automation, specifically excluded from effects of the still running timer. So there is a notion of override for the light, but it is not in sync with the occupancy item itself. Am I understanding the current implementation correctly? Since I rely on the occupancy item for resulting actions I need to have overrides reflected in the occupancy item.

So, I would be looking at listening to state changes of the light (as that is all we receive when a user switches a light from the light switch; there is no command being sent). In response we can adjust the occupancy item. Only complication is that we need to filter out light state changes that are the result from Occupancy Manager’s own commands sent to the light.
If I didn’t miss anything in the current implementation and if the above makes sense to you I suggest I go ahead and implement the above and send you a zip file to see what you think.

Beyond the above I have to warn you that I have a fair amount of additional proposals. If you have the patience for it I think it would be ideal if we could review each proposal (over time of course) briefly like I did with the state change handling above to make sure I am not on the wrong track. Then - once we agree on design - I can follow it up with a PR for the implementation. To make that efficient it would probably be a good idea to develop a test harness first so we can quickly and easily absorb change. I did that for my own POC framework that I had started with in JavaScript and it paid off very quickly. I should be able to port that over.

End goal would then be to get Occupancy Manager approved as a community project.

Well, I hope I didn’t scare you off. Please let me know what you think! Thanks again for putting Occupancy Manager out there.

13

Great, glad you have it working. I think with some work and vetting out others use case’s this would be a good community project.

This project started as a Rules DSL implementation but didn’t work well due to its limitations. I start working with Jython at the same time @5iver did all the great work it.

For your light example… the OM will see changes from local control and from OH.Currently, any changes (local or from OH) in a lights ON state will reset the occupancy timer for that area to occupied. Ie. a Switch changed to ON or a Dimmer changed to any value other than 0.

Modifying that behavior such that a light being turned to OFF to set the area Vacant will be quick and easy to add - I’ll look at that today.

I think the project could be generic enough that it would be easily usable by many. Very open to making changes. It will require users to make some core commitments to how they structure their implementation. On of the good and bad things about OH is there are many ways to solve a problem. But this actually adds complexity. For instance, in a script if your want to determine if its night or daylight out, there is no easy and no standardized way to do that.

14

I had to make a couple of small changes. You can now set an area to vacant using an Item (switch/dimmer) OFF event using the item metadata as shown below.

Switch			SF_MasterBathroomShowerExhaustFan_Switch									"Master  Shower Fan"					<fan>		(gSF_MasterBathroomShower, gBathroomExhaustFans,gOccupancyItem)	["Switchable","ExhaustFan"]	{alexa="disable",channel="mqtt:homie300:Queen:switch-14edd51:switch#switch", OccupancyEvent = "OnOff" [EndOccupiedTime = 0]}
15

@mjcumming, thank you for the fast response! Your change to set an area to vacant upon expiration works well.

I have now implemented my first production use case (wohoo!): A light with a straight forward motion trigger plus the light switch itself is a trigger: If the light switch is turned on the light turns on without a timer and the light gets locked for a configurable time (so the motion sensor won’t turn it off again for that duration). If the light switch is turned off similarly the light goes off and the light gets locked for a configurable time (so the motion sensor won’t turn it on again for that duration).

To implement that use case I expanded on the lock functionality including the lock timer you already had implemented. As I worked on it I realized the desired lock functionality was really just another occupancy area (where occupancy means locked). So I ended up extending Area to Area_Lock and then it was very easy to optionally allow adding an Area_Lock. The light switch is then simply a trigger for the Area_Lock leveraging all the existing configuration including timer configuration, providing the rich occupancy area functionality for locks as well.
I left the existing lock implementation unchanged to not break backwards compatibility - both can work side by side. However, I think we can now remove all existing lock code. With one exception: for locks you today support the notion of levels, which occupancy areas (and thus Area_Lock) don’t yet support . I suggest we lift the notion of levels into occupancy area itself - then it can be used for both occupancy and locks. I would envision creating a default Area_Lock if none is configured to be backwards compatible with current lock implementation and for ease of use.
I think Event_Contact_Door is no longer needed either. It now becomes simply an EventOnOff handler on the lock area.
What do you think? I am happy to implement that swap or would you like to give it a try?

A trigger item such as my light switch that both triggers an occupancy lock and is the action recipient of occupancy creates 2 challenges:

  1. If the lock gets set before the action (race condition) the action gets locked out. To avoid this I added a new ModifyBehavior = “OverrideLock”. A trigger configured with this behavior is allowed to override a lock.
  2. A light switch operation comes into the system simply as a state update. Any commands to the light switch, including commands as a result from occupancy actions, eventually also arrive as light switch state updates. The latter must not trigger locking the occupancy area. Since we can’t correlate light switch commands and the resulting state update deterministically I can only infer. This requires listening to incoming commands and correlating them to incoming state updates. For that we have to add a new rule. For now I am manually adding the rule. Eventually we can automatically generate this code just like you generate other rules.

Please note to implement the event handler that turns on but doesn’t trigger a timer I added support for BeginOccupiedTime < 0. That is just a short term hack to get the functionality and is not very intuitive. Similarly I think we could find a more intuitive way to express the recent change you made to EndOccupiedTime. But that is purely a usability refinement and we can review that at a later time - functionally everything is working just well.

I’ll copy my POC files below to give you an idea of how my use case looks in detail. I’ll submit the PR with the supporting code changes next. Looking forward to your feedback!

.items

String Mode "Mode: [%s]"
Group gArea_Trigger "Area Triggers"
Group gArea_Action "Area Actions"

Switch POCSensorMotion "POC Motion [%s]" (gOccupancyItem,gPOC_Trigger) {OccupancyEvent = "OnOff"}
Switch POCLight "POC Light (occupancy trigger) [%s]" (gOccupancyItem,gPOC_Trigger,OS_POC_Action) {OccupancyEvent = "OnOff" [BeginOccupiedTime = "-1", EndOccupiedTime = 0, ModifyBehavior = "OverrideLock"], channel="adorne:switch:home:atrium:power"}
Switch POCLightLock "POC Light (lock trigger) [%s]" (gOccupancyItem,gPOCLock) {OccupancyEvent = "OnOff" [BeginOccupiedTime = 1, EndOccupiedTime = 1], channel="adorne:switch:home:atrium:power"}

Group gPOC_Trigger "POC Trigger" {OccupancySettings = "" [ Time= 1]}
Switch OS_POC_Trigger "POC Occupancy State [%s]" (gOccupancyState,gPOC_Trigger,gArea_Trigger)
Group gPOCLock "POC Lock" (gPOC_Trigger,gOccupancyLock) {OccupancySettings = ""}
Switch OS_POCLock "POC Lock State [%s]" (gOccupancyState,gPOCLock)
Group OS_POC_Action "POC Action" (gArea_Action)

.sitemap

sitemap motion_poc label="Motion POC"
{
	Switch item=POCSensorMotion mappings=["ON"="On", "OFF"="Off"]
	Text item=POCLight
	Text item=OS_POC_Trigger
	Text item=OS_POCLock
	Text item=Mode
}

manual rule added

"""  
record commands
""" 

@rule ("record commands")
@when ("Item POCLight received command")

def POCLightReceivedCommand (event):
    try:
        # Record commands for our triggers for matching with the resulting state updates
        am.record_command("POCLight",event.itemCommand)
        am.record_command("POCLightLock",event.itemCommand)

    except:
        log.error (traceback.format_exc())
16

@5iver, as I am building out my above first production use case I ran into two enhancements for Area Triggers and Actions:

  1. Anti-flapping support for lux item
    As I have my lux item connected to an actual light sensor I typically see it change up and down around the threshold a number of times before it gets dark enough to be consistently below the threshold. This results in lights going on and off respectively. Anti-flapping solutions have been widely discussed (e.g. Design Pattern: Motion Sensor Timer). Since I found my lux item to flap over an extended amount of time (tens of minutes) I am suggesting a buffer based protection rather than a timer based one. A configurable threshold buffer (default 0) is added on either side of the threshold - only a breach of the threshold plus/minus the buffer triggers a lux event. If reading variations stay within 2x the buffer there is no flapping. A timer could be a nice complementary future addition (would allow to narrow the buffer as the timer runs).
  2. Manual override
    You implemented manual overrides by special casing state 99/100 and leaving a light in that state unchanged, which is very elegant due to its simplicity. However, I would like the user to be able to optionally pick any dimmer level and honor that as the override. This (along with any generic override use case) can simply be achieved by introducing an override item (similar to the lux item) - if set it overrides the usual brightness calculation. I also made the 99/100 override configurable with default to on for backwards compatibility when the 99/100 override is not the ideal solution (I think off would actually be a more intuitive default, where the user gets no override by default and can chose to turn on overrides using the 99/100 approach or an explicit override item. But that would of course break backwards compatibility - would you be open to that change?)

I will submit a PR for these two proposals shortly (I put them into a single PR hoping that would make it easier for you or do you prefer one PR per topic?). Note that I refactored light_action in the process to remove some duplicated code and added graceful handling when items are NULL (which I ran into at various times).
As a side note: I will also submit a PR to supply override items in OccupancyManager to work in conjunction with the Area Triggers and Actions overrides.
Let me know what you think!

17

Yes, always put a single piece of functionality into a single PR. But submit an issue describing the feature before doing work on it, or you may be wasting someone’s time. Discussions here in the forum get lost.

I already have this added as metadata… upper_buffer and lower_buffer. I made my own lux sensors and have them reporting only after a 10% change, so I don’t use this any more. I really like when the lights respond to clouds rolling by.

I recently outlined another method for manual override in the Area Triggers topic. I don’t see a need to complicate things and anyone can implement their own custom actions.

18

Got it.

I don’t see this in the code. I am assuming it is something you have implemented, but not yet committed. And I guess that is your very point - that without an issue I won’t find out. I’ll go ahead and open issues as appropriate.

1 Like
19

Hi @mjcumming, same then goes for additional OccupancyManager proposals? Should I retroactively create an issue for the Lock Area proposal/PR above? Have you had a chance to look at it yet?

20

Hi Mark,

Sorry for the delay’s - finally have some time to focus on this! Just starting to dig in to your changes. Quick question, you have modified the code to allow for a child area to have > 1 parent area. This breaks the fundamental hierarchy of rooms. Wondering why you need to use case. I think it will work but not sure this is a good idea. Thoughts?

Mike