Bayesian Group Suite - Activation Rule [3.2.0;3.4.9)

logo

This rule is one part of a bundle designed to allow the creation, configuration, and utilization of Bayesian probability calculations for some item states using the MainUI. The three parts are:

Credit: This is an OH3-ready implementation of @cweitkamp’s awesome Bayesian aggregation design pattern. So, much of the credit goes to him and you will want to read through his post for a more detailed explanation of the underlying concepts and very clear description and example of probability selection.

In short, Bayesian aggregation is a method of determining a binary state (e.g., on/off) from a combination of assumed likelihoods of related states. Often, for complex situations such as room presence, no absolute combination of factors accurately predicts even something as simple as a binary state. Bayesian probability logic takes into account the idea that some combination of factors is a good enough indication of a conglomerate state but has the advantage that not all factors have to contribute every time and some factors contribute more than others.

There are a few changes and updates from Christoph’s original Implementation.

  1. All the metadata for one group is collected in the group item instead of each individual item. This permits one item to contribute to more than one Bayesian group if required.
  2. There are many more tests for each likelihood than simple equality, such as < and > or test for list membership. See the Configuration Reference section below for a full description of the available options.
  3. Also included is a limited ability for a likeihood probability to be dynamically defined. For example, when the lights in a room are on, this could be considered a good indicator that someone is present in the room. However, my children tend to leave lights on when they exit rooms so if the lights in a room have just been turned on that is a strong indicator that someone is in the room, but if the lights have been on for more than an hour, that is a much weaker indicator of room occupancy (see Decay in the configuration reference for more information).
  4. There are two additional components available in the marketplace, a widget and a rule that works with the widget to help you manage and configure the Bayesian groups (see links above).

Installation and Use

After downloading this rule from the marketplace you only need to create one instance of the rule. The single rule template parameter that you need to input is the name of one Bayesian sensor group. This group item must have the following characteristics:

  • Bayesian Sensor non-semantic tag
  • Properly configured BayesianSensor metadata
  • Member items for each configured Bayesian probability

You can create one of these Bayesian Group items manually or use the widget and widget rule (linked above).

In addition to the group item you will need a proxy item of the appropriate type to receive the command after the probability has been calculated (see Configuration Reference below).

If you have more than one Bayesian sensor group, then you can add each group to the triggers of the same rule using the had a member change trigger.

Sensor Configuration

All of the configuration for a single Bayesian group sensor is maintained in the BayesianSensor metadata namespace of the group item. It is recommended that you use the additional two components of this suite (the widget and the rule for the widget) to make management and configuration of the sensors easier. This will allow the automatic creation of the group items with a complete metadata template, shortcut access to the metadata editor, shortcut access to the group item editor, and auditing of the metadata to ensure proper configuration.

Configuration Reference

Main group configuration options

Each of the following options is set in the metadata config:

Config option Required Explanation
proxy yes The name of the item that will receive commands based on the result of the probability calculation
threshold yes Numerical value between 0 and 1 that defines the boundary between the sensor being probably true and probably false
trueState no The command to be sent to the proxy item when the calculated probability is greater than or equal to the threshold (default = ON)
falseState no The command to be sent to the proxy item when the calculated probability is less than the threshold (default = OFF)
prior no The initial likelihood that the sensor is in a true state (default = 0.5 )
posterior no The name of a item (type: Number:dimensionless) in which to store the calculated posterior probability of the sensor
sensors yes Map of the individual items to be checked to compute the probability with the name of each item as a key (see Observation item configuration options below)

Audit

If you have used the widget and widget rule to check the metadata configuration then an audit property will also be found in the config metadata. You never have to add this property yourself it will be automatically generated. If the configuration contains no errors or warnings then the property will indicate that the configuration passed, otherwise it will show the errors and warnings. You can delete this key and its values if you wish without impacting the function of the Bayesian group, but the widget will not be able to show the results of the most recent audit if you do.

Observation item configuration options

For each item in the sensors map the following options are available:

Sensors option Required Explanation
testState yes The state relevant for the check of whether the item represents a true or false observation: value type depends on testType
pTrue yes Numerical value from 0 to 1 representing the probability that the main condition is true if this observation is true
pFalse yes Numerical value from 0 to 1 representing the probability that the main condition is true if this observation is false
testType no Short code specifying the kind of test performed on the item state (default test is basic equality) see table below for full list of options
decay no Map of options detailing time-based decrease in pTrue value, see table below for full list of map values

If testType is not specified then a basic test of equality between the item state and testState is performed and the observation is true if the equality is true. Other tests are defined using the testType as follows:

testType value testState value Test performed
gt numerical value true if item state is greater than testState
lt numerical value true if item state is less then testState
bt two-element array of numerical values true if item state is greater than first array element and less than second array element
neq any valid state true if item state is not equal to testState
list array of any length and element type true if item state is equal to any element in testState

The decay property causes the probability if true value (pTrue) of an observation to decrease over time from some reference time. Usually this will be the last time the item was changed but could be from some absolute time of day or other time point. This requires that there be an additional Type:DateTime Item that holds the timestamp of interest (the timestamp profile being the most obvious way of creating this).

Decay configuration Required Explanation
decayStep yes Numerical value between 0 and 1 to reduce the pTrue value with each timeStep
timestamp no Item name of the DateTime item that holds timestamp value (default is sensor item name with ‘_TS’ appended to the end)
timeStep no Number of minutes between each decrement of the pTrue value (default = 5 minutes)
decayMin no Numerical value between 0 and 1 representing the minimum value that pTrue can be decreased to (default = 0.6)
Examples

Here is an example of a properly configured BayesianSensor metadata with some annotations to help understand.

value: " "
config:
  proxy: Occupancy_Kitchen
  trueState: ON
  falseState: OFF
  sensors:
    Occupancy_Kitchen_Detection: #Item that indicates if the camera that sees kitchen area has detected a person
      pFalse: 0.2 #The camera also sees a few other areas such as the backyard through the back door
      pTrue: 0.9  #If a person has been detected by the camera it is very likely that there is a person in the kitchen
      testState: ON
    Speaker_Kitchen_Playing: #Switch indicating if sonos speaker is playing
      pFalse: 0.3 #Sometimes kitchen speaker is linked to other speakers in the house
      pTrue: 0.75 #Usually if the speaker in the kitchen is playing, someone is in the kitchen
      testState: ON
    Switch_BreakfastLight_OnOff: #Light over the table in the adjoining breakfast space
      pFalse: 0.4
      pTrue: 0.9
      testState: ON
      decay: #Kids have the tendency to leave the light on so over time this decreases the probability that  this light correlates with someone in the kitchen area
        timestamp: Switch_BreakfastLight_OnOff_TS #Name of the DateTime item that holds the last changed timestamp for the light switch
        timeStep: 5 #Decrease every five minutes
        decayStep: 0.05 #Decrease probability by 5%
        decayMin: 0.4 #If the light has been on long enough it hits a minimum equal to `pFalse` which means this observation has no impact on the final probability
    Switch_KitchenLight_OnOff:
      pFalse: 0.3
      pTrue: 0.9
      testState: ON
      decay:
        timestamp: Switch_KitchenLight_OnOff_TS
        timeStep: 5
        decayStep: 0.05
        decayMin: 0.4
    AutomationTime: #My systems time of day item
      testType: list #Check if the item state is equal to any of the elements in the given array
      pFalse: 0.1
      pTrue: 0.5
      testState: #Array of times of day we are likely to be in the kitchen (rarely is someone in the kitchen after bedtime or before the start of the morning
        - MORNING
        - AFTERNOON
        - EVENING
  prior: 0.3 #Baseline probability that someone is in the kitchen at any one time (my wife likes to cook a lot so someone is in the kitchen often)
  audit: #This metadata has been checked by the widget rule
    passed: true #The metadata is properly formatted
  posterior: Occupancy_Kitchen_Posterior #Each time a new probability is calculated it is stored in this item
  threshold: 0.95 #A high level of certainty (95% certain) is required to set the kitchen occupancy to ON

This next example is for a switch that triggers my time of day item to change over from WAKE_UP to MORNING. Over the years this has proven to be a very difficult task for the system to work out because of so many different possibilities between me, my wife, kids, weekdays and weekends, but this Bayesian group sensor is exceptionally accurate, even with only simple equality tests for all the different sensor observations.

value: " "
config:
  proxy: AutomationTime_Morning_Start
  sensors:
    AutomationTime:
      pFalse: 0.3
      pTrue: 0.99
      testState: WAKE_UP
    Phone_JustinsCell_ChargingState:
      pFalse: 0.5
      pTrue: 0.75
      testState: OFF
    Sensor_MasterBath_Motion:
      pFalse: 0.5
      pTrue: 0.9
      testState: ON
    Sensor_MasterCloset_Motion:
      pFalse: 0.5
      pTrue: 0.8
      testState: ON
    Switch_MasterBedroomFan_OnOff:
      pFalse: 0.5
      pTrue: 0.9
      testState: OFF
  prior: 0.5
  audit:  #This metadata has been checked by the widget rule
    warnings: #Two warnings have cropped up with this configuration for values that I have not specified and so are falling back to the default values instead.
      number: 2
      list:
        "0": trueState not set - default ON will be used
        "1": falseState not set - default OFF will be used
  posterior: AutomationTime_Morning_Start_Posterior
  threshold: 0.9

Language:

  • JSscripting ECMAScript 2021+

Dependencies:

  • JSscripting Add-on

Changelog

Version 0.2

  • Fixed template formatting issue

Version 0.1

  • initial release

Resources

https://raw.githubusercontent.com/JustinGeorgi/openhab-public/main/rules/bayesian_sensor_calculation.yaml

1 Like

Hello,
Thank you in advance for the work made, seems amazing.

I’m struggling a bit to set it up. I think my following question will help also other people.

Step 1)
Installed JSscripting Add-on
Installed all 3 app from marketplace

Step 2) Because I want to set up “manually” I add a Group in Model
containing two Switch point test items as following images

Step 3) I create a rule to calculate Bayesian probability as follow

and here I’m stucked…

I thing in step 4) I need to set the configuration parameters of the Bayesian group and items but I’m not able to find the metadata…
here what I see if “metadata” label of group is clicked…
seems strange to me do not see the category you mention in the main post…

Is all right the steps untill now ?

what is the correct way to proceed in configuration?

thank you

Custom metadata namespaces do not show up in the list (this is a much asked for feature…).

You have to press the Enter Custom Namespace button and manually enter BayesianSensor.

Hi all, I’ve been further tinkering with this bayesian sensor thing, and found some very good explanations of how to implement it on Homeassistant forum. There is a spreadsheet here very useful to model/test bayesian sensors behaviour.

Looking at it, I see that if we want both the ON and OFF state of an item to contribute information to the sensor state, we need to be able to have that item appear twice in the metadata. It could also be useful to be able to use an item multiple time, in case we want to implement different probabilties depending on a number item.

So i have edited the Activation function to allow this :

/*
This Script provides functions for the sensor control system.
*/

//Initialize
var runtime = require('@runtime');
var logger = Java.type('org.slf4j.LoggerFactory').getLogger('org.openhab.rule.bayesianSensor');
var ZonedDateTime = Java.type("java.time.ZonedDateTime");
var Duration = Java.type("java.time.Duration");
logger.info('Bayesian Sensor - Recalculating')

//Access MetadataRegistry
var FrameworkUtil = Java.type('org.osgi.framework.FrameworkUtil');
this.ScriptHandler = Java.type("org.openhab.core.automation.module.script.rulesupport.shared.ScriptedHandler");
var _bundle = FrameworkUtil.getBundle(ScriptHandler.class);
var bundle_context = _bundle.getBundleContext();
var MetadataRegistry_Ref = bundle_context.getServiceReference('org.openhab.core.items.MetadataRegistry');
var MetadataRegistry = bundle_context.getService(MetadataRegistry_Ref);
var MetadataKey = Java.type('org.openhab.core.items.MetadataKey');

//Bayesian Probability Calculation
function updatePrior (updPrior, updPTrue, updPFalse, updItem) {
    var numerator = updPTrue * updPrior;
    var denominator = numerator + updPFalse * (1 - updPrior);
    logger.info(updItem + ': ' + updPrior.toString() + ' -> ' + (numerator / denominator).toString());
    return numerator / denominator;
}

//Find All Bayesian Sensor Groups for Event Item
var parentGroups = items.getItem(event.itemName).groupNames;
var sensorGroups = parentGroups.filter(x => items.getItem(x).tags.includes('Bayesian Sensor'));

//Read BayesianSensor MetaData and Iterate Through Sensors for Each Impacted Sensor Group
for (var sensorGroup of sensorGroups) {
  var metaList = MetadataRegistry.get(new MetadataKey('BayesianSensor',sensorGroup));
  if (!metaList || !metaList.configuration) {
    logger.info(['Bayesian metadata not configured for ',sensorGroup].join(''));
  } else {
    var sensorConfig = metaList['configuration'];
    var sensorList = sensorConfig['sensors'];
    var prior = sensorConfig.prior || 0.5;
      for (var sensor in sensorList) {
        var checkItem = sensorList[sensor].item;
        
        if (sensorList[sensor].decay) {
          var now = ZonedDateTime.now();
          var changedTime = items.getItem(sensorList[sensor].decay.timestamp || checkItem + '_TS').rawState.getZonedDateTime();
          var decayTime = Duration.between(changedTime,now).toMinutes();
          var numSteps = Math.floor(decayTime / (sensorList[sensor].decay.timeStep || 5));
          var probTrue = Math.max((sensorList[sensor].decay.decayMin || .6),sensorList[sensor].pTrue - (numSteps * sensorList[sensor].decay.decayStep));
        } else {
          var probTrue = sensorList[sensor].pTrue;
        }
        switch (sensorList[sensor].testType) {
          case 'gt':
            if (items.getItem(checkItem).state > sensorList[sensor].testState) {
              prior = updatePrior(prior, probTrue,sensorList[checkItem].pFalse,checkItem);
            }
            break;
          case 'lt':
            if (items.getItem(checkItem).state < sensorList[sensor].testState) {
              prior = updatePrior(prior, probTrue,sensorList[sensor].pFalse,checkItem);
            }
            break;
          case 'bt':
            if ((sensorList[sensor].testState[0] > items.getItem(checkItem).state) && (items.getItem(checkItem).state > sensorList[sensor].testState[1])) {
              prior = updatePrior(prior, probTrue,sensorList[sensor].pFalse,checkItem);
            }
            break;
          case 'neq':
            if (items.getItem(checkItem).state.toString() != sensorList[sensor].testState) {
              prior = updatePrior(prior, probTrue,sensorList[sensor].pFalse,checkItem);
            }
            break;
          case 'list':
            if (sensorList[sensor].testState.contains(items.getItem(checkItem).state)) {
              prior = updatePrior(prior, probTrue,sensorList[sensor].pFalse,checkItem);
            }
            break;
          default:
            if (items.getItem(checkItem).state.toString() == sensorList[sensor].testState) {
              prior = updatePrior(prior, probTrue,sensorList[sensor].pFalse,checkItem);
            }
            break;
        }
      }
    if (sensorConfig.posterior) runtime.events.postUpdate(sensorConfig.posterior,prior.toString());
    runtime.events.sendCommand(sensorConfig.proxy,(prior >= sensorConfig.threshold) ? (sensorConfig.trueState || 'ON') : (sensorConfig.falseState || 'OFF'));
    logger.info([sensorGroup + ' new probability of ',prior, (prior >= sensorConfig.threshold) ? ' is over ' : 'is under ', sensorConfig.threshold].join(''));

  }
}

And now metadata should look like this :

value: " "
config:
  proxy: CuisinePresence_bayesianProxy
  trueState: ON
  falseState: OFF
  sensors:
    - item: NoiseLevel
      testState: 50
      testType: gt
      pTrue: 0.7
      pFalse: 0.2
    - item: Salle_mouvement
      testState: ON
      pTrue: 0.8
      pFalse: 0.2
    - item: Salle_mouvement
      testState: OFF
      pTrue: 0.01
      pFalse: 0.5

This however breaks the Modify rule but my Javascript is a bit limited

It will also break the widget. Technically you have converted sensors from an object to an array. This isn’t a huge leap (as you demonstrate, not many changes are required), but it does impact how the individual elements are accessed.

From a mathematical standpoint, however, this is just not necessary. There is no difference between an item contributing information when ON and OFF and just properly configuring your prior value while using only one item state.

Hello,
thanks for your help also, I made it work as a charm,
now I’m playing with settings.

Custom Metadata is triky to see because are not listed on item or group metadata list, but I can reach easelly with the widget and that is great.

^^ Thank you guys