Design Pattern: Using Item Metadata as an Alternative to Several DPs

Please see Design Pattern: What is a Design Pattern and How Do I Use Them for details on what a DP is and how to use them.

Problem Statement

There will be times when one has data that is associated with a specific Item. This data could be configuration data (e.g. the brightness for a light at certain times of day). It might be a flag indicating that some activity has occurred on that Item or needs to be done on that Item. It might be a different version of the Item’s name for use in logs or alert messages.

Concept

image
Relatively recently (OH 2.4, maybe earlier) OH has added the ability to set metadata on an Item. The metadata can be set statically or dynamically defined. When it is statically defined you either need to apply it to the Item using the REST API or in your .items file. Dynamic definition can occur from Rules.

Sorry Rules DSL folks, there is no way to access and modify metadata from Rules.

NOTE: if metadata is defined in .items file, all other metadata will be removed upon a reload of the .items file.

Metadata consists of three main parts:

  • Namespace: the root category, can contain multiple Values
  • Value: Single value for the namespace
  • Key/value pairs: an array of keys and their values

Here is an example.

Switch vCerberos_SensorReporter_Online "Cerberos sensorReporter [MAP(admin.map):%s]"
    <network> (gSensorStatus, gResetExpire, sensorReporters)
    { expire="2m,command=OFF",
      Static="meta"[name="cerberos sensorReporter", timeout=10]  }

In the above:

  • Static = Namespace
  • "meta" = Value
  • [name="cerberos sensorReporter"] = array of key/value pairs, only one pair in this example

If you only have one value for a given namespace, you can just define a value. If you have multiple key value pairs, you must define both the value and the list of key value pairs as shown in the example above.

A version of that Item with just the value defined would be

Switch vCerberos_SensorReporter_Online "Cerberos sensorReporter [MAP(admin.map):%s]"
    <network> (gSensorStatus, gResetExpire, sensorReporters)
    { expire="2m,command=OFF",
      name="cerberos sensorReporter" }

If just defining a single value, you can only store Strings. When using the list of key value pairs, you can use numbers as well. In Python, you will get a Java Number when you extract the key value so if you need a primitive int, call .intValue().

See https://openhab-scripters.github.io/openhab-helper-libraries/Python/Core/Packages%20and%20Modules/metadata.html for a list of all of the metadata functions supported by the JSR223 Helper Libraries.

Python Example

See Design Pattern: Human Readable Names in Messages for the theory of operation of these Rules and Items and the Rules DSL equivalent. The tl;dr is when a service or device goes offline the Rule generates an alert. When an alert is generated we set a flag indicating we have sent the alert. We also use that DP to convert the Item name to something a little more human friendly.

We will replace the Human Readable Names in Messages DP part and the Design Pattern: Associated Items with metadata on the Item.

Items

Group:Switch:AND(ON, OFF) gSensorStatus "Sensor's Status [MAP(admin.map):%s]"
  <network>

Group:Switch gOfflineAlerted

Switch vNetwork_cerberos "cerberos Network [MAP(admin.map):%s]"
  <network> (gSensorStatus, gResetExpire)
  { channel="network:servicedevice:cerberos:online",
    expire="2m",
    name="cerberos"  }

Switch vCerberos_SensorReporter_Online "Cerberos sensorReporter [MAP(admin.map):%s]"
    <network> (gSensorStatus, gResetExpire, sensorReporters)
    { expire="2m,command=OFF",
      name="cerberos sensorReporter" }

Notice how we are defining the human friendly name for the Item using statically defined metadata. Also notice how we do not have the associated _Alerted Item defined. We will be using metadata instead. For completeness, the Alerted metadata uses a full namespace,"value",["key":value] but since it’s only one value for the namespace a simple value would have worked.

Python

from core.rules import rule
from core.triggers import when
from core.metadata import get_value, get_key_value, set_metadata
import personal.util
reload(personal.util)
from personal.util import send_info
from threading import Timer
from core.actions import Transformation

# -----------------------------------------------------------------------------
# Python Timers for online alerts
alertTimers = {}

def alert_timer_expired(itemName, name, origState):
    status_alert.log.debug("Status alert timer expired for {} {} {}".format(name, origState, items[itemName]))
    del alertTimers[itemName]
    if items[itemName] == origState:
        send_info("{} is now {}".format(name, Transformation.transform("MAP", "admin.map", str(items[itemName]))), status_alert.log)
        set_metadata(itemName, "Alert", { "alerted" : "ON"}, overwrite=False)
    else:
        status_alert.log.warn("{} is flapping!".format(itemName))

@rule("Device online/offline", description="A device we track it's online/offline status changed state", tags=["admin"])
@when("Member of gSensorStatus changed")
def status_alert(event):
    status_alert.log.info("Status alert for {} changed to {}".format(event.itemName, event.itemState))

    if isinstance(event.oldItemState, UnDefType):
        return

    alerted = get_key_value(event.itemName, "Alert", "alerted") or "OFF"
    name = get_value(event.itemName, "name") or event.itemName

    #If the Timer exists and the sensor changed the sensor is flapping, cancel the Timer
    if event.itemName in alertTimers:
        alertTimers[event.itemName].cancel()
        del alertTimers[event.itemName]
        status_alert.log.warning(name +  " is flapping!")
        return

    '''
    If alerted == "OFF" and event.itemName == OFF than sensor went offline and we have not yet alerted
    If alerted == "ON" and event.itemName == ON then the sensor came back online after we alerted that
    it was offline
    '''
    status_alert.log.debug("Looking to see if we need to create a Timer: {} {}".format(alerted, event.itemState))
    if alerted == str(event.itemState):
        # Wait one minute before alerting to make sure it isn't flapping
        alertTimers[event.itemName] = Timer(60, lambda: alert_timer_expired(event.itemName, name, event.itemState))
        alertTimers[event.itemName].start()

Theory of Operation

When a device changes it’s online status we make sure it isn’t NULL or UNDEF. If it isn’t we then extract the alerted flag and the human friendly name from the metadata. Notice how we use a default value for both if they don’t exist.

If a timer exists we cancel it and log that the device is flapping.

If not and the alerted flag matches the Item’s new state we may need to send an alert. Create a timer.

If the Timer wasn’t cancelled, we generate the alert message and set the alerted flag to “ON” in the metadata.

Focusing on the metadata we have two namespaces we are working with. One, Static has the human friendly name for the Item which we use to produce nice and easy to read alerts and log messages. The second is the alerted flag which we use to keep track of whether we have generated an alert when the device went offline so we know to generate an alert when the device goes back online.

Previously, the name mapping was stored in a .map file and the transform Action used to get the Item’s human friendly name. Similarly, the alerted flag was a separate Item which we accessed and updated using the Associated Item DP.

Advantages and Disadvantages

Advantages:

  • allows for a reduction in Items to store configuration data and flags
  • lets one store metadata and configuration data with the Item reducing the number of places that this sort of information is defined
  • if using dynamically created metadata, the metadata is restored on OH restart (only for Items stored in JSONDB)

Disadvantages:

  • metadata is not accessible in Rules DSL
  • the REST API endpoints for metadata are not very robust making access and setting metadata through that mechanism awkward
  • PaperUI has no support for metadata

Related Design Patterns

Design Pattern How Used
Design Pattern: Human Readable Names in Messages The above is an alternative approach to this DP
Design Pattern: Associated Items In some cases, the above is an alternative approach to this DP

Edit: Added a bit more discussion about the metadata format and how it can be used.

3 Likes

Since get_key_value returns None if the key does not exist, you can do this. Looks like there are a couple places where you can use this.

    name = get_key_value(event.itemName, "Static", "name") or event.itemName

Basically, x = A or B means if A is falsey (False, 0, None, , etc.), then B.

I also like to set the timer interval in the Item metadata, but I don’t use the expire binding.

Here are some helpful metadata scripts that I’ve used for viewing/maintaining metadata…

# which lighting devices do not have a 'Morning' namespace?
from core.log import logging, LOG_PREFIX
from core.metadata import get_metadata

log = logging.getLogger(LOG_PREFIX + ".metadata")
no_metadata = "\n".join(map(lambda filtered_light: filtered_light.label, filter(lambda light: get_metadata(light.name, "Morning") is None, ir.getItem("gLight").getMembers())))
log.debug("The following devices in gLight do not have a 'Morning' namespace: \n\n{}\n".format(no_metadata))
# remove the HUEEMU namespace from all Items
from core.metadata import remove_metadata

for item in ir.getAll():
    remove_metadata(item.name, "HUEEMU")

I do that in a lot of places.

Grumble. Now this code is in like three places (I’ve always gotten a lot of mileage out using my alerting Rules as examples.

image

I was going to experiment with that at some point. I noticed that the Expire binding config ended showed up in the metadata when I looked at it in PaperUI. It got me to thinking if I could change the time dynamically. Seems the answer is “yes”?

I’ll go around and see if I can clean this up a bit across the several postings.

Thanks!

If the expire binding is storing values in Item metadata, I have no idea what would happen if you changed it. I’d guess that it is not listening for changes.

1 Like

The big question is whether the binding reads it every time I update the Item to it’s non-steady state or whether it only reads the metadata at system start and caches it. In the latter case, the changes would be ignored. In the former case, I could change the metadata and then postUpdate ON to start the Timer with the new time.

Speaking of listening for changes, another idea I had here was if it is possible to trigger a Rule based on changes to metadata. One use case would be to replace the Expire binding itself with a library class in the Rules. I could set the metadata to trigger a Rule that reads it and starts a Python or OH Timer.

I don’t see a compelling need to do something like that and I’m not even really sure I’ve thought it out thoroughly, but it would be interesting if it were possible to do that or something like that. Though maybe that’s stretching things a bit.

I do not see any RegistryChangeListener implemented in OHC, which I believe means that nothing is listening to metadata changes. Jython could definitely be used as a sandbox for this, before implementing in Java. This would be an interesting feature for OH3.

“if isinstance(event.oldItemState, UnDefType):
return”

So, if openhab is restarted, end the status changes to Alerted or OFF, and stays that way… what happens then? Nothing? Does the olditemstate matter? I would be interested in the current state…
Guess I’m missing something…

The Rule is triggered by a change in the Item’s state. If the Item changed from NULL or UNDEF (i.e. that is the old state) we ignore the change. It’s not a meaningful change for the Rule so we exit. When an Item changes from an UnDefType it most likely means that OH just restarted or your .items files reloaded and had their states restoreOnStartup.

In other words, when the state changes from an UnDefType, it’s not a new event for which we need to alert. It’s being restored to it’s most recently reported state.

Sorry, I don’t get it… I just try to understand.
The rule you have checks fi a sensor goes online/offline.
Suppose the sensor goes offline, openhab goes offline and is restarted.
The sensor is still offline and because of the restart of openhab the state will go from undef to offline… (I’m a new to openhab, I could be wrong of course!). So, even if the sensor stays offline forever, the rule will only run one (undef->offline) but do nothing?
ps: I’m a professional programmer on a completely different system (IBM As400). But a newbie to openhab and §(j)ython. But I try to understand the examples…

Nearly.
At system boot time, all Items begin life with state NULL.
That happens before any rules are active, so it’s not a “change” that will trigger any rule.
That’s the way they stay, until “something” updates them.

As rossko57 indicates when OH first starts or when you reload a .items file, the Item gets initialized to NULL. When you have restoreOnStartup configured, it then get’s restored to the most recent state saved in the database. This restore can trigger Rules to run. But we don’t want to run the Rule in that case because it’s not a “real” event. It’s just being restored to it’s old state.

If you don’t have restoreOnStartup, you don’t need this check so omit it.

UNDEF is a different situation entirely. Some bindings will set an Item to UNDEF to indicate it has no way to know what state is correct. For example, when the MQTT binding loses it’s connection to the MQTT broker, it will set the Items linked to Channels connected to that broker to UNDEF. This could be a case where the check above would cause missing a state change an alerting if when the connection is reestablished it’s a different state. I don’t have that situation in my system where this code was taken as an example. If you do, omit the check or only check for NULL and not UnDefType which includes both NULL and UNDEF.

Thanks a Rossko57 and you!
It came to my mind too, that, e.g. the expire function returns undef, if no action is defined…
So, I was getting closer.
Thank you very much. I hope other people also learn from this… :slight_smile:

How can I translate this Metadata into the Items file?

value: “”
config:
iconUseState: “true”
title: Temperaturwert
subtitle: Wohnzimmer
actionAnalyzerItems:
- KNXDevice_TemperaturwertWohnen
- KNXDevice_TemperaturSollwertWohnenAbsolut

->

…{channel=“knx:device:a48faddb:Temperaturwert_Wohnen”, stateDescription=""[pattern="%.1f °C"], listWidget=""[actionAnalyzerItems=“KNXDevice_TemperaturwertWohnen”, “KNXDevice_TemperaturSollwertWohnenAbsolut”]}

This code doesn´t work

I’m not sure how to define nested data like that. I think there are examples of that sort of setup in the Helper Library Mode community library.

For the most part, right now I have enough to learn about OH 3 to support those who are using the UI so I’m going to be pulling back from helping with text config issues. I don’t have enough time.