Tracking length of time Item is in state, in aggregate, by day

nolan_garrett · April 29, 2020, 5:23pm

I believe I could write some pretty horrific rules that may be able to do this - all of my attempts devolve into what feels like way too many lines of code for something that should be simple. Here’s what I want to do, and I am curious if any of the rules experts out here may have a simplified approach.

My goal is to track the amount of time I spend gaming (e.g. Xbox, Switch, or PS4 input) and the amount of time I watch TV (Apple TV or OTA inputs) in aggregate, on a day by day basis. I am using an LG WebOS TV. My concept was that I could track the current input state, but this became more convoluted when I found that the input state does not change when the TV is turned off, thus I need to only count time when the TV is both turned on, and on a specific input (or set of inputs).

Any thoughts on the most simple approach I could use here to get this data? My first attempt included a lot of cron timers and variables counting minutes, but it was very complex and my preference is to avoid timers. I do not need this data in realtime, I only need the aggregated data collected so I can send it to an external web service each day.

rlkoshak · April 29, 2020, 5:49pm

First of all, this is not an easy task.

One approach:

By “on a daily basis” I assume you are mainly interested in a summary at the end of the day and not an up-to-the-minute account. This simplifies things considerable.

At a high level:

trigger a Rule when either specific input Item changes or when the TV on/off state changes.
create a Number Item which will store the current usage
create a Number Item which will store the previous day’s usage
create a DateTime Item to store when it was the last time that we entered a period where you want to keep track of the time
in the Rule, if we changed to a time to track, update the DateTime Item with the current time stamp
if we exit a time to track (TV turns off, input changes to some other input that’s not tracked) calculate the amount of time that has passed since the timestamp in the DateTime Item and now, add that to the current usage Item
At midnight trigger a Rule that copies the state of the current usage Item to the previous day’s usage, if the TV is in a tracking state (midnight gaming ) reset the DateTime Item

Another approach:

trigger a rule every minute. If the TV is in a tracking state add one to the current usage Item
at midnight reset the current usage Item

Another approach:

set up persistence to store the state of a Number Item using an everyMinute strategy
write a Rule that sets that Item to 1 when it’s time to track and 0 when it isn’t
at midnight, populate the previous day’s usage Item with UsageTracking.sumSince(now.minusDays(1)) which counts up all the ones over the past day

Another approach:

write a Rule to store the state of a Switch Item that is set to ON when the in a tracking period and OFF when not
save that switch to InfluxDB or MySQL/MariaDB using an everyChange strategy
chart the usage using Grafana

The Red bars at the bottom show when the heater was ON.

The Discrete Plugin for Grafana might be more appropriate for this:

nolan_garrett · April 29, 2020, 6:50pm

Thank you! These are great ideas. I will think on them further and attempt to implement one.

Most likely this approach, as I already have persistence online and this ruleset would be easily implemented in my current setup. This is far more elegant than my incredibly convoluted approach.

kriznik · April 29, 2020, 6:54pm

I was wondering about same the other day … and still can’t really get why it is not an “easy task”
persisted data are persisted in time so basically what is needed is to extract persisted data with their timestamps and do the math.

I haven’t looked into that deeply but as graphs are generated basically from same informations, times should be available in some property of the item already, if am not mistaken ?

nolan_garrett · April 29, 2020, 7:59pm

I thought the same. My original rules concept was that I could simply look at the times between state changes, either within OH2 rules (preferably) or direct against my influxdb, and extract what I need. What I found was that calculating differences in timestamps for each input state change and simultaneously identifying the Power state of the TV turned into a very complicated ruleset. I did not attempt the approach direct against influxdb, as my preference is to keep my automation centralized within OH2 and rules whenever possible.

rlkoshak · April 29, 2020, 8:01pm

It’s easier for sure with persistence. But OH doesn’t expose a full query ability so you can’t, for example, get the date/time and state for all entries in the database for the past 24 hours. Without that you can’t easily get at those timestamps to do the math. So you have to work around that limitation by:

keeping track of the timestamps outside of persistence using Rules
storing data in the database that will allow you to use sumSince or one of the other persistence extensions to do the math for you (e.g. the third approach above)

Both of these work around approaches are not super easy to get right because you have all the edge cases to have to deal with (e.g. spanning midnight).

kriznik · May 15, 2020, 7:39pm

is there a way how to access these informations from OH somehow?
It sounds not very clean solution to track time separately when this information is already there…

rlkoshak · May 15, 2020, 8:35pm

The time isn’t available in OH itself. That’s why you have to keep track of it separately.

5iver · May 15, 2020, 9:48pm

Yes, if you use persistence. In a rule, use the REST API to get the persisted values for the time period in question, then iterate through them and tally up the length of time that the Item had the requested value. This is much easier to do in Jython.

kriznik · May 19, 2020, 5:35pm

and is Jython already easier to use instead of installing stuff in shell? just asking, not really following last updates tho.

maybe short sample would be handy…please?

5iver · May 19, 2020, 10:13pm

I’m not sure what you mean.

Here is a simple example script for using Jython to get the length of time in milliseconds that an Item was in each it’s states within a given time period. SwitchItems are tricky, but I think I got them working with this too. I have only done some simple testing, so let me know if you find any issues. Should be enough to play with… and much easier than the rules DSL.

import json

from java.sql import Timestamp

from core.actions import Exec
from core.log import logging, LOG_PREFIX

LOG = logging.getLogger("{}.TEST_2".format(LOG_PREFIX))


def get_PERSISTED_DATA(item_name, time_start, time_end):
    PERSISTED_DATA = Exec.executeCommandLine("/bin/sh@@-c@@/usr/bin/curl -s -X GET --header \"Accept: application/json\" \"http://localhost:8080/rest/persistence/items/{}?starttime={}&endtime={}\"".format(item_name, time_start, time_end), 5000)
    return json.loads(PERSISTED_DATA)


ITEM_NAME = "DS_Pantry_Contact"
LOG.warn("ITEM_NAME: {}".format(ITEM_NAME))

CURRENT_TIME = DateTimeType().zonedDateTime

# Use this for today's values
# RAW_PERSISTED_DATA = get_PERSISTED_DATA(ITEM_NAME, CURRENT_TIME, CURRENT_TIME.withHour(0).withMinute(0).withSecond(0).withNano(0))

# Use this for all of yesterday's values
RAW_PERSISTED_DATA = get_PERSISTED_DATA(ITEM_NAME, CURRENT_TIME.withHour(0).withMinute(0).withSecond(0).withNano(0), CURRENT_TIME.minusDays(1).withHour(0).withMinute(0).withSecond(0).withNano(0))
LOG.warn("RAW_PERSISTED_DATA: {}".format(RAW_PERSISTED_DATA))

PERSISTED_DATA = RAW_PERSISTED_DATA["data"]
LOG.warn("PERSISTED_DATA: {}".format(PERSISTED_DATA))

IS_SWITCH = itemRegistry.getItem(RAW_PERSISTED_DATA["name"]).type == "Switch"
#LOG.warn("IS_SWITCH: {}".format(IS_SWITCH)

# create dict where keys are each possible state
summations = {}
keys = list(set([historic_item["state"] for historic_item in PERSISTED_DATA]))
#LOG.warn("{}".format(keys))
for key in keys:
    summations[key] = 0

for index in range(1, int(RAW_PERSISTED_DATA["datapoints"])):
    #LOG.warn("{}".format(PERSISTED_DATA[index]))
    if not IS_SWITCH or (IS_SWITCH and (PERSISTED_DATA[index]["time"] != PERSISTED_DATA[index - 1]["time"]) and (PERSISTED_DATA[index]["state"] == PERSISTED_DATA[index - 1]["state"])):
        time_start = Timestamp(PERSISTED_DATA[index - 1]["time"]).getTime()
        time_end = Timestamp(PERSISTED_DATA[index]["time"]).getTime()
        summations[PERSISTED_DATA[index]["state"]] += time_end - time_start

LOG.warn("summations: {}".format(summations))

The logs look like…

2020-05-19 18:01:34.775 [WARN ] [jython.TEST_2] - ITEM_NAME: Mode
2020-05-19 18:01:34.827 [WARN ] [jython.TEST_2] - RAW_PERSISTED_DATA: {u'datapoints': u'5', u'data': [{u'time': 1589713200000L, u'state': u'Morning'}, {u'time': 1589720400000L, u'state': u'Day'}, {u'time': 1589752500000L, u'state': u'Evening'}, {u'time': 1589763600000L, u'state': u'Night'}, {u'time': 1589770800000L, u'state': u'Late'}], u'name': u'Mode'}
2020-05-19 18:01:34.827 [WARN ] [jython.TEST_2] - PERSISTED_DATA: [{u'time': 1589713200000L, u'state': u'Morning'}, {u'time': 1589720400000L, u'state': u'Day'}, {u'time': 1589752500000L, u'state': u'Evening'}, {u'time': 1589763600000L, u'state': u'Night'}, {u'time': 1589770800000L, u'state': u'Late'}]
2020-05-19 18:01:34.828 [WARN ] [jython.TEST_2] - summations: {u'Late': 7200000L, u'Evening': 32100000L, u'Night': 11100000L, u'Morning': 0, u'Day': 7200000L}

2020-05-19 18:01:49.983 [WARN ] [jython.TEST_2] - ITEM_NAME: US_Pantry_Contact
2020-05-19 18:01:50.035 [WARN ] [jython.TEST_2] - RAW_PERSISTED_DATA: {u'datapoints': u'3', u'data': [{u'time': 1589740735000L, u'state': u'ON'}, {u'time': 1589740763000L, u'state': u'ON'}, {u'time': 1589740763000L, u'state': u'OFF'}], u'name': u'US_Pantry_Contact'}
2020-05-19 18:01:50.036 [WARN ] [jython.TEST_2] - PERSISTED_DATA: [{u'time': 1589740735000L, u'state': u'ON'}, {u'time': 1589740763000L, u'state': u'ON'}, {u'time': 1589740763000L, u'state': u'OFF'}]
2020-05-19 18:01:50.036 [WARN ] [jython.TEST_2] - summations: {u'OFF': 0, u'ON': 28000L}

2020-05-19 18:01:55.038 [WARN ] [jython.TEST_2] - ITEM_NAME: DS_Pantry_Contact
2020-05-19 18:01:55.090 [WARN ] [jython.TEST_2] - RAW_PERSISTED_DATA: {u'datapoints': u'0', u'data': [], u'name': u'DS_Pantry_Contact'}
2020-05-19 18:01:55.091 [WARN ] [jython.TEST_2] - PERSISTED_DATA: []
2020-05-19 18:01:55.092 [WARN ] [jython.TEST_2] - summations: {}

Areas that need some attention:

If there are no state changes in the time period given
The first state change of the day
If it is a SwitchItem and there is just one change in the time period given