.members.sortBy[lastUpdate].last no longer returns most recently updated item

I have a rather simple rule below that used to function just fine, but after a recent upgrade, my “last updated” group member always returns the same item.

Here’s the rule:

rule "MotionSensorAlarm"
when
	Item gMotion received update
then

	if (VAR_MotionAlarms.state == ON || VAR_ArmHouse.state == ON) {
	
	    Thread::sleep(500)
	
	    val lastItem =  gMotion?.members.sortBy[lastUpdate].last as ContactItem
		val valueOfLastItem = lastItem.state.toString.toLowerCase
		val nameOfLastItem = lastItem.name
	    val timeOutMinutes = 20
	    
		var HashMap<String, ?> sensor = motionSensorsHash.get(nameOfLastItem)
		var roomName = sensor.get("room")
		
		logDebug("ashburn.MotionSensorAlarm", "####### Motion Alert Initiated: "+nameOfLastItem+" ("+valueOfLastItem+") #######")

but when watching the logs, no mater what motion sensor in the group triggered the update, the “office” motion sensor is always what members.sortby returns.

14:27:41.023 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_MasterBath_Motion changed from OPEN to CLOSED
14:27:41.028 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 5 to 4 through CT_FF_MasterBath_Motion
14:27:42.728 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
14:29:57.436 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 4 to 3 through CT_FF_Dining_Motion
14:29:57.439 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_Dining_Motion changed from OPEN to CLOSED
14:29:59.127 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
14:31:10.672 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 3 to 2 through CT_FF_NataliesBedroom_Motion
14:31:10.672 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_NataliesBedroom_Motion changed from OPEN to CLOSED
14:31:12.530 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
14:31:15.410 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_NataliesBedroom_Motion changed from CLOSED to OPEN
14:31:15.415 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 2 to 3 through CT_FF_NataliesBedroom_Motion
14:31:17.211 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
14:35:57.131 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_NataliesBedroom_Motion changed from OPEN to CLOSED
14:35:57.131 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 3 to 2 through CT_FF_NataliesBedroom_Motion
14:35:58.574 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
14:36:01.821 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_NataliesBedroom_Motion changed from CLOSED to OPEN
14:36:01.825 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 2 to 3 through CT_FF_NataliesBedroom_Motion
14:36:03.239 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######

Any suggestions?

There are other reports:

Some things to check:

  • check to see if all members of the group have a lastUpdate. Sometimes an upgrade can identify situations where you have an Item that is not restoredOnStartup or the persistence DB got reset and thus lastUpdate is returning null.
  • check that all the relevant Items are being saved to the persistence DB. How will depend on the DB used.

Follow the thread I linked to above.

That thread was great, thanks. I did add the null filter hoping that sounded pretty promising, but no luck.

val lastItem = gMotion?.members.filter[s|s.lastUpdate("influxdb") != null].sortBy[lastUpdate("influxdb")].last as ContactItem

So I followed advice and logged each group member’s update time so I could get a bit more verbose on the item selection process. Several items appear to have items way in the future (years)…

16:12:54.491 [INFO ] [marthome.event.ItemStateChangedEvent] - CT_FF_NataliesBedroom_Motion changed from CLOSED to OPEN
16:12:54.494 [INFO ] [ome.event.GroupItemStateChangedEvent] - gTotalActiveMotion changed from 7 to 8 through CT_FF_NataliesBedroom_Motion
16:12:55.045 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_NataliesBedroom_Motion 2017-07-12T16:12:54.488-05:00
16:12:55.078 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Kitchen_Motion 2017-07-12T15:38:35.959-05:00
16:12:55.104 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_JakesBedroom_Motion 2017-07-12T14:54:00.340-05:00
16:12:55.137 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Office_Motion 2195-04-12T17:26:00.484-05:00
16:12:55.165 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Lodge_Motion 2017-07-12T16:12:33.216-05:00
16:12:55.218 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_FamilyRoom_Motion 2017-07-12T15:38:36.475-05:00
16:12:55.269 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_MasterBath_Motion 2133-03-09T13:55:41.491-05:00
16:12:55.291 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_EX_Garage_Motion 2017-07-12T11:30:12.721-05:00
16:12:55.323 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_MasterBed_Motion 2017-07-12T16:11:31.597-05:00
16:12:55.353 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Dining_Motion 2017-07-12T16:07:34.582-05:00
16:12:55.381 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_LL_Motion 2017-07-12T00:00:00.151-05:00
16:12:57.367 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Initiated: CT_FF_Office_Motion (open) #######
16:12:57.376 [INFO ] [del.script.ashburn.MotionSensorAlarm] - ####### Motion Alert Extend Timer: CT_FF_Office_Motion #######

Any ideas why these dates would be there (I assume it was a recent occurrence as this rule broke just after I upgrade OH2 a couple weeks back).

I can attempt to purge those bad records out of influxdb… probably best to do so… else can I filter on dates on before now?

those dates look very fishy. I suspect the data got corrupted somehow and would purge them from InfluxDB. Filtering them out would just be a band aide on the real problem.

removed my last post… dug into influx side to figure out deletes with future dates. Problem was, when i started dissecting the data, influx showed the last two values in the table were from today (2017) for the office node, which influx claims last update was in 2195:

16:12:55.137 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Office_Motion 2195-04-12T17:26:00.484-05:00

> select * from autogen.CT_FF_Office_Motion WHERE time > now() - 9h limit 30
    name: CT_FF_Office_Motion
    time                value
    ----                -----
    1499864916567000000 0
    1499866255034000000 1

any suggestions on how i can clear OH’s lastupdate state?

lastUpdate will return the “biggest” date, not necessarily the one most recently entered into the database.

You need to create a query for entries with a timestamp larger than now. I’m not really great with InfluxDB so can’t really help with that.

yeh, that occurred to me as well… i’ll keep digging. Thanks.

How about just toggling the bad alarms using karaf?
Wouldn’t a change of state create an updated record (with the current date) in the DB?

that’s a great idea nekmech… I just toggled the two sensors that have the bad lastupdate date on them and they are still stuck with bad dates. I also restarted the OH service, but to no avail yet.

Maybe turn on persistence debug for Influx and toggle again in karaf.
Perhaps the debug info will say something about why it didn’t update the timestamp when you toggle the alarms.

that found it… let’s see if i can use that to purge it from influx…

19:49:24.115 [DEBUG] [.internal.InfluxDBPersistenceService] - got a query
19:49:24.116 [DEBUG] [.internal.InfluxDBPersistenceService] - descending ordering 
19:49:24.117 [DEBUG] [.internal.InfluxDBPersistenceService] - query string: select value from "autogen"."CT_FF_Office_Motion" ORDER BY time DESC limit 1
19:49:24.144 [DEBUG] [.internal.InfluxDBPersistenceService] - objectToState found a ContactItem
19:49:24.147 [DEBUG] [del.script.ashburn.MotionSensorAlarm] - ###### MOTION ALERT DEBUG ITEMS: CT_FF_Office_Motion 2195-04-12T17:26:00.484-05:00

Feel like i’m just spinning my wheels with influx here… lots of googling shows others are frustrated with deletes as well. Any influx feedback? Delete doesn’t seem to actually delete below:

> select value from "autogen"."CT_FF_Office_Motion" ORDER BY time DESC limit 1
name: CT_FF_Office_Motion
time                value
----                -----
7109159160484051136 1

> DELETE from CT_FF_Office_Motion WHERE time = 7109159160484051136
> select value from "autogen"."CT_FF_Office_Motion" ORDER BY time DESC limit 1
name: CT_FF_Office_Motion
time                value
----                -----
7109159160484051136 1

Sorry, not an Influx expert but perhaps instead of deleting these records, you can just update the “time” field?

Edit: If that’s possible, I would suggest setting a time that is before “now”. Then you can toggle the alarm again and see if you get a current date.

No, it wouldn’t because InfluxDB stores all the updates and when you call lastUpdate it returns the one with the “biggest” (i.e. farthest from midnight 1970/1/1) so the entries with the dates in the future will always be returned instead.

It doesn’t toggle the timestamp, it creates a whole new entry. And because the bad entry is in the future the new entry will not be returned by lastUpdate.

I wish I had more to offer. I’ve barely used IfluxDB myself. I do like nekmech’s idea of changing the bad entries if deletion is not working.

Just found this thread as I’m facing the exact same problem. My lastUpdate calculations with influx backend dont work any longer either.
Will check Influx data now to confirm. In my case it only the time when a PIR was last triggered. So I dont need too much of history. In worst case I’m going to delete the influx series and let OH2 re-create it.

I’m more concerned about how quirky dates made it into Influx. Might be a bug somewhere introduced with 2.1. Al least in my setup everything was working flawlessly before upgrading to 2.1

instead of deleting the series, I just renamed my item and now merge the two series from influx in my grafana graphs. Not elegant, but deleting this bad data from influx was too time consuming and i’m moving on