Need help with simplifying item time stamping rule

I’ve got a bunch of door, window, temperature, and humidity sensors that I use rules to keep track of when they last updated. The scripts I use have the same rules with only the device names changing. This is getting hard to manage as I add more and more sensors. Here’s a peak at the one that handles temperature and humidity:

import org.openhab.core.library.types.*

rule "Update time/date for Temperature_Living_Update"
when
  Item Temperature_Living received update
then
  postUpdate(Temperature_Living_Update, new DateTimeType())
end

rule "Update time/date for Humidity_Living"
when
  Item Humidity_Living received update
then
    postUpdate(Humidity_Living_Update, new DateTimeType())
end

rule "Update time/date for Temperature_Master_Update"
when
  Item Temperature_Master received update
then
  postUpdate(Temperature_Master_Update, new DateTimeType())
end

rule "Update time/date for Humidity_Master"
when
  Item Humidity_Master received update
then
    postUpdate(Humidity_Master_Update, new DateTimeType())
end

rule "Update time/date for Temperature_Office_Update"
when
  Item Temperature_Office received update
then
  postUpdate(Temperature_Office_Update, new DateTimeType())
end

rule "Update time/date for Humidity_Office"
when
  Item Humidity_Office received update
then
    postUpdate(Humidity_Office_Update, new DateTimeType())
end

Item definitions looks like this:

/* ===================== Environmental ===================== */
Number		Temperature_Living			"LR Temperature [%.1f &deg;F]"						<temperature>	(Living,Temperature,IndoorTemp)		{ mysensors="1;1;V_TEMP", imperihab="room:Living Room,label:Living Room Temperature,hygroId:Humidity_Living" }
DateTime	Temperature_Living_Update	"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]"	<status>		(Living)
Number		Humidity_Living				"LR Humidity [%.1f %%]"								<water>			(Living,Humidity,IndoorHum)			{ mysensors="1;0;V_HUM" }
DateTime	Humidity_Living_Update		"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]"	<status>		(Living)

Number		Temperature_Master			"Master Temperature [%.1f &deg;F]"					<temperature>	(Master,Temperature,IndoorTemp)		{ mysensors="0;1;V_TEMP", imperihab="room:Master,label:Master Temperature,hygroId:Humidity_Master" }
DateTime	Temperature_Master_Update	"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]"	<status>		(Master)
Number		Humidity_Master				"Master Humidity [%.1f %%]"							<water>			(Master,Humidity,IndoorHum)			{ mysensors="0;0;V_HUM" }
DateTime	Humidity_Master_Update		"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]"	<status>		(Master)

Number 		Temperature_Office 			"Office Temperature [%.1f &deg;F]" 					<temperature> 	(Office,Temperature)				{mqtt="<[broker:/ESPTEMP1/DHTTest/Temperature:state:REGEX((.*))]"}
DateTime	Temperature_Office_Update	"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]" <status>		(Office)
Number		Humidity_Office 			"Office Humidity [%.1f %%]"							<water> 		(Office,Humidity)					{mqtt="<[broker:/ESPTEMP1/DHTTest/Humidity:state:default]"}
DateTime	Humidity_Office_Update		"Last update [%1$tm/%1$td/%1$tY %1$tl:%1$tM %1$tp]" <status>		(Office)

I did come across a post by @rlkoshak addressing this very thing:

I’d like to modify this for my purposes, but I’m having a heck of a time understanding how the .members.filter works. what would the ‘door|door.changedSince’ and ‘dt|dt.name’ be for my purpose? I’m interested in capturing all changes and not just when the door/window are opened but also when they are closed as well as any time temp/humidity is reported.

I’m guessing the rule for my temperature updates would be something like:

rule "A New Temperature Reported"
when
        Item Temperature_Living received update or
        Item Temperature_Master received update or
        ItemTemperature_Office received update
then
    // Get the temperature that changed
    Temperature?.members.filter(Temperature|Temperature.changed.forEach[ Temperature |
        // Do stuff

        // Save the date/time for openings
        
            Temperature?.members.filter(dt|dt.name == Temperature.name+"_Update").head.postUpdate(new DateTimeType)
        
    ]
end

Also is this line complete? It seems like it ends oddly:

gDoorSensors?.members.filter(door|door.changedSince(now.minusSeconds(1))).forEach[ door |

Can someone tell me how to set up the part that detects what member of the group has changed and the part that passes that (?) on to the part that updates the timestamp for that device?

Here is the breakdown of gDoorSensors?.members.filter(dt|dt.name == door.name+"_Last_Opened").head

  • gDoorSensors - the name of the group
  • ? - don’t throw an error id gDoorSensors is empty, NOTE: I do not use this anymore.
  • .members - returns a List of Items that are a member of gDoorSensors
  • .filter - a function that applies a lambda (i.e. a kind of adhoc funciton) that returns a boolean to each Item in the List and builds a new list only with those Items where the lambda returns true
  • (dt|dt.name == "door.name+"_Last_Opened") - I actually use [ ] now instead of ‘( )’ to surround all lambdas now in my code even though the language will allow one to use parens sometimes. This helps distinguish lambdas from other language constructs. Anyway, in a lambda everything before the | are the arguments (i.e. things passed into the function) and everything after the | is the logic. So in this case the filter will call this lambda passing in the Item as the only argument. This Item I’ve named dt for this filter (stands for “door time”) but you could name it anything you want (except reserved words). The “body” of the lambda is a conditional that compares dt.name to door.name with “_Last_Opened” appended (NOTE: door was defined previously in the outer filter)
  • .head - the filter function returns a List of the Items that match the conditional in the body of the lambda. In this case we know that this List can contain at most one Item because no two Items can have the same name. The .head returns the first element in the List.

What makes this example a little complicated is that there is a filter within a filter.

So in words, what is happening in the complete code above is:

Loop through all the members of gDoorSensors and give me a list of all the Items that have changes in the last second. For each of the Items (we will call each Item “door” inside the loop) that have changed in the last second loop through all the members of gDoorSensorsTime and give me a list of all the Items that start with the same name as “door” and end in “_Last_Opened”. We know there can only be one so get the first Item from the list and postUpdate right now as the new DateTime.

So you need to do the following:

  • Create a Group for your Temperature Items (as a naming convention I start all my group names with a ‘g’) and add all your Temperature Items to this group. If all you are doing is tracking when they were last updated you can probably add your Humidity Items to this group as well. For my example I’ll call this group “gTempHum”
  • Create a separate Group for your Temperature and Humidity Update Items. Luckily you have already used a naming convention where we can construct the Update Item’s name from its corresponding Number Item’s name. for this example I’ll call this group “gTempHumUpdate”.
  • There is a bit of tricky behavior in the way openHAB processes events from groups. The only way you can trigger a rule using a Group is by triggering on updates. However, because of the way that updates are rolled up into the Group’s state multiple updates are generated for each change in a group’s Item. Consequently the rule will be triggered multiple times per change in an Item. Most of the time that doesn’t matter but because you are tracking times that is undesirable here. So you need to list all your Temperature and Humidity Number Items as the triggers to the rule.
  • Finally, use the filter to grab all the Items that have changed in the past second and for each of those update its corresponding Update Item.
    // I've broken this up for come clarity
    val changed = gTempHum.members.filter[sensor|sensor.changedSince(now.minusSeconds(1)]
    changed.forEach[sensor |
        // if you only care about one state (e.g. like the above which cares about OPEN), add an if statement here
        val updateItem = gTempHum.members.filter[up|up.name == sensor.name+"_Update"].head
        updateItem.postUpdate(new DateTimeType)
    ]

Hopefully you now see that this rule is wholly incorrect.

  • You need a group for Temperature declared in your Items
  • After the filter(, you have to choose a non-reserved name for the variable name. Because “Temperature” is presumably your Group name you need to call it something else (temp perhaps).
  • The call to find which Items have recently changed you use the method changedSince(now.minusSeconds(1)) or some other desired amount of time. NOTE: you also need persistence set up for all these Items.
  • After the call to changedSince() you need to close out your filter method with a close paren. At this point you have a List of Items that have changed in the last second.
  • Again, in the forEach the stuff before the ‘|’ is the variable name and must not be a reserved word (i.e. Item names, Group names, etc), again we can use “temp”.
  • The stuff after “Save the date/time for openings” is actually mostly correct. The only problem is you will need to rename “Temperature” to whatever you name the variable after the “forEach[”.

As written it is not complete. the [ opens a new context which gets closed three lines later (the last character before the “end”). Everything that is between the [ door | and that last ] is the body of the forEach loop. Everything between those symbols will be executed on each Item in the List.

2 Likes

Very informative! Thank you for taking the time to write this. I do have a group set up for temperatures (Temperature) but I did not include that in the items definition I posted. Apologies for that.

So if I’m reading this correctly, I should do something like this, right?

rule "A New Temperature Reported"
when
        Item Temperature_Living received update or
        Item Temperature_Master received update or
        Item Temperature_Office received update
then
    val changed = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(1)]
    changed.forEach[sensor |
    val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        updateItem.postUpdate(new DateTimeType)
    ]
end

I’m not at home to try this but I’m eager to get this going. This is so much easier to manage than all those individual rules. Thanks again for the help!

Looks good. the only change I would make is to indent the “val updateItem” to clearly show that it is inside the context of the forEach. This is more for human readability. functionally it looks correct.

Just tried it and I’m getting this error:

Aug 19 10:46:45 raspberrypi start.sh[2901]: 2016-08-19 10:46:45.428 [ERROR] [o.o.c.s.ScriptExecutionThread ] - Error during the execution of rule 'A New Temperature Reported': Segment cannot be null

after receiving and update like this:

Aug 19 10:46:45 raspberrypi start.sh[2901]: 2016-08-19 10:46:45.245 [INFO ] [runtime.busevents ] - Temperature_Office state updated to 80.6

The whole rule looks like this:

import org.openhab.core.library.types.*

rule "A New Temperature Reported"
when
        Item Temperature_Living received update or
        Item Temperature_Master received update or
        Item Temperature_Office received update
then
    val changed = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(1)]
    changed.forEach[sensor |
		val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        updateItem.postUpdate(new DateTimeType)
    ]
end

Any ideas what that might be referring to?

That’s a new one on me. I wonder what a Segment is.

Add some logging to see if we can narrow down which line the error is on.

    logInfo("Temp", "Received an update from a temp sensor")
    val changed = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(1)]
    changed.forEach[sensor |
        logInfo("Temp", "A sensor that changed is " + sensor.name)
	val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        logInfo("Temp", "The corresponding update Item is" + updateItem.name)
        updateItem.postUpdate(new DateTimeType)
    ]

Here’s the output:

2016-08-19 11:22:05.713 [INFO ] [c.internal.ModelRepositoryImpl] - Refreshing model 'environmental.rules'
2016-08-19 11:22:49.519 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor
2016-08-19 11:22:49.523 [ERROR] [o.o.c.s.ScriptExecutionThread ] - Error during the execution of rule 'A New Temperature Reported': Segment cannot be null

You have persistence configured for these items?

For the temperatures yes and I just checked and the MYSQL database is being updated. I do not have persistence for the update items, however. Is that necessary?

Is MySQL your default persistence? It isn’t necessary to persist the Update Items, though they will become undefined on an OH restart or rules file change.

Yes, it is my default/preferred persistence. Understood about restart/rules change. I am most interested in being able to tell if the sensor has updated recently so that I can know that the reading is valid for the current time frame and also be able to determine if the sensor has died as the last update would be older than expected.

Are you putting these Updates on your sitemap? If not there is no need for a separate Item. You can just call changedSince, lastUpdate, etc to see when the last update was received.

Just for grins, add “mysql” as the second argument to changedSince

changedSince(now.minusSeconds(1), "mysql")

That probably isn’t it but when nothing obvious is wrong, start testing the things you assume are right.

OH! and I see a typo, there is a missing paren on changedSince as it is written. Typo in the posting or in your actual config?

Are you using Designer?

val changed = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(1))]

Added mysql but still errors. I also corrected the missing paren.

I do have Designer installed but I really don’t know how to digest the errors it usually shows. I tried loading my file in and was greeted with this:

I’ve never seen that before. I took a look at the file and saw that it had the ^M files in it. I’m familiar with this and thought that I had Notepad+++ set to write out Unix files but apparently not. I thought this was the problem so I used dos2unix and got rid of the windows control characters. I tried reloading in Designer and it still gives this error. I don’t know if it has a cache of some sort but I can’t close the file and reopen it successfully. I always get the above error message.

This is what the rules look like now:

import org.openhab.core.library.types.*

rule "A New Temperature Reported"
when
        Item Temperature_Living received update or
        Item Temperature_Master received update or
        Item Temperature_Office received update
then
    logInfo("Temp", "Received an update from a temp sensor")
    val changed = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(1), "mysql")]
    changed.forEach[sensor |
        logInfo("Temp", "A sensor that changed is " + sensor.name)
        val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        logInfo("Temp", "The corresponding update Item is" + updateItem.name)
        updateItem.postUpdate(new DateTimeType)
    ]
end

Thanks again for your help and your patience.

I did some manipulation of the file and got it to load in designer. This is the error I’m seeing:

Am I missing an import or something?

Also:

This is cool for my temp/humidity sensors, but I do store values for my alarm items (door, windows) as I’m interested in seeing when these were last open/closed.

Hmmm. Typically when I see a lot of errors all piled up upon each other there is really only one real problem.

I logged into my server real quick and I think the problem is “changed” is a reserved word. Try renaming changed to “changedSensors” or something like that.

That did get rid of the error, but I’m still not getting the updates. See part of my log. You can see the errors before the modification was made and after it is echoing the loginfo that it’s running the rule:

2016-08-19 12:48:59.077 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor
2016-08-19 12:48:59.083 [ERROR] [o.o.c.s.ScriptExecutionThread ] - Error during the execution of rule 'A New Temperature Reported': Segment cannot be null
2016-08-19 12:49:59.149 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor
2016-08-19 12:49:59.153 [ERROR] [o.o.c.s.ScriptExecutionThread ] - Error during the execution of rule 'A New Temperature Reported': Segment cannot be null
2016-08-19 12:50:15.363 [INFO ] [c.internal.ModelRepositoryImpl] - Refreshing model 'newtemp.rules'
2016-08-19 12:51:00.783 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor
2016-08-19 12:51:59.262 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor
2016-08-19 12:52:59.330 [INFO ] [org.openhab.model.script.Temp ] - Received an update from a temp sensor

So progress has been made, but I’m not sure what’s wrong outside of what designer is saying:

That’s the only line that I see with any sort of error. Are those error really legitimate? I have other rules that use “mysql” just fine. The errors I see in Designer are cryptic to me, but I’m not a programmer so that could be one of my problems.

The word 'mysql' in not correctly spelled - Designer spell checks Strings and mysql isn’t in the dictionary. This isn’t really an error

Couldn't resolve... - This usually means you have a typo in a name of something you are referring to. I’m using a version of Designer a few versions back so I don’t see this, but I’ve seen others report that the new Designer no longer recognizes now and it never has recognized several of the actions like sending notifications and the like. In these cases run the rule and if there isn’t an error in the logs complaining about not finding a reference or something like that, the problem is in Designer, not your rule.

So, there are no “real” syntax errors in the rule.

The rule is running, hurray! However what appears to be happening is it isn’t finding any Items that have been updated in the past second. What might be happening is that the rule is starting to execute before this latest update has been saved to your database. Add a tiny sleep as the first line of the rule and see if that helps.

Thread::sleep(250)
val changedSensor = Temp...

Experiment with the sleep value (milliseconds so 250 is a quarter of a second) until you find a time that is as short as possible but always guarantees that persistence has finished. I’ve used 100 msec in my setup in the past but I’m using an embedded db and running on a pretty beefy machine.

Do you know where I might download the windows version of the Designer you are using? I’d prefer not seeing the Couldn't resolve errors.

I made the change you suggested and still nothing. I tried modifying it so that it waits 2.5 sec and then searches now to minus 5 seconds like this:

import org.openhab.core.library.types.*

rule "A New Temperature Reported"
when
        Item Temperature_Living received update or
        Item Temperature_Master received update or
        Item Temperature_Office received update
then
    logInfo("Temp", "Received an update from a temp sensor")
    Thread::sleep(2500)
    val changedsensor = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(5), "mysql")]
    changedsensor.forEach[sensor |
        logInfo("Temp", "A sensor that changed is " + sensor.name)
        val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        logInfo("Temp", "The corresponding update Item is" + updateItem.name)
        updateItem.postUpdate(new DateTimeType)
    ]
end

Still nothing. :frowning:

I’m not using a Windows version. I’m running Designer on Ubuntu 16. The main difference is I downloaded it six months ago or so and never updated for the past two or three updates to OH.

I’m not sure how easy it is to get at the old ones.

Just to make sure we are not silently failing, add a log between val changedsensor and the forEach and one at the end.

then
    logInfo("Temp", "Received an update from a temp sensor")
    Thread::sleep(2500)
    val changedsensor = Temperature.members.filter[sensor|sensor.changedSince(now.minusSeconds(5), "mysql")]
    logInfo("Temp", "Found " + changedsensor.size + " updated sensors")
    changedsensor.forEach[sensor |
        logInfo("Temp", "A sensor that changed is " + sensor.name)
        val updateItem = Temperature.members.filter[up|up.name == sensor.name+"_Update"].head
        logInfo("Temp", "The corresponding update Item is" + updateItem.name)
        updateItem.postUpdate(new DateTimeType)
    ]
    logInfo("Temp", "Rule end")
end

Anything longer than a second is way more time than needed. So if it isn’t returning any matches there is something wrong with persistence or something like that.

I’m running headless on the RPi so the windows version is my only real option short of installing X and displaying on my windows machine.

New output shows this:

Aug 19 13:58:19 raspberrypi start.sh[702]: 2016-08-19 13:58:19.103 [INFO ] [org.openhab.model.script.Temp ] - Found 0 updated sensors

So you’re right, it is failing to find a changed sensor. How do I verify that persistence is working? I see entries in my database. Here’s how I have it configured:

Temperature* : strategy = everyChange

Note: when I checked the persistence setting I did note more ^M in my file. I originally set up OH on Windows but have recently moved to RPi. I’ve run dos2unix on them, rebooted, and still have the same thing happening. I have left it set to 2.5sec wait and minus 5 seconds for the changedSince.