[SOLVED] MQTT, JSONPATH transformation & strange results in Basic UI

Tags: #<Tag:0x00007f6182c4bd88> #<Tag:0x00007f6182c4bc70> #<Tag:0x00007f6182c4bba8>

EDIT I have figured my mistake and posted an update below.

To anyone whom might have a clue…

Im setting up a basic home alarm type system. I have a Sonoff RF bridge with Tasmotta firmware on it. I have joined/paired a couple of door sensors to it, which can send OPEN/CLOSED/TAMPER notifications, however, these arrive through a JSON via MQTT and up in the form of an alphanumeric number, which is unique to each of the door sensors (see the frontdoor and backdoor map files below for more information).

The Sonoff RF bridge dumps out the information in the following format, when an action is triggered on the door sensors:

FROM THE MQTT BROKER

{"RfReceived":{"Sync":9460,"Low":410,"High":920,"Data":"123B","RfKey":"None"}}

What I intended for, was a simple page in Basic UI, where I convert this information into Frontdoor - STATUS, Backdoor - STATUS etc.

So I have used JSONPATH to pull out the DATA from the MQTT broker text (as above), so I would just get 123B (Code for closed on 1 of the sensors), which I am then using 2 individual map files on each of my Items, to convert that to OPEN/CLOSED/TAMPER, for simple display in the Basic UI. The reason there are two map files, is that if I use 1 map file for both sensors, then a change on sensor, is reflected in the sitemap/BasicUI, as being both sensors changing. Hence the use of 2 x map files… which may be the issue…somehow.

I have a problem though… When one sensor is triggered, the information for the other sensor disappears on the BasicUI interface and wont re-appear until I trigger another sensor, at which point the triggered sensor will show its status, but then the other sensors information goes blank.

Has anyone any ideas why? or how I should change what Im doing here?

ITEMS

String  FRONTDOOR "Front Door [MAP(frontdoor.map):%s]"	{mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived.Data)]"}
String  REARDOOR  "Rear Door [MAP(backdoor.map):%s]"	{mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived.Data)]"}

SITEMAP

Text 	item=FRONTDOOR
Text	item=REARDOOR

frontdoor.map

123A=TAMPER
123B=CLOSED
123C=OPEN

backdoor.map

456A=TAMPER
456B=CLOSED
456C=OPEN

Thanks in advance.

I see the error of my ways now.

Both my ITEM strings are simultaneously processing whatever is sent by the RF Bridge device. 1 of my ITEMS (frontdoor or backdoor) will know how to map the string to something useful (open/closed etc)… and the other wont be able to map it to anything, hence it effectively nulls out in the BasicUI.

As such, I believe my way forward will be to have 1 ITEM that collects the data from the MQTT broker, and then create a somewhat complex ruleset that identifies which device & status (as the codes are unique to each device) and update a completely different Text Item accordingly.

@ewrw i did not work a lot with jsonpath jet, but i think you problem is that when the broker send a message both items recieve it. And both filter out the requested $.RfReceived.Data which is valid for the one map transformation but not for the other. Hence one work the other shows nothing.

So best would be to only trigger the map when the jsonpaht is meant for the item.
This can be achieved by using a regex. I think you could then also reduce it to one map file using group.

Somthing like this, not tested. You should also be able to find informations in the net.

this should work on one item

JSONPATH($.RfReceived[?(@.Data=~ /123.*/ )].Data[0] )

I see what you are getting at with the REGEX (Im appalling with REGEX), but I can see how it would filter in theory.

I (eventually) came to the same conclusion as you about both items receiving it! Nice to hear someone agree! :slight_smile:

I will give that a go and see what happens!

Thanks

I gave it a shot, but no luck it appears. I briefly removed my map file from the item, so I could see the raw data but none appears, here is the full line I used:

{mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived[?(@.Data=~ /123.*/ ).Data[0]]]"}

I hunted the log files and receive a generic error about “An error occurred while transforming a JSON expression”. Im not familiar enough with either REGEX or what can be done within the confines of the mqtt string to diagnose what the error may be…but as you said, it was untested, so perhaps OpenHab just wont accept that string.

Missing ] after )].Data

Missing ) after [0])

so basically open and closing name spaces are wrong.

I think … think I have the line right now:

{mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived[?(@.Data=~ /123.*/ )].Data[0])]"}

Theres one [ at the start, which pairs out with the ] at the end… and the others map up to where I believe you said they should be? So I think the format is now correct?

Im not using a map file at the moment, just %s to return the raw data, and what Im getting in the interface is [] … an [ and a ]

Have I gotten the line correct based on your thoughts?

Maybe just data without brackets. As i said i am no pro with jsonpath. Just what i read from the manual. And i did not test it.

@ewrw edit:

This

{mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived[?(@.Data=~ /123.*/ ).Data] ]"}

Should return this

[
   "123B"
]

I did an exact copy/paste to ensure I had no mistakes written in, and strangely, I still just get the [ ] as the return.

I did try removing the [ and ] from just after RfReceived and the one after .Data… and I still get the [ and ]… Even after deleting my internet history and re-starting the browser (perhaps because without the extra [ and ], the line wont process and its keeping the data through persistence).

I have however settled on a solution.

I am using 1 Item to pull in my original MQTT line and then Im not mapping, but using rules to push the command out to other variables. e.g.

rule "Alarm Sensors"
when
    Item ALARMDATA received update
then 
	if (ALARMDATA.state == "123A") SENSORWINDOW.postUpdate("Closed")
	if (ALARMDATA.state == "123B") SENSORWINDOW.postUpdate("Open")
	if (ALARMDATA.state == "123C") SENSORWINDOW.postUpdate("Tamper")
	if (ALARMDATA.state == "456A") REARDOOR.postUpdate("Closed")
	if (ALARMDATA.state == "456B") REARDOOR.postUpdate("Open")
	if (ALARMDATA.state == "456C") REARDOOR.postUpdate("Tamper")
end

Its perhaps not as elegant, will require a small amount of system overhead and may process slightly slower by a few milliseconds… but it works and I will only have 8 actual sensors to add in.

So I do have a working solution at this time… however, like yourself, Id like to understand JSONPATH better and perhaps I will re-visit this in future (I will post an update if I do).

Thanks for your time, thoughts and help with looking into this, it was really appreciated!!

You can make it a bit easier by naming the Items for the contacts as follows.

String  contact_123 "Front Door" ...
String  contact_456 "Rear Door" ...

And make a rule that is valid for all Items.

rule "Alarm Sensors"
when
    Item ALARMDATA received update
then 
        // Get name of Number Item by removing the numbers
        val contactNumber = ALARMDATA.state.toString.substr(1,3)
        val stateOf = ALARMDATA.state.toString.substr(3,1)

        logInfo("Contact", "Number "+ contactNumber +" changed to "+ stateOf ) 

        // post the new value to the Number Item
	if (stateOf == "A") { 
           postUpdate( "contact_"+ contactNumber.toString , "Closed" ) 
        }
	else if (stateOf == "B"){ 
           postUpdate( "contact_"+ contactNumber.toString , "Open" ) 
        }
	else if (stateOf == "C"){ 
           postUpdate( "contact_"+ contactNumber.toString  , "Tamper" ) 
       }
end

Ahh… yes!!! Thats brilliant, much cleaner, easier and less code to have to amend!

Thank-you so much!! :slight_smile: Ill go and implement it now.

Can you please confirm if you got this working. And please post your confing for this (items, sitemap, rules etc). Have been struggling a few nights with this.

Hi Tom

The answer to your question of can I confirm I got this working, is both a a yes and a no. The code above works. The 1 issue I did struggle with, which I didnt know at the time, was that the VAL lines (for some reason) had to be initially declared in the top of my RULES (literally at the very top of the rules file). This is listed in the documentation https://docs.openhab.org/configuration/rules-dsl.html#the-syntax on the “Variable Declarations” area.

So you would need to put these at the TOP of the rules file:

        val contactNumber = ALARMDATA.state.toString.substr(1,3)
        val stateOf = ALARMDATA.state.toString.substr(3,1)

If you dont put them there, you get an error in the log about parsing the rules file on the line numbers where the VAL’s are declared.

So that was issue 1 for me… Issue 2 for me was that in the examples above, I listed that my sensors very simply separated Open/Closed/Tamper via the use of A, B and C… so the code example above is reading if there is an A, B or C sent via the sensors and then updating the contact_XXXXX site map entry… However, I ended up with a bunch of sensors that dont work with A, B and C, but give a random code e.g.

Open - USH3438
Closed - AUF8234
Tamper - IAS0238

Obviously, there is NO simple way, when you have multiple sensors with random codes like the above to progmatically use a simple 1 off rule on its own to pick out the difference between multiple sensors… Well, not easily.

I did look at using a MAP file, that would list each unique code and what that code mapped to… But I found an issue with … I cant recall exactly what it was, but somewhere along the line, I found that my variable couldnt work correctly because of the translation… I cannot remember it it was within the ITEMS, RULES or SITEMAP I had a problem… but I did.

So, I have ended up with having to define each code from the sensor and then the action taken for each code, within my rules file… which does look like a lot of code and it could probably be cleaned up by someone with more code knowledge than myself. but I dont have a huge rules file anyway and the code does work.

ITEMS - Creates a variable to pull in the code from a sensor, via a MQTT source

String ALARMSENSOR1 {mqtt="<[mybroker:tele/RFBridge/RESULT:state:JSONPATH($.RfReceived.Data)]"}

ITEMS - I created a variable which is On/Off which is basically is the alarm On or Off (is someone in the house)
Switch HOUSEOCCUPIED “Status”

ITEMS - Creates a variable for use within the sitemap, that will display the Open/Closed/Tamper status

String FRONTDOOR "Front Door [%s]"	<doorsensor>	(Alarmsensors)

ITEMS - I also wanted to record the LAST sensor triggered information, what time it was triggered and the alert type (Open/Closed/Tamper etc)

String 	ALERT		         		"Sensor Triggered [%s]"			<alerticon>
String 	ALERTTIME	         		"Alert Time [%s]" 				<alerticon>
String 	ALERTTRIGGER         		"Alert Trigger [%s]"     		<alerticon>

SITEMAP - As simple as displaying the variables you created above

    Frame label="Security" {	
		Switch item=HOUSEOCCUPIED				mappings=["ON"="Armed","OFF"="Disarmed"]
		Text label="Alarm System" icon="alarm1"{
			Frame label="Last Alert" {	
				Text 		item=ALERT			
				Text		item=ALERTTIME	
				Text		item=ALERTTRIGGER
			}
			Frame label="Sensor Status" {	
				Text		item=FRONTDOOR
				Text		item=REARDOOR
				Text		item=SENSORWINDOW
			}
			Frame {	
				Text 		item=MOTIONSENSOR1	
				Text		item=LANDINGSENSOR		
			}
		}
	}

RULES - At the top of my rules file I have imported Java date/time

import java.text.SimpleDateFormat
import java.util.Date

RULES - As for a rule to work with 1 sensor, because they are using different codes, each rule would look like the following

For my example below, use the following example as being sent by the sensor:
Open - USH3438
Closed - AUF8234
Tamper - IAS0238

The code will say that IF the alarm is ARMED, and the sensor sends Open/closed, then I DO want a telegram notification message sent to my phone with the details of that sensor & the date and time, along with updating the ALERT, ALERTTRIGGER and ALERTTIME variables.

If the alarm is DISARMED, and the door is Open/Closed then it wont send a notification to my phone OR update the ALERT, ALERTTRIGGER and ALERTTIME variables.

Either of the above actions will update the sitemap text variables to show the current state of the sensor as Open/Closed.

For Tamper events, I have chosen to recieve a notification via Telegram, even if the alarm is disarmed.

So yes, I have ended up with one rule like the below for each sensor, which is messy, but it works, my rules file is not very large anyway and I thought I would re-visit it one day in the future to see if there was a better way to do this, when working with sensors that send codes you cannot easily delinite programmatically.

rule "Alarm sensor 1 - Front Door"
when
    Item ALARMSENSOR1 received update
then 
	if (ALARMSENSOR1.state == "AUF8234" && HOUSEOCCUPIED.state == ON) 	{ FRONTDOOR.postUpdate("Closed") 		
			sendTelegram("TELEGRAMBOTNAME", "Alarm - Front Door Closed" + Current_DateTime.state.format(" %1$td/%1$tm/%1$tY at %1$tH:%1$tM")) 							
			ALERT.postUpdate("Front Door")
			ALERTTRIGGER.postUpdate("Closed")
			ALERTTIME.postUpdate(Current_DateTime.state.format("%1$td/%1$tm/%1$tY at %1$tH:%1$tM")) }						
	if (ALARMSENSOR1.state == "AUF8234" && HOUSEOCCUPIED.state == OFF) 	{ FRONTDOOR.postUpdate("Closed") }
	if (ALARMSENSOR1.state == "USH3438" && HOUSEOCCUPIED.state == ON) 	{ FRONTDOOR.postUpdate("Open") 			
			sendTelegram("TELEGRAMBOTNAME", "Alarm - Front Door Open" + Current_DateTime.state.format(" %1$td/%1$tm/%1$tY at %1$tH:%1$tM")) 								
			ALERT.postUpdate("Front Door") 	
			ALERTTRIGGER.postUpdate("Open")
			ALERTTIME.postUpdate(Current_DateTime.state.format("%1$td/%1$tm/%1$tY at %1$tH:%1$tM")) }						
	if (ALARMSENSOR1.state == "USH3438" && HOUSEOCCUPIED.state == OFF) 	{ FRONTDOOR.postUpdate("Open") }
	if (ALARMSENSOR1.state == "IAS0238" && HOUSEOCCUPIED.state == ON) 	{ FRONTDOOR.postUpdate("Tamper")
			sendTelegram("TELEGRAMBOTNAME", "Alarm - Front Door Tamper - Alarm was Armed" + Current_DateTime.state.format(" %1$td/%1$tm/%1$tY at %1$tH:%1$tM"))  			
			ALERT.postUpdate("Front Door") 	
			ALERTTRIGGER.postUpdate("Tamper")
			ALERTTIME.postUpdate(Current_DateTime.state.format("%1$td/%1$tm/%1$tY at %1$tH:%1$tM")) }						
	if (ALARMSENSOR1.state == "IAS0238" && HOUSEOCCUPIED.state == OFF) 	{ FRONTDOOR.postUpdate("Tamper") 		
			sendTelegram("TELEGRAMBOTNAME", "Alarm - Front Door Tamper - Alarm was Disarmed" + Current_DateTime.state.format(" %1$td/%1$tm/%1$tY at %1$tH:%1$tM")) }
end

Obviously, I dont know exactly what sensors/devices you are working with and what they send you, but hopefully this gives you some help figuring out what you need to do.

1 Like

Thank you Will!

Nice code. Some minor adjustments in my setup still needed.

By the way I’m using an ITead/Sonoff model DW1 door sensor, tiny booklet says it is a “CD100S wireless magnetic door detector”. I’m using it with a Sonoff RF Bridge 433.
Unfortunately it only sends a single MQTT message each time door is opened. No closed or tamper signal with this sensor.

I have the sonoff bridge too and Im using Tasmota firmware on it, though for the door/window sensors I used Kerui D026 which are a bit more than the Sonoff door sensors, but not too expensive. They do support the Open/Closed and Tamper… so if you really need one that does all 3 and dont want to spend too much, have a look at those.

Good luck!!

1 Like

Thanks Will,
Cool, your example got my Sonoff RF Bridge working for me.
As a Start of Code get me going to implement my own Stuff …

A bit of Disappointment are the Sonoff door sensors, a orderd a bunch of them, but only the open event is detected.
someone any idee to change this ???

regards

Hi Michael

Its a limitation of what the actual door sensors will send, rather than the bridge or openhab etc… So the only way to change this on the Sonoff sensors would be to dump new firmware on the sensors to send a closed signal (Im not claiming any such firmware exists… to my knowledge anyway… and even then I dont know how easy it would be to re-program them). This is why I went with the Kerui D026 sensors and they support all 3 actions out the box and have a good battery life of about 10-12 months (plus they are relatively cheap if you hunt around). The open/closed is performed by the magnetic relay switch and the tamper is supported by a push button on the back of the sensor, that activates if the sensor was removed from the door/window (which I dont think the Sonoff sensors have a button on the rear of the units, so couldn’t do tamper anyway).

Maybe if you could do a mixed solution and use a couple of the Kerui D026 for your doors in the house and the Sonoff’s for the windows or something.

Probably not the answer you were after, but hopefully it will save you a few frustrating hours hunting the internet for an answer!

Best of luck!

1 Like

Hmm, maybe do some hardware modifications … second reed relais ?

Well a order one Kerui D026 for Test :smile: 7,28 Euro by AliExpress.

In the mean time, finally i got my RF 433 Remote Control working with your Code.
This Stupid Control sends 4 Different Data codes for each key press.

Question: in a rule, is there a way to log the state ?
What i tried to do is something like this, where RFSensor is my Item:

rule "React on RFSensor update"
when
    Item RFSensor received update
then
     if (RFSensor.state == "4AFBB0" || RFSensor.state == "417430" || RFSensor.state == "479E40" || RFSensor.state == "40E770")  {
        FB_EasyHome_A.sendCommand(ON)
    } else if (RFSensor.state == "431A10" || RFSensor.state == "4CC190" || RFSensor.state == "458DA0" || RFSensor.state == "493C20") {
        FB_EasyHome_A.sendCommand(OFF)
    } else {
        logInfo("unknown rf Data", RFSensor.state)
    }
end

this gives me a Nullpointer Exception in the else statement, i don’t understand ??

regards

missing toString

1 Like

:smile::smile::smile::smile: Thanks, tested and works