Strange String behaviour on 'ON' / 'OFF'

tramlaan · December 3, 2015, 8:41am

I am confronted to a strange problem (OpenHab 1.6).

I am sending an MQTT message to an OPENHAB item
String SESKEY “Session Key Option (0-05) [REGEX(TEST=On,OFF=Off ):%S]” { mqtt="<[mymosquitto:Home/RF/nb/+/0/05:state:default]" }

The message I am sending is ‘ON’ or ‘OFF’ (uppercase) which is not accepted, however ‘on’, ‘On’, ‘oN’ and ‘Off’, ‘OFf’ are well accepted.
Other strings with uppercase or lowercase are always accepted, the issue is only with ON and OFF

Using the map file (en.map)
Site map includes
Text item=SESKEY label=“Session Key Option (0-05) [MAP(en.map):%S]”
CLOSED=closed
OPEN=open
undefined=unknown
ON=On
OFF=Off
Doesn’thelp

Neither a regular expression
Text item=SESKEY label=“Session Key Option (0-05) [REGEX(ON=on,OFF=off):%S]”

However using a Switch Item, I have to use ‘ON’ of ‘OFF’ uppercase to activate the inbound binding(<[mymosquitto:Home/RF/nb/040/0/44:state:default]”).
Now reading the documentation I see that the actual syntax should be ‘Off’ or ‘On’.

So a bunch of questions:

Why is it a test done for the upper / lowercase on a string On or Off only?
Why is the translation service not working (or what did I wrong)?
Why is the syntax ON/OFF uppercase for a switch and not On/Off as defined in the documentation (because I have to make exception between a switch to be ON or OFF an a variable that is ON or OFF?

Thanks in advance
Robert
FYI

ITEMS FILE

Group Ground_Floor “”
Group Living “”
Switch Living_Light_Street “” { mqtt=">[mymosquitto:Home/RF/sb/040/0/44:command:ON:default],>[mymosquitto:Home/RF/sb/040/0/44:command:OFF:default],<[mymosquitto:Home/RF/nb/040/0/44:state:default]" }
Number Temperature_Living “Temperature_Living_Street_Side” (Ground_Floor,Living) { mqtt="<[mymosquitto:Home/RF/nb/040/2/40:state:default]" }
Number Humidity_Living “Humidity_Living_Street_Side” (Ground_Floor,Living) { mqtt="<[mymosquitto:Home/RF/nb/040/2/42:state:default]" }
Group Overview “”
Switch Living_KAKU “” { mqtt=">[mymosquitto:home/HA_gateway/sb/node016/dev17:command:ON:1_D],>[mymosquitto:home/HA_gateway/sb/node016/dev17:command:OFF:0_01332798_0_00]", mqtt="" }
String Coordinator_90 “Coordinator Error Messages (3-90) [%S]” { mqtt="<[mymosquitto:Home/RF/nb/251/3/90:state:default]" }
String Coordinator_91 “Coordinator Event Messages (3-91) [%S]” { mqtt="<[mymosquitto:Home/RF/nb/251/3/04:state:default]" }
String Mote_92 “Mote Error Messages (3-92) [%S]” { mqtt="<[mymosquitto:Home/RF/nb/251/3/92:state:default]" }
String Mote_93 “Mote Event Messages (3-93) [%S]” { mqtt="<[mymosquitto:Home/RF/nb/251/3/93:state:default]" }
Group Network_Messages “”
Group Configuration_ACK “Configuration Acknowledgement” { binding="" }
Number NETWID “Network Identification (1-01) [%d]” { mqtt="<[mymosquitto:Home/RF/nb/+/1/01:state:default]" }
Number NODEID “Node Identification (1-02) [%d]” { mqtt="<[mymosquitto:Home/RF/nb/+/1/02:state:default]" }
Number COORID “Coordinator Identification (1-03) [%d]” { mqtt="<[mymosquitto:Home/RF/nb/+/1/03:state:default]" }
String ENCKEY “Encrypted Key (3-04)” { mqtt="<[mymosquitto:Home/RF/nb/+/3/04:state:default]" }
String SESKEY “Session Key Option (0-05) [REGEX(TEST=On,OFF=Off ):%S]” { mqtt="<[mymosquitto:Home/RF/nb/+/0/05:state:default]"

SITEMAP FILE

sitemap Home label=“Home Automation”
{
Frame item=Ground_Floor label=“Ground_Floor” {
Text item=Ground_Floor label=“Ground_Floor” {
Text item=Living label=“Living” {
Switch item=Living_Light_Street label=“Lamp (Street Side)”
Text item=Temperature_Living label=“Temperature (Street Side) [%.1f °C]”
Switch item=Living_KAKU label=“Living_KAKU”
Text item=Humidity_Living label=“Humidity (Street Side) [%.1f %%RH]”
}
}
}
Frame item=Overview label=“Status_Overview” {
Text item=Overview label=“Recordings” {
Frame item=Living {
Text item=Overview label=“Living”
Chart item=Temperature_Living label=“Last Hour Temp [%.1f °C]” icon=“temperature” period=hrefresh
Chart item=Humidity_Living label=“Last Hour Humidity [%d %%RH]” icon=“humidity” period=hrefresh
Webview label=“Graph” icon=“chart” url=“https://192.168.0.253:8443/habmin/chart.html?chart=Living1&service=rrd4j&period=30000&refresh=15” height=10
}
}
}
Frame item=Network_Messages label=“Network Errors and Events” {
Text item=Network_Messages label=“Network Messages” {
Text item=Coordinator_90 label=“Coordinator Error Messages (3-90) [%s]”
Text item=Coordinator_91 label=“Coordinator Event Messages (3-91) [%s]”
Text item=Mote_92 label=“Mote Error Messages (3-92) [%s]”
Text item=Mote_93 label=“Mote Event Messages (3-93) [%s]”
}
Text item=Configuration_ACK label=“Configuration Acknowledge” {
Text item=NETWID label=“Network Identification (1-01) [%d]”
Text item=NODEID label=“Node Identification (1-02) [%d]”
Text item=COORID label=“Coordinator Identification (1-03) [%d]”
Text item=ENCKEY label=“Encryption key (3-04) [%s]”
Text item=SESKEY label=“Session Key Option (0-05) [MAP(en.map):%S]”
}
}
}

watou · December 3, 2015, 10:28am

The reason this happens is probably here. This usage of the TypeParser class is a bit naïve in that it takes a String and tries to find the most specifically matching openHAB Type that would match it. In the case of the MQTT binding, the strings you receive like ON, OFF, OPEN, CLOSED, UP, DOWN, something that can be parsed as a hue/saturation/brightness, a percentage, a decimal number, a date and then, lastly, a string. Since your String is ON, you would end up with an OnOffType kind of state, suitable for a Switch item, for example.

A preferable approach in bindings would be to explicitly define what kind of state type you wanted, and not use the TypeParser approach, but this would probably be a breaking change to the MQTT binding, which is probably unwise at this point.

Also, are you sure that is a valid REGEX transformation?

tramlaan · December 3, 2015, 10:50am

Hi,
Thanks for the answer, but:
OpenHAB is still magic (in other words for a mortel as me, it is very complex to use and the tons of explanations doesn’t help me a lot).
So I understand that OnOff is a kind of key string that is “always” parsed by OpenHab … MQTT binding, I can live with that.
My problem is the syntax inconsistency with the Switch which is looking of an ONOFF string, which makes difficult to applied the same string for different items.
Is it no way to MAP, TRANSLATE, TRANSFORM a string before its is interpreted by OpenHab? Is tho issue purely related to the implementation of the MQTT Binding?

Regarding REGEX, I am using OpenHABmin to configure OpenHAB, and REGEX is one of the options for transformation.

Robert

watou · December 3, 2015, 11:04am

Please try to state what exactly you want to do, and what is instead happening. I can point out a couple things in the meantime:

You want to use %s and not %S.
You are trying to transform the string that is being presented in the UI by doing

String SESKEY "Session Key Option (0-05) [REGEX(TEST=On,OFF=Off ):%S]" { mqtt="<[mymosquitto:Home/RF/nb/+/0/05:state:default]"

in your item definition, whereas you probably want to actually transform the received state of the item itself, not just how it’s displayed:

String SESKEY "Session Key Option (0-05) [%s]" { mqtt="<[mymosquitto:Home/RF/nb/+/0/05:state:YOUR_TRANSFORM(XXX)]"

Now all I need to know is what string is being published (i.e., what string you would like to publish) to the MQTT broker under the Home/FB/nb/+/0/05 topic, and what you want to transform it to. That will determine what you should replace YOUR_TRANSFORM(XXX) with.

tramlaan · December 3, 2015, 12:02pm

I Watou,

My REGEX in the example was just a test, I was trying to verify a translation, so in one of the test I used the string TEST to see if it was translated to On.

Shortly, I have an Arduino script that uses MQTT. This script sends periodically the state of an actuator.
The MQTT message has the form:
( MQTT Topic ) (MSG)
Home/RF/nb/040/0/44 ON
Home/RF/nb/040/0/44 OFF

I can activate the actuator for test via the mosquitto command line
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/sb/040/0/44 -m ON
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/sb/040/0/44 -m OFF

nb: is the direction (northbound)
sb: is the direction (southbound)
040: is the node
0: is a Boolean like parameter (1/0 or ON/OFF)
44: is the parameter associated to the actuator

This works perfectly with the item definition
Switch Living_Light_Street "" <light> { mqtt=">[mymosquitto:Home/RF/sb/040/0/44:command:ON:default],>[mymosquitto:Home/RF/sb/040/0/44:command:OFF:default],<[mymosquitto:Home/RF/nb/040/0/44:state:default]" }

Note that in both directions UPPERCASE is used

Now concurrently to this I want to set ON and OFF parameters (using the same Arduino function)
One of the parameter is for instance a session key that can be turned ON or OFF
As feedback of the setting command I expect to receive the status back in the same form as the one I’ used to send it.
So sending the command:
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/sb/040/0/05 -m ON

05: is the session key parameter

returns
Home/RF/nb/040/0/05 ON (UPPERCASE)

Now in order to fetch this answer back with OpenHab I did create a wild card Item
String SESKEY "Session Key Option (0-05) [%S]" <network> { mqtt="<[mymosquitto:Home/RF/nb/+/0/05:state:default]" }

Changing the case from %s to %S doesn’t change anything for the string parsing (only display changes from upper to lower case)
The parsing only works if I use in the MQTT message all the possible forms of ‘on’ or ‘off’ like oFF,Off,OfF, oFf, off, oFf, but not OFF (uppercase)
Which sounds stupid isn’t it!

Now if I change the Switch Item reply to:
Home/RF/nb/040/0/44 On
Home/RF/sb/040/0/44 Off
The switch state is not updated anymore :%)

The only workaround I see is to change the arduino code to reply differently for a Switch Item (with ON / OFF) and a String Item (with On / Off) in order to cope with the exotic behaviour of OpenHab, this is great !!!

I was expecting to use some kind of translation into OpenHab to cope with that issue, but for a reason that I can’t explain it doesn’t work.

Robert

watou · December 3, 2015, 12:40pm

I don’t understand why you have a 0 or 1 embedded in the MQTT topic, but you are also publishing/subscribing the string ON or OFF. Why does this make sense?

What is the intended purpose of using a wildcard in your MQTT topic? In theory, your openHAB SESKEY item will be updated when any node publishes a matching topic?

Could you be more specific as to what “the parsing only works” means? In your definition of the SESKEY item directly preceding that statement, I don’t see any attempt at parsing, so I’m afraid I still don’t follow, and don’t know if you are getting an error message in the log, or what exactly.

This is because On and Off are not valid string versions of the OnOffType internal to openHAB. They have to be ON or OFF in order to create a state that can be used with a Switch item. You could transform the received
On or Off in the <[mymosquitto:Home/RF/nb/040/0/44:state:default] part of your Switch item by replacing default with a transform like MAP(onoffcases.map) for example.

As an aside, you can simplify this item by using * in the outbound command:

Switch Living_Light_Street "" <light> { mqtt=">[mymosquitto:Home/RF/sb/040/0/44:command:*:default],<[mymosquitto:Home/RF/nb/040/0/44:state:default]" }

Not knowing more, I would suggest that you think through your MQTT topics again, making sure they exactly match what you are trying to achieve, and why you are using a wildcard and what you are intended to achieve with that. Maybe I’m being dense, but I still don’t have a complete picture of why the topics are organized this way.

Regards, John

steve1 · December 3, 2015, 1:01pm

Like John, I’m wondering what you mean by “parsing” in this context.

tramlaan · December 3, 2015, 1:25pm

> >> don't understand why you have a 0 or 1 embedded in the MQTT topic, but you are also publishing/subscribing the string ON or OFF. Why does this make sense?
Forget about 1 or 0, it is par to the Arduino process not MQTT
Southbound is publishing from MQTT broker to Arduino, which is able to interpret a message with 1 or ON and 0 or OFF,
Northbound publication is from arduino to the MQTT broker / OpenHab will only be ON or OFF

>> What is the intended purpose of using a wildcard in your MQTT topic? In theory, your openHAB SESKEY item will be updated when any node publishes a matching topic?
Yes, the wildcard + sign replaces the node number. Configuration of a node is not done (yet) via OpenHab but via command line interface, the idea is to have a Openhab frame that summarises the result of the last MQTT command of a particular parameter. I use wild card to fetch a particular parameter regardless of node, just to confirm that the action was taken.

> >> Could you be more specific as to what "the parsing only works" means? In your definition of the SESKEY item directly preceding that statement, I don't see any attempt at parsing, so I'm afraid I still don't follow, and don't know if you are getting an error message in the log, or what exactly.

I mean by that when I send to MQTT Broker / Openhab interface any string that is not ON or OFF uppercased, it is displayed on the WEB interface, nothing happens when I use ON or OFF
like:
Message is correctly displayed with:
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/nb/040/0/05 -m Off
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/nb/040/0/05 -m OFf
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/nb/040/0/05 -m ofF
….
NOT When I send
mosquitto_pub -u ‘MQTT’ -P ‘MQTT’ -t Home/RF/nb/040/0/05 -m OFF

>> This is because On and Off are not valid string versions of the OnOffType internal to openHAB. They have to be ON or OFF in order to create a state that can be used with a Switch item. You could transform the received 
On or Off in the <[mymosquitto:Home/RF/nb/040/0/44:state:default] part of your Switch item by replacing default with a transform like MAP(onoffcases.map) for example.

Well it is not like that I interpret the documentation when specifying OnOff type with a defined case I expect the same case structure is used, how should I suppose to know that for a switch, the it is ON or OFF (beside the fact that I have experiment it).

Now my point is not to modify the Switch behaviour, I just want to know why if I use a String I may not use ON or OFF uppercase?

NB: case you please give me an example of the usage of a MAP statement that should work, until now I could not find a way to let it work.

Thanks in advance
Robert

steve1 · December 3, 2015, 1:39pm

I just did a test with a String item and there’s no problem using ON and OFF for updates or commands, so that doesn’t seem to be the direct cause of your issue.

Have you been closely monitoring your log files (openhab.log, events.log) for any unusual log messages? Have you enabled DEBUG-level logging for the MQTT binding? That might also give some clues about what’s happening.

watou · December 3, 2015, 1:42pm

What happens if you have an item defined exactly like this:

String SESKEY "Session Key Option (0-05) [%s]" <network> { mqtt="<[mymosquitto:Home/RF/nb/040/0/05:state:default]" }

have it appear in your sitemap exactly like this:

Text item=SESKEY

and then issue this command from a command line:

mosquitto_pub -u 'MQTT' -P 'MQTT' -t Home/RF/nb/040/0/05 -m OFF

?

tramlaan · December 3, 2015, 1:44pm

Steve,

I had a look in the openhab.log and events.log, tree are no specific errors .
I will start to debuf MQQT to see if any issue with that.
Robert

steve1 · December 3, 2015, 1:46pm

I’d recommend looking not only for errors, for for anything unusual like unexpected or missing events. In other words, use the logs to determine whether the events are exactly what you’d expect in your scenario.

tramlaan · December 3, 2015, 2:04pm

Hi Watou
Same Problem

sitemap Home label="Home Automation"
{
	Frame item=Ground_Floor label="Ground_Floor"  {
		Text item=Ground_Floor label="Ground_Floor"  {
			Text item=Living label="Living"  {
				Switch item=Living_Light_Street label="Lamp (Street Side)" 				
				Text item=Temperature_Living label="Temperature (Street Side) [%.1f °C]" 				
				Switch item=Living_KAKU label="Living_KAKU" 				
				Text item=Humidity_Living label="Humidity (Street Side) [%.1f %%RH]" 				
			}			
		}		
	}	
	Frame item=Overview label="Status_Overview"  {
		Text item=Overview label="Recordings"  {
			Frame item=Living  {
				Text item=Overview label="Living" 				
				Chart item=Temperature_Living label="Last Hour Temp [%.1f °C]" icon="temperature" period=hrefresh				
				Chart item=Humidity_Living label="Last Hour Humidity [%d %%RH]" icon="humidity" period=hrefresh				
				Webview label="Graph" icon="chart" url="https://192.168.0.253:8443/habmin/chart.html?chart=Living1&service=rrd4j&period=30000&refresh=15" height=10 				
			}			
		}		
	}	
	Frame item=Network_Messages label="Network Errors and Events"  {
		Text item=Network_Messages label="Network Messages"  {
			Text item=Coordinator_90 label="Coordinator Error Messages (3-90) [%s]" 			
			Text item=Coordinator_91 label="Coordinator Event Messages (3-91) [%s]" 			
			Text item=Mote_92 label="Mote Error Messages (3-92) [%s]" 			
			Text item=Mote_93 label="Mote Event Messages (3-93) [%s]" 			
		}		
		Text item=Configuration_ACK label="Configuration Acknowledge"  {
			Text item=NETWID label="Network Identification (1-01) [%d]" 			
			Text item=NODEID label="Node Identification (1-02) [%d]" 			
			Text item=COORID label="Coordinator Identification (1-03) [%d]" 			
			Text item=ENCKEY label="Encryption key (3-04) [%s]" 			
			Text item=SESKEY label="Session Key Option (0-05) [%s]" 			
		}		
	}	
}

Group	Ground_Floor	""	<groundfloor>
Group	Living	""	<groundfloor>
Switch	Living_Light_Street	""	<light>	{ mqtt=">[mymosquitto:Home/RF/sb/040/0/44:command:ON:default],>[mymosquitto:Home/RF/sb/040/0/44:command:OFF:default],<[mymosquitto:Home/RF/nb/040/0/44:state:default]" }
Number	Temperature_Living	"Temperature_Living_Street_Side"	<temperature>	(Ground_Floor,Living)		{ mqtt="<[mymosquitto:Home/RF/nb/040/2/40:state:default]" }
Number	Humidity_Living	"Humidity_Living_Street_Side"	<humidity>	(Ground_Floor,Living)		{ mqtt="<[mymosquitto:Home/RF/nb/040/2/42:state:default]" }
Group	Overview	""	<chart>
Switch	Living_KAKU	""	<bulb>	{ mqtt=">[mymosquitto:home/HA_gateway/sb/node016/dev17:command:ON:1_D],>[mymosquitto:home/HA_gateway/sb/node016/dev17:command:OFF:0_01332798_0_00]", mqtt="" }
String	Coordinator_90	"Coordinator Error Messages (3-90) [%S]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/251/3/90:state:default]" }
String	Coordinator_91	"Coordinator Event Messages (3-91) [%S]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/251/3/04:state:default]" }
String	Mote_92	"Mote Error Messages (3-92) [%S]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/251/3/92:state:default]" }
String	Mote_93	"Mote Event Messages (3-93) [%S]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/251/3/93:state:default]" }
Group	Network_Messages	""	<network>
Group	Configuration_ACK	"Configuration Acknowledgement"	<network>	{ binding="" }
Number	NETWID	"Network Identification (1-01) [%d]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/+/1/01:state:default]" }
Number	NODEID	"Node Identification (1-02) [%d]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/+/1/02:state:default]" }
Number	COORID	"Coordinator Identification (1-03) [%d]"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/+/1/03:state:default]" }
String	ENCKEY	"Encrypted Key (3-04)"	<network>	{ mqtt="<[mymosquitto:Home/RF/nb/+/3/04:state:default]" }
String	SESKEY	"Session Key Option (0-05) [%s]"	<network>	{ binding="<[mymosquitto:Home/RF/nb/040/0/05:state:default]" }

steve1 · December 3, 2015, 2:10pm

@tramlaan, looking at the MQTT binding source code I think I might see the issue (maybe @watou can confirm since this sounds like what he was describing earlier).

The MQTT subscriber implementation will take a message payload and try to convert it to an openhab State or Command (a State in this case). If iterates through a list of candidate state types and attempts to parse the message payload using that type. The first candidate type is OnOffType and the last is StringType. If looks to me that an “ON” or “OFF” string will be parsed by the OnOffType and then that value will be posted as an update to the StringItem. However, the OnOffType is not an acceptable type for a StringItem so it ignores the update.

If you turn on DEBUG logging for “org.openhab.core.internal.items”, I think you’ll see a message starting with “Received update of a not accepted type” for your item.

tramlaan · December 3, 2015, 2:38pm

Steve,

Now the stupid question
How do I start this debugging function?
Robert

watou · December 3, 2015, 2:39pm

Yes; like I offered in my first reply:

[WARN ] [ore.internal.items.ItemUpdater] - InstantiationException on org.openhab.core.library.types.StringType

tramlaan · December 3, 2015, 2:42pm

Does it explain that all case variants of Of On/Off are working beside the ON/OFF one?
Robert

watou · December 3, 2015, 2:44pm

Yes; I tried to explain that at the top of this topic.

If you really want a String item to receive updates from the MQTT binding, you could change your SESKEY item to Switch, and define a String item with no binding defined for it, and write a simple rule like:

rule SESKEYStringVersion
when
  Item SESKEY received update
then
  SESKEYstring.postUpdate(SESKEY.state.toString)
end

But if you are only going to send OFF and ON to your SESKEY item’s topic, just make it a Switch item.

steve1 · December 3, 2015, 2:44pm

Take a look at the logback.xml file in the openhab configuration. Add a line like…

<logger name="org.openhab.core.internal.items" level="DEBUG"/>

However, given @watou’s reply, I’m about 95% sure this is the issue. It’s a problem with the MQTT binding. I’m wondering why the developer didn’t try to parse the incoming message based on the acceptable types of the item instead of a hard-coded list of types that apply to every item.

watou · December 3, 2015, 2:51pm

An issue across all bindings is a lack of consistency in how Types are chosen for Items. There are a wide variety of approaches, each with their own strengths and weaknesses, but the lack of consistency can lead to fair bit of confusion. The “deluxe” approach would be to choose the most specific Type that is accepted by the Item to which it is being posted, so the user always gets what s/he wants. This is on my to-do list for the Nest and Ecobee bindings, but the issue is much broader than that.

The “deluxe” approach described above would be better than my initial suggestion.