MqTT Binding (in OpenHAB 2.5.1) transformationPatternOut MAP not working as expected

tbnobody · January 19, 2020, 12:26pm

Hello,

I try to migrate an item using the 1.x MqTT binding to the new one.
The previous item defintion looked like this:

Switch KG_Wohnraum_StarPower
    "Stern Status"
    {mqtt="
    >[mosquitto:home/1/XMAS/animation:command:ON:2],
    >[mosquitto:home/1/XMAS/animation:command:OFF:255],
    <[mosquitto:home/1/XMAS/animation:state:OFF:255],
    <[mosquitto:home/1/XMAS/animation:state:ON:default]
    "}

Animation = 255 means off and Animation = 2 (or anything else) means on

I tried to match that setup using the new binding:
Thing File:

Thing mqtt:topic:mosquitto:star
    (mqtt:broker:mosquitto)
{
    Channels:
        Type switch : power [
            stateTopic = "home/1/XMAS/animation",
            transformationPattern = "MAP:star_power.map",
            commandTopic = "home/1/XMAS/animation",
            transformationPatternOut = "MAP:star_power.map",
            retained = true,
            on="ON", off="OFF"
        ]
}

Item File:

Switch KG_Wohnraum_StarPower
    "Stern Status"
    {channel="mqtt:topic:mosquitto:star:power"}

star_power.map File:

0=ON
1=ON
2=ON
3=ON
4=ON
255=OFF
OFF=255
ON=2

If I write something to the topic using mosquitto_pub the switch changes from off to on or from on to off --> great

But if I press the switch within my sitemap the topic is set to “ON” --> not so great, I assumed that it should be 2 (based on the map file)
In the log file I get the following messages:

2020-01-19 13:22:26.185 [ome.event.ItemCommandEvent] - Item 'KG_Wohnraum_StarPower' received command OFF
2020-01-19 13:22:26.188 [nt.ItemStatePredictedEvent] - KG_Wohnraum_StarPower predicted to become OFF
2020-01-19 13:22:26.197 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Incoming payload 'ON' not supported by type 'NumberValue'
2020-01-19 13:22:26.197 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command '2' not supported by type 'OnOffValue': No enum constant org.eclipse.smarthome.core.library.
types.OnOffType.2
2020-01-19 13:22:26.212 [vent.ItemStateChangedEvent] - KG_Wohnraum_StarPower changed from ON to OFF

What am I doing wrong?

rossko57 · January 19, 2020, 1:27pm

You’re using a MAP, you don’t want those as well.
It’s not clear to me what order the binding does stuff in, but if tries to apply those before the MAP you might expect errors?

tbnobody · January 19, 2020, 2:30pm

I tried it at first without the on/off paremters. No success at all. It’s still publishing “ON”.

rossko57 · January 19, 2020, 3:00pm

That’s exactly what it should do when you specify on=“ON”
(In case it’s not clear, the on= off = gets used both for inbound and outbound)
So, you need to remove that instruction.
Having done so, I would advise a binding restart to make sure it has picked up minor edits.

Then you can show us the current config and current problem, if any.

tbnobody · January 19, 2020, 3:29pm

I removed the on/off parameter and performed a complete openhab restart (not just the binding).

If the current topic value is 2 and I press the switch I get the following log message:
2020-01-19 16:27:24.002 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Command ‘2’ not supported by type ‘OnOffValue’: No enum constant org.eclipse.smarthome.core.library.
types.OnOffType.2
2020-01-19 16:27:24.015 [vent.ItemStateChangedEvent] - KG_Wohnraum_StarPowerTest changed from ON to OFF

and the value of the topic is changing to “ON”.

The thing config looks like this:

Type switch : power [
            stateTopic = "home/1/XMAS/animation",
            transformationPattern = "MAP:star_power.map",
            commandTopic = "home/1/XMAS/animation",
            transformationPatternOut = "MAP:star_power.map",
            retained = true
        ]

rossko57 · January 19, 2020, 3:51pm

I’m not sure what that means. You mean the last message payload on the topic was 2? From earlier comment, that has been correctly mapped to your openHAB Item having state ON,yes?

Which switch pressing are we talking about, something at the device or OH UI?

I assume you’ve installed MAP transformation service, I’s expect to see a more meaningful log if not.
Your “bidirectional” map file looks reasonable to me.

That part’s easy ;you have subscribed stateTopic to the same topic as commandTopic, so everything you command in openHAB is immediately (attempted to be) reflected back in the channel state, even if it doesn’t fit.
That will happen regardless of any remote device also subscribing to or publishing to the topic.

tbnobody · January 19, 2020, 4:43pm

Thank you for your help so far.

Yes, the mapping from the broker towards openhab seems to work correct. If I publish anything to the broker (with mosquitto_pub) the value (on or off) is displayed correctly.

I am pressing the virtual button in the openhab UI.

I understood it, that the binding is working in the following way:

Value of an item is changed (e.g. button changed from ON to OFF)
Value is put through the transformationPattern method (then the value is e.g. 255)
The transformed value is published to the broker (e.g. 255 is published)
The transformed value is again received by openhab (e.g. 255 is received)
The received value is put into the transformationPatternOut method (e.g. 255 is transformed to OFF)
The received transformed value is put as new value into the item again (e.g. item is set to OFF while it was OFF)

Is this theoretical flow correct?

rossko57 · January 19, 2020, 5:24pm

Nope,

When you poke an openHAB UI it sends a command to an Item. Commands are very distinct from Item states - although for a Switch they happen to have the same names.

So,

UI “button” sends command to OH Item.
Any binding channels linked to that Item get the command
(In general, but there are exceptional ways to configure some back to front)
The MQTT binding processes this command through transformationPatternOut then publishes to commandTopic

That’s the end of that, assuming all works. Note the Item state never changed.
So here, with OFF command we should get MAPped payload 255?

But meantime, OH’s autoupdate feature may kick in (it’s active by default)
This is like a dummy binding listening for commands and making predictions about the likely resulting state. You’ll see these predictions in your events.log.
Why? Because if you poke the UI you expect a response, not a long delay while messages are exchanged with some remote device. So it’s just guesswork for the sake of speed-up appearance. Doesn’t involve MQTT at all.
Likely, your Item gets a fairly quick state update here, a guessed OFF.

Independently of all that, this channel of your happens to be subscribed to the identical topic by stateTopic. “Someone” just published to that topic (you) so the broker will send the subscribed Thing a copy of your recent message.
Now the inbound transformationPattern will be used to process the payload into something hopefully suitable for a Switch type channel i.e. ON or OFF.
Here we’d expect to get 255 and MAP/convert to OFF, probably to no real effect if autoupdate has already updated Item state to OFF

So the flow and interaction with parameters is rather different, but so far as I can see your current config should work in practice.
To help diagnosis, you might temporarily change your commandTopic to something else to break the chain and figure whether the outbound or inbound leg is at issue.
Outbound MQTT transformation was pretty broken before 2.5, but you say you have that version.

I note you’ve marked the topic to be retained at the broker, but I don’t think this plays into this sequence? It might, if the remote device is also publishing to the same topic with or without retained. Who wins there, I wonder.

tbnobody · January 19, 2020, 6:00pm

Thank you for this very detailed explanation. This make many things clearer.
I changed the thing definition to this (and performed a openhab restart):

Type switch : power [
            stateTopic = "home/1/XMAS/animation_in",
            transformationPattern = "MAP:star_power.map",
            commandTopic = "home/1/XMAS/animation_out",
            transformationPatternOut = "MAP:star_power.map",
            retained = true
        ]

The log shows the following lines:

2020-01-19 18:48:41.353 [ome.event.ItemCommandEvent] - Item 'KG_Wohnraum_StarPowerTest' received command ON
2020-01-19 18:48:41.365 [nt.ItemStatePredictedEvent] - KG_Wohnraum_StarPowerTest predicted to become ON
2020-01-19 18:48:41.398 [vent.ItemStateChangedEvent] - KG_Wohnraum_StarPowerTest changed from OFF to ON

The animation_out topic is ALWAY “ON”, even if the sent command is OFF.

If I publish anything to animation_in (using mosquito_pub) openhab reacts as expected: 255 = OFF, every other number = ON.

The following output shows the list of installed mqtt components:

openhab> bundle:list | grep MQTT
223 x Active x  80 x 1.14.0                  x openHAB MQTT Binding
232 x Active x  80 x 1.14.0                  x openHAB MQTT Transport Bundle
251 x Active x  80 x 2.5.1                   x openHAB Add-ons :: Bundles :: MQTT Broker Binding
252 x Active x  81 x 2.5.1                   x openHAB Add-ons :: Bundles :: MQTT Things and Channels
253 x Active x  82 x 2.5.1                   x openHAB Add-ons :: Bundles :: MQTT HomeAssistant Convention
254 x Active x  82 x 2.5.1                   x openHAB Add-ons :: Bundles :: MQTT Homie Convention
255 x Active x  80 x 2.5.0                   x openHAB Core :: Bundles :: MQTT Transport

rossko57 · January 19, 2020, 8:12pm

Alright, getting closer.
The MAP looks right but ineffective … or overriden.

I’m wondering if your channel has some lingering nastiness cached from messing with the on= off= stuff

I’d try making a new channel with just minimal outbound stuff on yet another topic

Type switch : powertest [
            commandTopic = "home/1/XMAS/animation_out_test",
            transformationPatternOut = "MAP:star_power.map",
        ]

You can link that to the existing Item as well

Switch KG_Wohnraum_StarPower
    "Stern Status"
    {channel="mqtt:topic:mosquitto:star:power", channel="mqtt:topic:mosquitto:star:powertest"}

The idea is to get a known “clean” topic published, with no history.
Then use mqtt-fx to see what pops out.

tbnobody · January 19, 2020, 9:10pm

I created a completly new topic

Type switch : powertest2 [
            commandTopic = "home/1/XMAS/animation_out3",
            transformationPatternOut = "MAP:star_power.map"
        ]

Added it to an item:

Switch KG_Wohnraum_StarPowerTest
	"Stern Status Test"
	{channel="mqtt:topic:mosquitto:star:powertest2"}

Toggled the switch in the UI:

2020-01-19 22:07:35.785 [ome.event.ItemCommandEvent] - Item 'KG_Wohnraum_StarPowerTest' received command ON
2020-01-19 22:07:35.800 [nt.ItemStatePredictedEvent] - KG_Wohnraum_StarPowerTest predicted to become ON
2020-01-19 22:07:35.869 [vent.ItemStateChangedEvent] - KG_Wohnraum_StarPowerTest changed from OFF to ON

And sadly the result is always the same… the value of the topic is just “ON”.

rossko57 · January 19, 2020, 9:16pm

That appears to be a binding regression then.

Does is still payload ON for an OH OFF command, as reported earlier?

Just to prove a point, make a MAP file for ON / OFF to apple/orange or something like and try that in the testing channel

tbnobody · January 19, 2020, 9:29pm

I created map file called “apple.map”

ON=apple
OFF=orange

And changed the thing file:

Type switch : powertest2 [
            commandTopic = "home/1/XMAS/animation_out3",
            transformationPatternOut = "MAP:apple.map"
        ]

Pressed the button in the UI:

2020-01-19 22:25:32.154 [ome.event.ItemCommandEvent] - Item 'KG_Wohnraum_StarPowerTest' received command OFF
2020-01-19 22:25:32.166 [nt.ItemStatePredictedEvent] - KG_Wohnraum_StarPowerTest predicted to become OFF
2020-01-19 22:25:32.168 [WARN ] [rm.AbstractFileTransformationService] - Could not transform '0' with the file 'apple.map' : Target value not found in map for '0'
2020-01-19 22:25:32.226 [vent.ItemStateChangedEvent] - KG_Wohnraum_StarPowerTest changed from ON to OFF
2020-01-19 22:25:36.807 [ome.event.ItemCommandEvent] - Item 'KG_Wohnraum_StarPowerTest' received command ON
2020-01-19 22:25:36.810 [WARN ] [rm.AbstractFileTransformationService] - Could not transform '1' with the file 'apple.map' : Target value not found in map for '1'

And now I understand it even less
Where is the 1 or 0 coming from?

rossko57 · January 19, 2020, 9:41pm

Aha … now we see what’s happening !!
Binding is “pre-transforming” an OFF command to ‘0’ etc.

With your original star_power.map that would score a “hit” of course, by chance.

Armed with this knowledge you can fix your original problem for now with both a in-map as you had before, and an out-map mapping 0/1 to whatever you actually wanted.

But this is a binding bug. My guess is the on=/off= stuff is kicking in where it isn’t wanted.

Thedannymullen · January 19, 2020, 9:49pm

I struggled with this same parameter. I am happy to see that we may be getting to the solution.

I would love to have the mapping work as expected also.

rossko57 · January 19, 2020, 10:40pm

I would bet this also affect rollershutters and dimmers.

rossko57 · January 20, 2020, 4:08pm

Okay, well we have simple test case; a Switch Item command fails to get mapped because it appears to auto transform to 0/1 before feeding to the wanted transform.

Who will raise a github issue? I am not an MQTT user. Do we know know if this worked at 2.5.0?

tbnobody · January 20, 2020, 5:55pm

Not sure of this issue also occoured in 2.5.0…
But while looking for a repo to report the issue I found this very recent commit:

Didn’t look at the source in detail but it may fix the issue.
Will see if I see a nightly build somewhere.