MQTT Migration I'm missing something (but what?)

vespaman · June 27, 2020, 3:53pm

So I have spent almost the whole day to read threads and trying to get this to work.
Like many others, I’m struggling with going from textual mqtt1 to textual mqtt2.
So the good thing, is that I know my broker is working as it should.

I thought I should start simple - just get the radon reading from my 3 radon sensors, that can be found on my broker in this path;
/house/radon/radoneye024e01 etc.

My old config:

broker.url=tcp://127.0.0.1:1883
broker.clientId=openhab

My old item (first sensor only)

Number  radon_1     "Radonnivå 1 pCu/L [%.2f]"         {mqtt="<[broker:/house/radon/radoneye024e01:state:default]"}

My new things file:

Bridge mqtt:broker:mosquitto "Mosquitto" [ host="127.0.0.1", port=1883, secure=false, clientID="openhab" ]
{
    // RadonEye1/2/3
    Thing topic radon1 "RadonEye 1" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye024e01" ]
    }

    Thing topic radon2 "RadonEye 2" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye15f4c5" ]
    }

    Thing topic radon3 "RadonEye 3" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye104432" ]
    }
}

My three things:

Number  radon_1   "Radonnivå 1 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon1:pci_l"}
Number  radon_2   "Radonnivå 2 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon2:pci_l"}
Number  radon_3   "Radonnivå 3 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon3:pci_l"}

Both broker and the three devices shows as online in UI.
(Broker shows as online even though I select any ip address, so maybe online is not online with broker, but online/available to oh.)

But: Here oh tries to tell me something - I just don’t understand it! I have a broker, it is online.

The channel link is exactly as entered in item file, so why is it not active here. (I guess the missing broker warning text is the reason.)

Restarting oh, this is all I get in the log:

17:40:07.857 [TRACE] [ng.mqtt.handler.AbstractBrokerHandler] - Subscribed org.openhab.binding.mqtt.homeassistant.internal.discovery.HomeAssistantDiscovery@1a18c528 to discovery topic homeassistant/# on broker mqtt:broker:mosquitto
17:40:07.858 [TRACE] [ng.mqtt.handler.AbstractBrokerHandler] - Subscribed org.openhab.binding.mqtt.homie.internal.discovery.Homie300Discovery@7768c16d to discovery topic +/+/$homie on broker mqtt:broker:mosquitto
17:40:07.861 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from UNINITIALIZED to INITIALIZING
17:40:07.887 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
17:40:07.889 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
17:40:07.890 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
17:40:07.902 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from INITIALIZING to OFFLINE
17:40:07.907 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
17:40:07.909 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
17:40:07.910 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
17:40:07.941 [DEBUG] [c.MqttChannelStateDescriptionProvider] - Adding state description for channel mqtt:topic:mosquitto:radon3:pcu_l
17:40:07.941 [DEBUG] [c.MqttChannelStateDescriptionProvider] - Adding state description for channel mqtt:topic:mosquitto:radon1:pcu_l
17:40:07.941 [DEBUG] [c.MqttChannelStateDescriptionProvider] - Adding state description for channel mqtt:topic:mosquitto:radon2:pcu_l
17:40:07.942 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
17:40:07.943 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
17:40:07.943 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
17:40:08.180 [INFO ] [o.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '127.0.0.1' with clientid openhab
17:40:08.186 [TRACE] [ng.mqtt.handler.AbstractBrokerHandler] - Subscribed org.openhab.binding.mqtt.homeassistant.internal.discovery.HomeAssistantDiscovery@1a18c528 to discovery topic homeassistant/# on broker mqtt:broker:mosquitto
17:40:08.187 [TRACE] [ng.mqtt.handler.AbstractBrokerHandler] - Subscribed org.openhab.binding.mqtt.homie.internal.discovery.Homie300Discovery@7768c16d to discovery topic +/+/$homie on broker mqtt:broker:mosquitto
17:40:08.366 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from OFFLINE to ONLINE
17:40:08.385 [DEBUG] [hab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:topic:mosquitto:radon2:pcu_l to topic: house/radon/radoneye15f4c5
17:40:08.387 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
17:40:08.389 [DEBUG] [hab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:topic:mosquitto:radon3:pcu_l to topic: house/radon/radoneye104432
17:40:08.391 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
17:40:08.426 [DEBUG] [hab.binding.mqtt.generic.ChannelState] - Subscribed channel mqtt:topic:mosquitto:radon1:pcu_l to topic: house/radon/radoneye024e01
17:40:08.427 [INFO ] [ome.event.ThingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE

What am I doing wrong?

rossko57 · June 27, 2020, 4:08pm

I think that’s just a fixed reminder message, whether you have a working broker or not.

Anything to see in your events.log ?

vespaman · June 27, 2020, 4:19pm

No, events.log just shows about the same information as openhab.log if I grep on mqtt:

2020-06-27 17:40:07.861 [hingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from UNINITIALIZED to INITIALIZING
2020-06-27 17:40:07.887 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
2020-06-27 17:40:07.889 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
2020-06-27 17:40:07.890 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from UNINITIALIZED to UNINITIALIZED (BRIDGE_UNINITIALIZED)
2020-06-27 17:40:07.902 [hingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from INITIALIZING to OFFLINE
2020-06-27 17:40:07.907 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
2020-06-27 17:40:07.909 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
2020-06-27 17:40:07.910 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from UNINITIALIZED (BRIDGE_UNINITIALIZED) to INITIALIZING
2020-06-27 17:40:07.942 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
2020-06-27 17:40:07.943 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
2020-06-27 17:40:07.943 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from INITIALIZING to OFFLINE (BRIDGE_OFFLINE)
2020-06-27 17:40:08.366 [hingStatusInfoChangedEvent] - 'mqtt:broker:mosquitto' changed from OFFLINE to ONLINE
2020-06-27 17:40:08.387 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon2' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
2020-06-27 17:40:08.391 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon3' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
2020-06-27 17:40:08.427 [hingStatusInfoChangedEvent] - 'mqtt:topic:mosquitto:radon1' changed from OFFLINE (BRIDGE_OFFLINE) to ONLINE
2020-06-27 17:47:29.616 [me.event.ThingUpdatedEvent] - Thing 'mqtt:topic:mosquitto:radon1' has been updated.

hafniumzinc · June 27, 2020, 4:25pm

vespaman:

My three things:

Number  radon_1   "Radonnivå 1 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon1:pci_l"}
Number  radon_2   "Radonnivå 2 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon2:pci_l"}
Number  radon_3   "Radonnivå 3 pCu/L [%.2f]"         {channel="mqtt:topic:mosquitto:radon3:pci_l"}

You shouldn’t need “:mosquitto” in your channel. Try:

Number  radon_1   "Radonnivå 1 pCu/L [%.2f]"         {channel="mqtt:topic:radon1:pci_l"}
Number  radon_2   "Radonnivå 2 pCu/L [%.2f]"         {channel="mqtt:topic:radon2:pci_l"}
Number  radon_3   "Radonnivå 3 pCu/L [%.2f]"         {channel="mqtt:topic:radon3:pci_l"}

You may also want to check the capitalisation on clientid in your broker parameters - I think it has to be all lowercase clientid, not clientID.

Lastly, depending on your version of openHAB, you may need to restart openHAB every time you change the things file, which can get very boring…!

hafniumzinc · June 27, 2020, 4:32pm

vespaman:

My new things file:

Bridge mqtt:broker:mosquitto "Mosquitto" [ host="127.0.0.1", port=1883, secure=false, clientID="openhab" ]
{
    // RadonEye1/2/3
    Thing topic radon1 "RadonEye 1" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye024e01" ]
    }

    Thing topic radon2 "RadonEye 2" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye15f4c5" ]
    }

    Thing topic radon3 "RadonEye 3" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye104432" ]
    }
}

Actually, looking at this maybe also try:

Thing definition change from:

Thing topic radon1 "RadonEye 1" {
....

to

Thing mqtt:topic:radon1 "RadonEye 1" {
....

(Do that for all three of your things, of course)

vespaman · June 27, 2020, 4:42pm

So, none ot clientid or mosquitto in item file did not make any difference.

Sorry, should have mentioned that I am on 2.5.4. The only super boring thing is that I have an upnp exception everytime I stop OH, so it takes a couple of minutes to do it… But I’m doing it regardless.

hafniumzinc · June 27, 2020, 4:43pm

And did you change your thing definition too from my second post?

vespaman · June 27, 2020, 4:47pm

Did it now, but no change unfortunately.

vespaman · June 27, 2020, 4:58pm

The thing is that I have trouble finding the root cause. I mean, I don’t know if it is the bridge or the devices that are broken. The log, even at trace, does not show anything. No new values coming in, even no error when I configure the bridge/broker to a different IP address.
Everything is just online regardless, and no errors in log.

So strange.

If I list bundles, I get:

302 │ Active │  80 │ 1.1.2                   │ hivemq-mqtt-client
307 │ Active │  80 │ 2.5.6                   │ openHAB Add-ons :: Bundles :: MQTT Broker Binding
308 │ Active │  81 │ 2.5.6                   │ openHAB Add-ons :: Bundles :: MQTT Things and Channels
309 │ Active │  82 │ 2.5.6                   │ openHAB Add-ons :: Bundles :: MQTT HomeAssistant Convention
310 │ Active │  82 │ 2.5.6                   │ openHAB Add-ons :: Bundles :: MQTT Homie Convention
311 │ Active │  80 │ 2.5.0                   │ openHAB Core :: Bundles :: MQTT Transport

Is there a package that shouldn’t be there among these, or am I missing some?

Edit:
I’m using this set-up to activate more log stuff:
log:set TRACE org.openhab.binding.mqtt
But maybe this is not correct, seeing that there are multiple bundles for MQTT?

H102 · June 27, 2020, 5:43pm

Try changing the host IP from 127.0.0.1 to the actual IP of your server. After making the change you may need to stop OH, clean the cache, then restart OH a few times.

hafniumzinc · June 27, 2020, 6:29pm

Oh my, if that fixes it that would be one annoying bug!

@vespaman I think I’ve found it - your item has the wrong channel name: pci_l vs pcu_l

Try the below:

things

Bridge mqtt:broker:mosquitto "Mosquitto" [ host="127.0.0.1", port=1883, secure=false, clientid="openhab" ]
{
    // RadonEye1/2/3
    Thing mqtt:topic:radon1 "RadonEye 1" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye024e01" ]
    }

    Thing mqtt:topic:radon2 "RadonEye 2" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye15f4c5" ]
    }

    Thing mqtt:topic:radon3 "RadonEye 3" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye104432" ]
    }
}

items

Number  radon_1   "Radonnivå 1 pCu/L [%.2f]"         {channel="mqtt:topic:radon1:pcu_l"}
Number  radon_2   "Radonnivå 2 pCu/L [%.2f]"         {channel="mqtt:topic:radon2:pcu_l"}
Number  radon_3   "Radonnivå 3 pCu/L [%.2f]"         {channel="mqtt:topic:radon3:pcu_l"}

vespaman · June 28, 2020, 9:44am

@hafniumzinc Thanks! That got me closer. How on earth did I miss this? I have stared at these lines, my eyes where getting sore.

However, still, it did not work.
But now at least the channel is indicated as ‘in use’ in paper UI on the devices. Good.

What I have learned sofar:
clientid must be clientID, otherwise binding will generate a random id (not particularly important in my case, but it may help others).

Then I tested with a ‘/’ in the beginning of the mqtt topic path row - and now it works!

The declaration syntax “Thing mqtt:topic:radon1” gives a warning when loading the things file:

10:01:56.345 [INFO ] [del.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model 'mqtt.things', using it anyway:
Provide a thing type ID and a thing ID in this format:
 <thingTypeId> <thingId>

But if I instead use “Thing topic radon1” syntax, it is silent when loading. But I then it does not work instead, so I left it to the ‘:’ syntax with the errors/warnings.
What I don’t understand, is that why this syntax works for some people and others need to use the ‘:’ syntax.

This has been a tough battle, too tough imho. I’m not complaining, but log files should provide some sort of error information, docs could have given an example etc.

For others struggling my final things file looks like this when it works:

Bridge mqtt:broker:mosquitto "Mosquitto" [ host="127.0.0.1", port=1883, secure=false, clientID="openhab" ]
{
    // RadonEye1/2/3
    Thing mqtt:topic:radon1 "RadonEye 1" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="/house/radon/radoneye024e01" ]
    }

    Thing mqtt:topic:radon2 "RadonEye 2" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="/house/radon/radoneye15f4c5" ]
    }

    Thing mqtt:topic:radon3 "RadonEye 3" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="/house/radon/radoneye104432" ]
    }
}

Thanks everyone, and especially to @hafniumzinc!
Now I have 4 other things with 230 new channels to add all in all… Maybe I’ll have some hours off from this before starting…

hafniumzinc · June 28, 2020, 9:48am

vespaman:

The declaration syntax “Thing mqtt:topic:radon1” gives a warning when loading the things file:
10:01:56.345 [INFO ] [del.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model 'mqtt.things', using it anyway:
Provide a thing type ID and a thing ID in this format:
 <thingTypeId> <thingId>

Yeah, I get this too, and just ignore it now!

And good spot on the leading “/” - I should have noticed that too! Oh well, glad it’s working!

You’re right - I’ve had it wrong this entire time!

rossko57 · June 28, 2020, 10:34am

It does tell you what to do about it. Every Thing has a type, as well as a unique ID. Some bindings support many types; MQTT generic has only Thing type topic but you should still tell it which.

Thing mqtt:topic:radon1 "RadonEye 1" {

Faulty - no type given

Thing topic radon1 "RadonEye 1" {

Good - type is given, topic
The radon1 part will be used to build a UID derived from the Bridge that “owns” this Thing, in the hierarchy that you built in your xxx.things file.
In this case that would be
mqtt:topic:mosquitto:radon1
because

it is an MQTT binding thing - not hue or zwave
it is a topic-type Thing
it “belongs” to a Bridge thing that you named mosquitto

The topic Thing UID isn’t much use to you on its own, but of course you will see it when you give your topic Thing some channels, because your channels will inherit this as part of their own UID

Bridge mqtt:broker:mosquitto "Mosquitto" [ host="127.0.0.1", port=1883, secure=false, clientID="openhab" ]
{
    // RadonEye1/2/3
    Thing topic radon1 "RadonEye 1" {
    Channels:
        Type number : pcu_l "pCu/L" [ stateTopic="house/radon/radoneye024e01" ]
    }

We can predict the UID of that channel -
mqtt:topic:mosquitto:radon1:pcu_l
but to be honest that is hard work and error prone.
Load your xxx.things file, let PaperUI show you your Things … and use the screen shot as in your very first post to copy/paste the displayed channel UID into your xxx.items file

vespaman · June 28, 2020, 11:34am

Aha, thanks for explaining!

I suppose I have tried this before, but without the exact combination of all other variables involved, it now works. And no error in log file!

hafniumzinc · June 28, 2020, 1:00pm

It looks like there’s a difference if you define the thing within the bridge itself (“owns”, as you describe) versus having the bridge defined elsewhere.

Because I have the bridge defined in a separate things file, I must use Thing mqtt:topic:radon1 (to use the example from this topic). If I use Thing topic radon1 then I get the following in the log (essentially the opposite of what @vespaman had in his):

13:57:11.153 [INFO ] [del.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model 'wled.things', using it anyway:
Provide a thing UID in this format:
 <bindingId>:<thingTypeId>:<thingId>

Apologies @vespaman, I should have taken that into account too! It’s probably worth pointing out that this is all in the docs, but the difference is so subtle in an otherwise complex topic that it’s very easy to miss!

rossko57 · June 28, 2020, 1:40pm

Yes, a topic type Thing has to be associated with Bridge thing to work.
It cannot guess, you may have several Bridges.
That can be done by nesting things in one file, or you have to tell the topic thing explicitly. That’s done using round brackets -

Thing topic mqtt:topic:radon1 "RadonEye 1" (mqtt:broker:mosquitto) {

or is it

Thing topic radon1 "RadonEye 1" (mqtt:broker:mosquitto) {

I’m not sure

hafniumzinc · June 28, 2020, 2:35pm

The first one!