Things files with MQTT 2.4

fkhairallah · December 25, 2018, 8:13pm

Spent a frustrating couple of days fighting with trying to setup a Things file with MQTT 2.4 broker.

I’m trying to connect to a Mitsubishi heatpump using the embedded MQTT broker. The documentation for creating the ‘things’ file is contradictory at times – below is the configuration I’m working with.

Clean install of stable 2.4.0 on RPi. Nothing else going on in the system.

I get the error message below everytime a topic gets updated.

Help!

The simplified Things files is:
Bridge mqtt:broker:HeatpumpMqttBroker “Heatpump MQTT Broker” @ “living” [
host=“localhost”, qos=1,retain=false

]
{
Thing topic living_heatpump “Living Heatpump” @ “living” {
Channels:
Type number : roomTemperature “Room Temperature” [stateTopic=“heatpump/living/status”, transformationPattern=“JSONPATH:$.roomTemperature” ]
Type string : compressor “Compressor” [stateTopic=“heatpump/living/status” ]
}

Items:
String firstfloor_hp_operating “1F heat pump operating” (gSensors, gPersistEvery1Minute) { channel = “mqtt:topic:living_heatpump:compressor” }
Number living_hp_room_temp “Living room Temperature [%.0f F]” <temperature> (gTemperature, gLiving, gPersistEvery1Minute) { channel=“mqtt:topic:living_heatpump:roomTemperature” }

Generates the following error message:
-0-
2018-12-25 15:00:32.174 [INFO ] [el.core.internal.ModelRepositoryImpl] - Refreshing model ‘heatpump.things’
2018-12-25 15:00:32.207 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to ‘localhost’ with clientid paho92643800603500 and file store ‘C:\Users\fady\Desktop\OPENHA~1.0\userdata\mqtt\localhost’
2018-12-25 15:00:32.214 [WARN ] [moquette.spi.impl.SessionsRepository] - Session does not exist. CId=paho92643800603500
2018-12-25 15:00:32.215 [WARN ] [moquette.spi.impl.SessionsRepository] - Session does not exist. CId=paho92643800603500
2018-12-25 15:00:37.011 [WARN ] [moquette.spi.impl.SessionsRepository] - Session does not exist. CId=d02e4a9f-91b7-4d3d-9676-bea1c3c129a9
2018-12-25 15:00:37.013 [WARN ] [moquette.spi.impl.SessionsRepository] - Session does not exist. CId=d02e4a9f-91b7-4d3d-9676-bea1c3c129a9
2018-12-25 15:00:37.070 [ERROR] [quette.server.netty.NettyMQTTHandler] - Unexpected exception while processing MQTT message. Closing Netty channel. CId=d02e4a9f-91b7-4d3d-9676-bea1c3c129a9
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.read0(Native Method) ~[?:?]
at sun.nio.ch.SocketDispatcher.read(Unknown Source) ~[?:?]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source) ~[?:?]
at sun.nio.ch.IOUtil.read(Unknown Source) ~[?:?]
at sun.nio.ch.SocketChannelImpl.read(Unknown Source) ~[?:?]
at io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:288) ~[235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1108) ~[235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:345) ~[235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [235:org.eclipse.smarthome.io.mqttembeddedbroker:0.10.0.oh240]
at java.lang.Thread.run(Unknown Source) [?:?]

opus · December 25, 2018, 8:38pm

Are the broker and things ONLINE? I didn’t spot a reason why not, besides the missing username and password for the broker, but you could have setup yours like that.

The channel definition of your items do miss the "HeatpumpMqttBroker " between topic and the thing-name.

fkhairallah · December 25, 2018, 8:56pm

The broker is configured with no security and I’m able to post to it from another client. The post generates the error message so something it taking place

The Thing syntax is where I’m getting confused. Some documentation says I should use mqtt:topic:broker:thing but that doesn’t work when enclosed inside a Bridge definition.

opus · December 25, 2018, 9:01pm

That syntax does work if the things (broker and thing) are defined via PaperUI, when defining them via file ( as you did) you need to specify the broker. You can verify that on your PaperUI, it will show the actually needed syntax of your channel in both cases!

fkhairallah · December 25, 2018, 9:51pm

Opus – my home has 6 heat pumps so I thought a file approach will be more efficient - especially as I go through multiple iteration in the development phase. (I also plan on creating a binding for the Mistu heat pump later on).

I would love to use the single PaperUI defined broker, but haven’t been able to get the syntax right. So I defined a Bridge and here we are.

Would love to know how to ‘use the paperUI to show you the right syntax’?
I can define a Generic MQTT thing link it to the broker, then what?

opus · December 25, 2018, 11:33pm

If you have defined a broker and a device-thing via file, they will show up on PaperUI. Hopefully online!
When looking into the device-thing you will see the channels created and underneath each channel the needed syntax for it.

fkhairallah · December 25, 2018, 11:43pm

I did a bit more digging and it seems that the system creates a second MQTT client (Bridge?) at restart and the two clients are conflicting?

What is the syntax to define an existing bridge for a Things file? I saw
Thing mqtt:topic:myTopic (mqtt:broker:theBroker) {….} somewhere in the documentation but VSCode does not seem to like it.

Again I am asking for how to define a Thing in a separate Thing file using an existing broker.

hakan · December 25, 2018, 11:53pm

The following is a small snippet from my demo setup, showing how you can split the broker definition from the generic thing:

first, the bridge.things file:

Bridge mqtt:broker:hotti "MQTT Broker: Hottis Mosquitto"
[
    host="my-mqtt-host",
    port=1883,
    secure="AUTO",
    username="my-username",
    password="my-password"
]

next, the clock.things file

Thing mqtt:topic:hotti:hotticlock_one  "MQTT Thing: HottiClock One"
     (mqtt:broker:hotti) @ "Living Room"
{
    Channels:
        Type string : raw "Raw"
            [
                stateTopic = "fablab02/CLOCK",
                commandTopic = "fablab02/CLOCK"
            ]
}

and lastly my items file:

String HottiClock_One_RAW "Hotticlock One RAW Data [%s]"
(gMQTT, gHottiClock)
{
    channel = "mqtt:topic:hotti:hotticlock_one:raw"
}

Hope this helps a little bit.

fkhairallah · December 25, 2018, 11:59pm

hakan - it helps a lot. I was getting a parsing error (EOF) when I used this syntax:

Thing mqtt:topic:living_heatpump (mqtt:broker:HeatpumpMqttBroker) “Living Heatpump” @ “living” {…}

Your works and the file now parses correctly. However, the original error message persists. I will remove my own broker definition and try to use one defined in the paperUI to see what happens.

fkhairallah · December 26, 2018, 12:05am

Seems to work with defining the Broker in the paperUI and then ‘coding’ it in the files using hakan’s syntax.

I understand that a single MQTT client exists for server/port combo, but I made sure the only client was the one I defined in the bridge and that triggered the error.

Is this an issue that should be reported?

David_Graeff · December 26, 2018, 12:22am

Do you have mqtt1 installed as well?
Did you configure a broker via a mqtt.cfg file for mqtt2?

In both cases an unwanted broker connection could exist.

That looks serious. Are you publishing binary data via mqtt? I’m not sure if the embedded broker can handle that.

And you are running on windows. I haven’t really tested the embedded broker on windows. Might as well be a cause.

fkhairallah · December 26, 2018, 12:26am

Fresh install on windows. Did nothing with MQTT v1.x
Mqtt.cfg empty.

No binary data. Simple JSON ({roomTemperature:64,compressor:ON,mode:‘HEAT’})

The same problem existed on an RPi instance, but I can’t be 100% sure since I was trying many things at the same time.

opus · December 26, 2018, 12:30am

@hakan:
You are using a thing syntax that I haven’t come around. What is the meaning of “( mqtt:broker:hotti)”? The bridge that this thing is connected to?
Where can I find that syntax documentated?

David_Graeff · December 26, 2018, 12:34am

Are you using MQTTLens? (Because of this bug report: https://github.com/andsel/moquette/issues/404)

fkhairallah · December 26, 2018, 12:40am

No MQTTLens I used mosquito_sub

David_Graeff · December 26, 2018, 12:47am

Because the embedded broker works for me on Linux and for another developer on MacOS, it seems to be that Windows is the culprit here. A quick workaround is to use an external broker like Mosquitto.

fkhairallah · December 26, 2018, 11:55am

As I said earlier, tried and failed on RPi. But will try Mosquito to see if I get different results.

opus · August 28, 2019, 10:47am

Are you aware that you answered a post from 8 month ago?
Posting an example which is using the embedded broker and stating that you can’t get it to work with the embedded broker sounds weird.

RiotMode · November 24, 2019, 4:10am

how can i put more than one “thing” in my mqtt.thing file, without breaking something ?
i have a working setup with 1 thing and 2 channels , but as soon as i put another thing in it, the whole stuff breaks.

opus · November 24, 2019, 6:17am

Which version are you on?
It sounds as if you are on 2.4, for this one that is a known bug, you have the restart the mqtt bundle or switch to a milestone version . Or change the setup from .things file to PaperUI.