[SOLVED] MQTT Mosquitto broker issue

Hi guys,

I’m trying to setup the Mosquito MQTT broker but didn’t had any luck so far.
For now, I’m using the embedded MQTT broker from Moquette but since that started to gives me some errors like:

2020-08-02 22:06:31.025 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Incoming payload ‘on’ not supported by type ‘NumberValue’

and some more netty errors, I started searching for a fix and read everywhere that the best solution is to get ride off the integrated broker since the developer isn’t contributing anymore, and start using Mosquitto for example.

So I just installed the Mosquito app from the sudo openhabian-config CLI, that gave me no errors. Then I created the /etc/openhab2/services/mqtt.cfg file as followed:

mosquitto.url=tcp://localhost:1883
mosquitto.qos=1
mosquitto.retain=true
mosquitto.async=false
mosquitto.user=user
mosquitto.pwd=pass

I changed the port number from the integrated broker to 1882 so the old devices won’t be able to connect the old broker.

Next step was creating the new broker through the PaperUI > Inbox > Manually add thing > MQTT Broker and filled all the fields like in the mqtt.cfg

The broker goes online but offline as soon as I change a thing it’s bridge to the new broker, it’s offline and not working of course.

Now I uninstalled all the Mosquito components and deleted all the files so the integrated broker is working again, did I do something wrong or is there a better way to do this?

Please don’tmix up mqtt1 and mqtt2 addon!

For mqtt2 there is no such file as mqtt.cfg. Instead, you have to setup the broker connection via Paper UI -> configuration -> Things -> + (the big blue one) -> MQTT Binding -> Add manually (or manually add Thing) -> MQTT Broker (NOT! System MQTT Broker)
Give the broker a meaningful name, e.g. Moquitto.and also set the Thing ID to a meaningful name, e.g. mosquitto.

1 Like

Hi Udo,

Thanks for the quick reply, then I already see some mistakes I made.
But the method that I attempt should work right? Or should I first uninstall the embedded broker since they may have some kind of conflict?

Only if you have the V1 MQTT binding installed. It’s recommended to use the V2 binding though - follow @Udo_Hartmann instructions to setup the MQTT broker.

By default, both the embedded broker and Mosquitto use the same port - they will fight! Only have one installed at a time!

Ah ok, I only used the v2 MQTT binding so it is going to be a big-bang migration then.
I will first make an image if the current system so I can easily roll-back if necessary.

Changing brokers will not fix this issue, by the by.
This is one of your channels that you have configured as a number type, receiving an MQTT message with text that it doesn’t understand.

Hmmm yea I saw some articles about that as well, so you should need the JSONPath Transformation, but I have it already installed.

Is there anything else you can do to fix it?

Depends what you’re trying to do.

Currently, it looks like you’re trying to force the word “on” into a Number channel. A Number channel can only receive numbers, not words (or strings), so openHAB will reject it and let you know.

If you give us more of an idea of your thing configuration, and the JSON message you’re trying to capture, we might be able to suggest something!

I’ve 2 shelly dimmers working with MQTT, they use the stock factory firmware.
So I used the API docs from shelly (that can be found here: https://shelly-api-docs.shelly.cloud/#shelly-dimmer-sl-overview) to figure out how to use MQTT with it.

I think the problem can be found in the Number value of the dimmer channel, you’ll have to add a “Outgoing Value Format” to the properties of the channel.
I’m using this value: {“turn”: “on”, “brightness”:%s}

Besides the error in the logging, it is working. When I change the brightness of the dimmer and the light is turned off, it sends the on command to it and it will turn on.
I assume that this value: {“brightness”:%s} will also work, but the lights won’t be turning on with a dimmer number change.

What will be the best solution then, use the value: {“brightness”:%s} and create a rule to turn on the lights when the number change or is there a better solution for it?

The error report is about an incoming MQTT payload. Have you set your stateTopic to the same as your commandTopic, or something?

It’s worth noting that there is a Shelly binding that doesn’t require working with MQTT. It might save a good deal of effort.

If you’ve already a bunch of MQTT 1.x Items, you can run both the old 1.x and new 2.5 binding at the same time and migrate over little by little.

If you’ve a lot of similar devices, I recommend creating and making sure one Generic MQTT Thing works. Then using the REST API docs to pull the JSON for that Thing, copy it to a text editor and change the topics and IDs as appropriate. Then use the REST API docs to add the new thing from that edited JSON.

That’s a good point, I checked it and it seems to be an state topic indeed.

For both number channels on the dimmers I used shellies/shellydimmer-/light/0 but this topic sends back the ON/OFF state, so that’s giving me the error.

But they don’t have a specific state topic that gives the value in numbers, they only have this topic: shellies/shellydimmer-/light/0/status and this contains a lot more information that I don’t need, but also the brightness in numbers, this is the information that it will get:

{
“ison”, /* whether the channel is turned ON or OFF /
“has_timer”, /
whether a timer is currently armed for this channel /
“timer_started”, /
unix timestamp of timer start; 0 if timer inactive or time not synced /
“timer_duration”, /
timer duration, s /
“timer_remaining”, /
if there is an active timer, shows seconds until timer elapses; 0 otherwise /
“mode”, /
always white /
“brightness” /
output brightness, 1..100 */
}

So I think I need to do something with “Incoming Value Transformations” on the channel properties isn’t it?

I know there is a Binding for it but I like the MQTT protocol more because it works a lot faster then with the binding.

I only have MQTT 2.x items, but it’s a good idea to use the REST API docs to pull the JSON for the similar devices, that will be much faster, thanks for the tip!

Presuming that is not the information you get, but rather a guide to interpreting it? And what you really get is a JSON packet with key-value pairs?

To extract one value using key name in MQTT channel is everyday stuff, you"d use a JSONPATH transformation.

Correct, and as @rossko57 says, use a JSONPATH transformation. Probably like:

JSONPATH:$.ison

For the on/off switch. Dimmer would be:

JSONPATH:$.brightness

Those are just guesses though - you haven’t shown us the actual JSON string published by the Shelly.

Thanks, that was the right guess :slight_smile:
I was already playing around with it but I used brackets and he didn’t seem to like that, got this error:

2020-08-03 18:41:55.774 [WARN ] [t.generic.ChannelStateTransformation] - Transformation service “JSONPATH for pattern $.brightness” not found!

2020-08-03 18:41:55.781 [WARN ] [ab.binding.mqtt.generic.ChannelState] - Incoming payload ‘{“ison”:false,“has_timer”:false,“timer_started”:0,“timer_duration”:0,“timer_remaining”:0,“mode”:“white”,“brightness”:2}’ not supported by type ‘NumberValue’

But without the brackets as you posted it, it reads the right numbers, now I’m curious if the other Netty error is also going but I’ll check the logs for that on a later moment.

Still want to do the Mosquitto migrate since I read somewhere that the broker will stop working in a later update of openhab, I don’t want that to happen and I’ll have to rush the migrate :innocent:

hmmmm, that is interesting

just want to make a comment on this. Apparently the Moquette project has not had much activity for awhile and people consider it unmaintained but the version used in OpenHAB is actually a fork maintained by our own Jan (aka J-N-K). Last time we found a bug in the code, Jan actually fixed it in our fork and me thinks if we find another bug, he’d probably be willing to take a look. For the record I also use the internal mqtt server (Moquette) and it works a treat. As soon as an issue comes up in this forum (such as this one) everybody quickly points out that Moquette is unmaintained and the advise is to switch to Mosquito. While setting it up seems easy for some, the internal mqtt server is even easier (two or three clicks) and judging from this comment…

it isn’t so easy for everybody

2 Likes

I know, it’s been working pretty good for me as well, but I won’t be happy if the feature suddenly gets removed and all my mqtt devices stops working. Because I saw this article: https://www.openhab.org/addons/integrations/mqttembeddedbroker/

And this sentence:

It is recommended to switch to an external broker like mosquitto. The Moquette add-on will be removed in a future release of openHAB.

That’s why I want to migrate, just to be ready for future updates :wink:

1 Like

I’m now on a holiday, so when I’m back I want to continue the mosquitto migration, but suddenly my logging shows errors on the moquette broker 2 times a second or so… So I’m unable to read anything on the log viewer, there is a rule that works but I’m missing out one of the val statements so I wanted to check what was going on but I’m not able to do so this way, this is the error:

2020-08-10 17:51:09.839 [ERROR] [.moquette.broker.NewNettyMQTTHandler] - Unexpected exception while processing MQTT message. Closing Netty channel. CId=DVES_E0EF11
java.nio.channels.ClosedChannelException: null
at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:607) [bundleFile:4.1.42.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.close(DefaultChannelPipeline.java:1364) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:621) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:605) [bundleFile:4.1.42.Final]
at io.netty.channel.ChannelDuplexHandler.close(ChannelDuplexHandler.java:79) [bundleFile:4.1.42.Final]
at io.moquette.broker.metrics.BytesMetricsHandler.close(BytesMetricsHandler.java:66) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:621) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:605) [bundleFile:4.1.42.Final]
at io.netty.channel.ChannelDuplexHandler.close(ChannelDuplexHandler.java:79) [bundleFile:4.1.42.Final]
at io.moquette.broker.metrics.MessageMetricsHandler.close(MessageMetricsHandler.java:65) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:621) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:605) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:467) [bundleFile:4.1.42.Final]
at io.netty.channel.DefaultChannelPipeline.close(DefaultChannelPipeline.java:969) [bundleFile:4.1.42.Final] at io.netty.channel.AbstractChannel.close(AbstractChannel.java:231) [bundleFile:4.1.42.Final]
at io.moquette.broker.NewNettyMQTTHandler.channelRead(NewNettyMQTTHandler.java:62) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.moquette.broker.metrics.MQTTMessageLogger.channelRead(MQTTMessageLogger.java:46) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.moquette.broker.metrics.MessageMetricsHandler.channelRead(MessageMetricsHandler.java:50) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328) [bundleFile:4.1.42.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:302) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.moquette.broker.metrics.BytesMetricsHandler.channelRead(BytesMetricsHandler.java:51) [bundleFile:?]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [bundleFile:4.1.42.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [bundleFile:4.1.42.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [bundleFile:4.1.42.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931) [bundleFile:4.1.42.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [bundleFile:4.1.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) [bundleFile:4.1.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635) [bundleFile:4.1.42.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552) [bundleFile:4.1.42.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) [bundleFile:4.1.42.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044) [bundleFile:4.1.42.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [bundleFile:4.1.42.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [bundleFile:4.1.42.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_262]

The device in the error identified by the CID can’t be found when I’m checking my devices. Anyone know how to fix this?

So, just a couple of days back from my holiday and just finished the MQTT Broker migration from Moquette to Mosquitto and the errors are gone, thanks you all for the help :slight_smile:

I just removed the Embedded broker: PaperUI > Add-Ons > Misc > Moquette Broker MQTT > Uninstall, checked if the config files were removed, rebooted and installed the Mosquitto broker through sudo openhabian-config. After completing the config, migrated all my things to the new broker and it is working like a charm again!

1 Like