Migration guide MQTTv1 to v2?

So, MQTT suddenly stopped working. I check the docs and the forums - surprise, apparently, a new version of the binding is out and the old version was dropped more or less without warning.
Now I’m literally sitting in the dark here as I replaced all lighting and lots of other things with Sonoffs.

Is there any easy and to the point migration guide from MQTTv1 to MQTTv2 which I can follow without having to go through the whole OpenHAB documentation and start learning the new architecture from bottom up?

1 Like

I’m sorry to sound frustrated but you are the 100th person noticing this. Don’t update to a new version before reading the release notes! If you would have read them, you would have been pointed to a blog post explaining MQTTv2 and the differences and also that mqtt1 and mqtt2 can coexist. Just install the old binding again.

1 Like

Well, sorry to be frustrating but I am frustrated, sitting literally in a dark apartment…
I can’t seem to find the old binding. I went from daily builds to 2.4.0-stable hoping to find the old binding but the addon installer only shows me the 2.4.0 binding unless it’s under some different name which I have overlooked.

Forum search… Sigh.

Enable legacy bindings in Paper UI. How to do that is left as an exercise.

3 Likes

I actually knew that one… Just didn’t realize that it has been set to disabled. Thanks!
Interestingly enough, the 1.13 version of the binding was shown as installed when I enabled the legacy binding switch and after a restart came on immediately. Thanks @David_Graeff!
Now I also have the peace to do more reading :slight_smile:, not sitting in the dark with everything dead anymore :slight_smile:.

It took me an hour to find this forum post - getting the exact right keywords was the issue (as usual). I was running the M8 snapshot, then after updating to the final release, everything stopped. I did read the release notes here: https://github.com/openhab/openhab-distro/releases/tag/2.4.0#breaking-changes-that-require-manual-interaction-after-the-upgrade
And there is no mention of a new/changed setting causing most of your setup to break. So like the OP I was also very frustrated. However it’s working now, and this page also provides a link that promises to hopefully help me get things migrated.

I would however suggest that the page linked from the binding in the Paper UI (to the docs), then also provide a link to https://www.openhab.org/blog/2018-12-16-mqtt-arrives-in-the-modern-openhab-2-x-architecture.html - as with out this thread, I would not have known about this blog post.

Thanks.

1 Like

David, I really appreciate the work you’ve done but I have to admit the way the new mqtt-binding was introduced is imho not ok. I installed OH along with other packages using raspbian’s apt. Yes, I didn’t read the changelog for OH (do you for every single package while doing apt-get upgrade?) but I would have been expected OH telling me, that I’m running an old config along with a new binding. In my opinion this is necessary if a running binding gets (implicitely) exchanged by a newer one that is not downward compatible. At least it should have been noted on the binding page on how to reactivate the old binding. What in fact happened is: mqtt didn’t work any longer and there was no error-msg.

Maybe to you all that stuff is totally self-explaining but as I seems, this is not the case for every OH-user (like me). Best way (imho) would have been to offer v2 as new binding or asking what to do while install-process.

Don’t get me wrong, I like the new implementation and managed (beside of still having problems with mqtt-retain) after a few hours to get my whole mqtt-stuff running with v2, but the way the binding was introduced is at best “in need of improvement”.

The problem is man power. Someone would have to write a conversion script.

The developer guide forbids to put in version numbers into binding ids, so “mqtt2” is not an option, it had to be named “mqtt”. A bit of a dilemma. But I’m not feeling personally responsible, I just developed the binding. Luckily, there are not many OH 1.x bindings left, where the community would need to go through such a pain. (http2 will arrive for OH 2.5 though).

1 Like

Maybe somebody that is responsible is going to read my post :slight_smile:

I am with Torsten here. The thing is that OpenHAB isn’t just any arbitrary piece of software. It’s a home automation system with MQTT being one of the core protocols of today’s home automation. MQTT suddenly not working means, the home automation suddenly not working. Assuming that OpenHAB does take itself seriously and not just as something to toy around with, “home automation not working” is more than just a little inconvenience. I only live in an apartment but I already have all lighting and pretty much all electrical appliances and also part of the heating controlled through OpenHAB. I literally had to move furniture and change cabling just to get the desktop computer on to start diagnosing. Imagine somebody actually running his house with OpenHAB to the full extent of possibilities… We currently have -18°C here in Finland. Can you imagine the consequences of e.g. the heating being inoperable…?
In my opinion, the project should decide on how seriously it takes itself. If the project decides that it takes itself fully seriously, then I see the need to establish processes and procedures to make sure, a normal update does not break existing functionality because there’s just too much at stake for the users.
Or then the project decides that it’s just a “science fair experiment” - to show what’s possible. Then the project should reconsider the way it does it’s marketing and put a very big “experimental only - not for productive use” label on the homepage and every page of the docs.
I don’t mean to sound angry or anything like that. I’m a Linux user since 1995 and Linux is my primary desktop OS since 1996. I know how open source works, the problems with manpower, coordinating developers, communities, etc. I’m just saying that OpenHAB should maybe undergo a review of it’s objective and strategies and based on the results of those, review it’s processes and/or marketing.

1 Like

Not wishing to further fuel the frustration some users have clearly felt, but just to add another perspective - I upgraded my OH 2.4 (I am using the docker build), but did not have any of the issues with mqtt that some users have had. My system is still running the 1.x version of mqtt. This was not removed automatically by OH, nor did OH install the version 2 mqtt binding by itself.

As far as I can see OH did exactly what it should have (in my case) and my system came up and continued to run exactly as before, using the old binding.

Moreover, I have now (via PaperUI) installed the new v2 mqtt binding, and have it running alongside the v1 binding, whist I gradually migrate my numerous topics over. Again, there has been no issues in running the two versions side by side.

My sincerest thanks to @David_Graeff for all his tremendous efforts in getting us this v2 binding. I know how much effort it is not only to develop and document bindings, but then field all the support queries - many of which could probably have been avoided by reading the documentation. One thing I do agree with in the comments above (I think it was @sillyfrog) is that a link to the blog post in the new binding documentation page would have been helpful as there is a lot more detail there than in the main documentation page.

To avoid any misunderstandings, I should again add that my post is not meant in anyway to imply that users have not had difficulties with the upgrade, as clearly many have.

Can I make two constructive suggestions, based on your description of how important your home automation is to you?

  1. Create a small OpenHAB environment similar to, but separate from the devices which control your home. When a new release is published, wait at least a week (for major issues to be found and documented), then install the update in your Acceptance Testing environment and use it to understand the changes, and check the features you use meet your needs.
    For MQTT, this could be as simple as one Sonoff Basic, and a RaspberryPi running OpenHABian. You don’t need a separate network, but just be careful with configuration so test doesn’t connect to live accidentally.
    Some systems may require more hardware (e.g. a Z-Wave controller), but this is a very small price to pay for the systems reliability you desire.
  2. Review your automation systems, looking for Single Points of Failure and ways to add resilience. As an example, retaining light switches is a good idea as they give another way to turn critical lights on if something fails.
    Use your recent experience for continuous improvement - what was particularly difficult, and how could it be achieved differently?

These two points can be easily combined - e.g. if your main hardware fails, restore the backup (you do keep backups, right? :slight_smile: ) onto your Acceptance Test hardware and use it whilst the main hardware is being repaired.

I personally like RPi hardware as the steps to swap test/live are simple physical swaps of memory cards and cables, which has less fat-finger risk than remembering the address ranges and server names used for Docker/ VM instances.

Finally, I can recommend taking the time to watch presentations from the recent Smart Home Day 2018 which give insight into exactly how the project sees itself. There is also a very interesting video of David explaining his work on MQTT v2.0 binding changes.

Good advice in general. Just that I didn’t install a new release. I just ran an update - or actually, the computer did automatically. I wouldn’t install a new release without a sandbox test, or at least, without a scheduled time in which downtime wouldn’t be a problem.

Besides that, I have to say that OpenHAB is not in the center of my interest, so I don’t follow things that closely. I do run it, yes, but the center of my interest is more networking, IP communications and drones - and a bunch of non-computer-related things.
I know that people who are engaged in open-source projects often assume that every user has a focus of interest on their project and wants to be an expert but sometimes, people just wanna be users :smiley:.

I’m the same (running in Docker), and going from the M8 release to the final broke everything. Maybe you selected “Include Legacy 1.x Bindings” option earlier?

Unfortunately I have found that the v2 MQTT won’t work for me as I want some (and only some) of my MQTT messages to be retained, and it’ll only support a single configuration per port/hostname. At the moment I have a “normal” config, and a “retain” config, both to the same port/host, and just set the retain flag there. (I have an idea to just setup a CNAME to work around this, but a bit annoying really). I’m still just getting going with v2, but l have a lot of things to get installed and up and running before making the switch fully (I’ll get there, just time…)

Every channel can be configured to send retained messages separately. mqtt1 is restricted, because there you set retain for an entire broker connection.

I tried:

Bridge mqtt:broker:MQTTretain [ host="192.168.2.49", secure=false, username="***", password="***", clientID="openHAB2retain", retainMessages=true ] {
	Thing mqtt:topic:uc_deep_sleep "uC-Deep-Sleep" {
	Channels:
		Type switch: esp_sleep_mqtt "ESP8266-DeepSleep" [ stateTopic="openhab2/in/EspStandby", commandTopic="openhab2/out/EspStandby", on="Standby;", off="NoStandby;", retainMessages=true, qos=1 ]
	}

But this doesn’t work. PaperUI indicates retain for the broker but for the channel, it’s unchecked. Beside of


Provide a thing type ID and a thing ID in this format:
 <thingTypeId> <thingId>

openhab.log doesn’t show any warnings.
So question: What did I configure wrong?

Wouldn’t it be interesting to mention this in your binding documentation? Retain is listed as being a parameter of the bridge (without showing an example for it).

The docu is outdated then ^^

Just looked up the source code: https://github.com/eclipse/smarthome/blob/master/extensions/binding/org.eclipse.smarthome.binding.mqtt.generic/src/main/java/org/eclipse/smarthome/binding/mqtt/generic/internal/generic/ChannelConfig.java#L38

The configuration name is “retained”.