[CLOSED] MQTT binding version 2.4 (Pre-release !)

I already did that. I made some mqtt tests some time ago with owntracks and so mosquitto was already installed and configured for external access. But now i only use it internal.

user/password is already enabled in mosquitto.conf

@David_Graeff
Ok, will open another thread if i have any further questions.

1 Like

Will I be able to use MQTT v1 ongoing or will I be forcibly migrated to v2.4 which Iā€™m a bit wary of as everything currently works really well? I do use the eventbus on MQTT.

Even better would I be able to use both?

Do not enable both.

There is no forced migration. I suspect that the MQTT 1.x version binding will still be around for as long as it continues to work. However, realize that there will likely be little to no continued development on the MQTT binding and at some point there will be a change to OH, the MQTT specification, or some library the 1.x version relies upon that will break it and there will be no one willing to spend the time to fix it.

So eventually you will want to migrate to the 2.x version of the binding. But I suspect that you will be able to continue using the 1.x version for years.

Personally, I think the 2.x version is a huge improvement over the 1.x and would encourage you to not wait until you have to migrate. Itā€™s also always less stressful and easier to migrate when you have time then it is to migrate because you have to.

If you see the discussion above between myself and David, you will see that there will be an easy way to replicate the MQTT Event Bus using a couple of simple Rules. As I continue to figure out the Experimental Next Gen Rules Engine, I expect that these will be made available as a library as the NGRE matures and we come up with a good library distribution mechanism.

The structural change between MQTT1 and MQTT2 is substantial - as indicated by the major version number. OH2 changed bindings and thus MQTT1 to MQTT2 makes that leap.

But this recommendation regarding using both MQTT1 & MQTT2 gives me pause:

I agree that one would not want that as the long term operational situation. But during migration, I sure would like to have that option.

Iā€™ve been monitoring this thread, and the migration does not look trivial. Itā€™s not just the need to go through declaring or auto-discovering MQTT Things (and channels). Then one has to link Items to the appropriate channels. As 90% of my devices use MQTT, this migration will impact a large portion of my environment. And then there is the impact to Rules. Perhaps by mistake, but my Items serve as both receive and send within a single declaration. I donā€™t use the MQTT Action (because I read of conflicts with the binding). Thus I sense that Iā€™ll end up with more Items under MQTT2 and that will, in turn, impact the Rules where these Items are used as well as the Rules logic.

That long paragraph to say, Iā€™d love to have the option to migrate my Items and Rules in small chunks to control the ensuing chaos and inevitable bugs. I just donā€™t have the OH coding expertise nor the freedom to have separate Dev/Test/Prod environments. I will end up with Dev/Test/Prod all basically on my single Prod OH environment. Yes, Iā€™ll perform backups, but Iā€™ll have ā€œoldā€ automations and migrated automations all in production on the same OH server.

Mike

I could be wrong, but Iā€™m pretty sure the two are not compatible. If you have both installed neither will work.

Unless you are using the publish action there is no impact to Rules. If you are then there is likely to be a minimal search and replace to change the publish action to what ever David is cooking up to replace that functionality.

You donā€™t really need completely separate hardware or even environments. You just need a separate instance of OH that you can turn on and off when you are developing and testing.

  1. Do a manual install of OH but stop short of having it start automatically.
  2. Put a copy of your config in the manually installed OH. This includes contents of /etc/openhab2 which goes in conf and the contents of /var/lib/openhab2 which goes in userdata.
  3. When you want to work on the migration, stop the production OH, start the manually installed OH
  4. Edit and test until you run out of time
  5. Stop the manually installed OH and start back up the production OH.
  6. Once the changes are done, migrate the changes made to the production OH.

Itā€™s not ideal (separating the dev from production would be better) but it will get you there.

One technical aspect that I have seen with all legacy 1.x and new 2.x bindings (which never work in parallel) is the config pid. Both use the same space (e.g. org.openhab.xyz ā€¦ in this case it is: org.openhab.mqtt)

I donā€™t know if it is possible (or desirable) to change this in the MQTTv2 setup to be able to store in a separate pid the config parameters. And I donā€™t know if this is the only limitation in having a legacy and a new binding working together (for sure there are more).

I was going to say I donā€™t think I understandā€¦ but I can say with complete confidence that I definitely donā€™t understand :wink: Manual installā€¦ Is this not what I did with my original openHABian install? Can I have two OH instances on the same Raspberry Pi? By this I mean, can I have two installations on the one set of media (FYI - I moved the root to a USB thumb drive)?

I have not taken the container plungeā€¦ which I guess would allow me to spin up the Prod container or the Dev/Test container. Perhaps my first question ought to beā€¦ how can I take my existing Prod environment and make a container of it?

Mike

1 Like

Guys, you are in a lucky position here. MQTT 1.x and MQTT 2.x will work independently
and will not influence each other. It just costs double memory and two configuration files.

They use different service PIDs (org.openhab.mqtt vs org.eclipse.mqtt).

Cheers, David

5 Likes

No. See openHAB on Linux | openHAB

Yes if one is apt installed, which is what openHABian does, and the other is manually installed. And if you keep the default settings, they canā€™t run at the same time, though you probably donā€™t want that anyway.

Copy your existing /etc/openhab2 and /var/lib/openhab2 folders somewhere like /opt/openhab2/conf and /opt/openhab2/userdata respectively. Then mount theses folders into the container for conf and userdata as documented in the openhab docker image readme on dockerhub.

Thatā€™s excellent !!! This will help alooot with the migration from v1 to v2 on production environments !!!

We can gradually add new Items bound to v2 while the existing v1 Items still operate and then we can gradually decommission v1 Items when we verify proper operations of v2 Items (in updates, rules, etc) !!!

When our OH2 system will be ā€œcleanā€ of v1 Items, we can then remove MQTTv1 binding and its configs all together (or even keep it as a backup)

Thatā€™s a first :slight_smile: :+1:

3 Likes

Well, not critical anymoreā€¦ but still very good to know. Iā€™ll definitely bookmark this!

Hi @David_Graeff, Iā€™ve been testing the flag to interpret the MQTT topic value as a command instead of a state update.

Unfortunately this still doesnā€™t lead to the desired behaviour, however I think the issue may now be outside of the binding. I can see that the item linked to the MQTT channel receives the command, the issue is that this command is not sent to the Yeelight channel.

Another issue Iā€™ve found is that none of the channel types seem to accept INCREASE and DECREASE commands.

Iā€™m wondering if we need @Kai or someoneā€™s help to refine how we model things like remotes and other devices that can send commands via MQTT, or maybe some guidance around Items and profiles etc.

The API documentation of the postCommand function that I used is not exactly illuminating. There is one other way. Sending the command directly to the eventbus. There is no written rule but I guess bindings shouldnā€™t interfere with the eventbus directly. When Iā€™ve got some time Iā€™ll have a look.

@David_Graeff

Now i am on the latest 2.4 again (1434) and hav done some investigation.

1.) I rebuild all generic items. Channel Id with all lower caps
2.) If fresh created the Things goes online and works well.
3.) After restarting OH mqtt Things are offline with the message ā€œUninitializedā€ in Paper UI
4.) If i open the thing the error is more specific ā€œUninitialized - Handler Configuration pendingā€
5.) If i delete a channel the thing goes immediatly online
6.) After recreating the deleted Channel everything is fine until i restart openHAB again

Any ideas on that? in the log file i didnt find anything

Same with M6

I had that happen too. Make sure there are no errors, any red underlining or missing required fields in the Channel Things. For instance it seems most things need an inbound topic, even if there isnā€™t one. Just fill it in with anything.

1 Like

There where no error, but i defined a switch for an unreach state, that has only a state value, because it should be read only an therfore no command ist entered. The first Item is workung now. I will see what happens over the next days

Thanks for the feedback

@David_Graeff I hope you can fix it!
Maybe the payload is a string and not yet converted into an int/float
The error from the console (last milistone build):

2018-11-22 19:05:26.501 [WARN ] [eneric.internal.generic.ChannelState] - Incoming payload '30' not supported by type 'NumberValue'
java.lang.IllegalArgumentException: Value must be between 0 and 100
	at org.eclipse.smarthome.core.library.types.PercentType.validateValue(PercentType.java:58) ~[?:?]
	at org.eclipse.smarthome.core.library.types.PercentType.<init>(PercentType.java:53) ~[?:?]
	at org.eclipse.smarthome.core.library.types.DecimalType.as(DecimalType.java:160) ~[?:?]
	at org.eclipse.smarthome.core.internal.items.ItemStateConverterImpl.convertToAcceptedState(ItemStateConverterImpl.java:58) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.profiles.ProfileCallbackImpl.sendUpdate(ProfileCallbackImpl.java:135) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.profiles.SystemDefaultProfile.onStateUpdateFromHandler(SystemDefaultProfile.java:54) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.CommunicationManager.lambda$9(CommunicationManager.java:477) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.CommunicationManager.lambda$11(CommunicationManager.java:497) ~[?:?]
	at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.CommunicationManager.handleCallFromHandler(CommunicationManager.java:493) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.CommunicationManager.stateUpdated(CommunicationManager.java:475) ~[?:?]
	at org.eclipse.smarthome.core.thing.internal.ThingManagerImpl$1.stateUpdated(ThingManagerImpl.java:167) ~[?:?]
	at org.eclipse.smarthome.core.thing.binding.BaseThingHandler.updateState(BaseThingHandler.java:287) ~[?:?]
	at org.eclipse.smarthome.binding.mqtt.generic.internal.handler.AbstractMQTTThingHandler.updateChannelState(AbstractMQTTThingHandler.java:222) ~[?:?]
	at org.eclipse.smarthome.binding.mqtt.generic.internal.generic.ChannelState.processMessage(ChannelState.java:148) ~[?:?]
	at org.eclipse.smarthome.io.transport.mqtt.internal.ClientCallback.lambda$3(ClientCallback.java:90) ~[?:?]
	at java.util.ArrayList.forEach(ArrayList.java:1257) [?:?]
	at org.eclipse.smarthome.io.transport.mqtt.internal.ClientCallback.messageArrived(ClientCallback.java:90) [201:org.eclipse.smarthome.io.transport.mqtt:0.10.0.oh240M6]
	at org.eclipse.paho.client.mqttv3.internal.CommsCallback.handleMessage(CommsCallback.java:354) [198:org.eclipse.paho.client.mqttv3:1.0.2]
	at org.eclipse.paho.client.mqttv3.internal.CommsCallback.run(CommsCallback.java:162) [198:org.eclipse.paho.client.mqttv3:1.0.2]
	at java.lang.Thread.run(Thread.java:748) [?:?]

The payload is ALWAYS a string. Thatā€™s the only thing that MQTT can send. Strings.
So everything has to be converted to a String before being sent to the broker.
I donā€™t know how to fix that. I just explained that bit.

Well oke, 0 is the only payload value that works!

I beg to differ. AFAIK the specification allows the payload to be binary data. A specific implementation and/or convention (e.g. ā€œHomieā€), on the other hand, may further restrict this to only allow strings (tyically UTF-8).