MQTT Thing always offline

Hello all, I did an upgrade of openhabian on a Raspi 4 from 4.1.1 to 4.3.1 and figured out that one thing (thermostat_data) of an EMS-ESP heatpump gateway is permanently offline. All other things of the same device are working properly. Data are connected via MQTT, auto discovered via Home Assistant. In 4.1.1 this problem didn’t exists. I tried several options including updating the firmware of the gateway, upgrade to 4.3.1 and finally did a new openhabian installation with a 64 bit image, added just MQTT binding and added the scanned things. Same here, thermostat_data is offline. If I connect items, they are all updated. So communication is working. After a restart items are all or partly disconnected. Only warning message in the log is
“[WARN ] [core.thing.internal.ThingManagerImpl] - Could not normalize configuration for ‘mqtt:homeassistant:MQTT_Broker:WP_ems_2Desp2_2Dthermostat’ because the thing type was not found in registry.”
When using the generic MQTT thing option, there is no problem, but it is a lot of manual work to add all channels to the thing.
Is there someone else who had this problem? How can it be fixed?

Furthermore I’ve seen that all linked items disapear in the thing view when pressing the save butten and this also with online things. They are not really gone, after a a reboot they appear again. Anyway it is confusing after linking dozens of items to channels and after saving, they disapear from being linked to thing channel.

From my experience, after many trial-and-error attempts, I have reverted from configuration via the GUI back to configuring through thing and item files.

This is especially true for the MQTT binding. While it does have various auto-discovery functions, such as the Home Assistant auto-discovery in my case, it makes too many assumptions or errors, leading to non-functional things. Once adopted via auto-discovery, there is no way to manually edit or change them.

That’s why I exclusively configure my MQTT and KNX things and items via files. The advantage here is that I can fix identified errors “in batch”, so to speak.

When updating a model via files, the old items are cleanly removed and redefined from scratch. This ensures that the currently effective model is properly defined and free of “leftovers”.

If units or data types have been changed, restarting the MQTT broker further helps to update all topics.

One last hint. Do not activate persistence for your items, until you have reached a “production ready” state. This is due to the fact, that changing a data type quantity of your things will break your persistence database. While in theory it is possible to migrate data in the persistence for changed items, it is still a topic for experts on databases and needs manual intervention using native database tools.

First off: how do you (or your EMS-ESP heatpump) determine, it’s online? Are you talking about the Thing status? or do you mean the Device Available Payload?

He already told us, indirectly, through his error message. The message says that the Homeassistant auto discovery went wrong. The MQTT binding could not determine the type of device and therefore does not know how to create the topics, especially the availability topics.

As I already wrote, there is no way to edit or update the device after being auto discovered. For a lot of devices in my arsenal, there is at least one channel, that cannot be used with Homeassistant auto discovery.

Hence that is, why I suggest to go back to file based Thing and Item definitions.

ah. ok. I don’t use auto-discovery, but UI for all my MQTT-things. works fine.

This is fine for your first five to ten items. But if you have a medium sized smarthome with around 100 to 300 Channels and as much items, you do not want to do that over GUI. Especially not if you did an error and have to go through 20 items in the second or third layer (item → edit → links) to correct this.

1 Like

*cough *
I’ve got more than 1000 topics with some smarthome devices sending JSON with multiple items in them. It’s still manageable and I don’t ever want to go back to text-configuration. I had multiple errors back then and the advantages don’t benefit the disadvantages to find a simple typo within loads of text items. GUI makes it more comfortable.
given, we still lack mass manipulation over a bunch of similar items in the GUI. on the weekend I re-iterated manually over a bunch of temperature items and channels to make sure, everyone has the correct UoMs connected. But still: I still don’t think it would’ve saved me time to do this via text. I’ll bet, I would’ve gotten loads of compiling errors with me digging through textual definitions and logs… :wink:

Fair enough. I had a very different experience.

My experience is, that the GUI does its best to help keep configuration errors at a minimum.
You still have to “translate” documentation into “how do I insert this into the GUI”, but it’s worth doing so. What did I lose on time and temper dealing with loads of textual configuration and then it was either a missing Linux coding or a simple typo or some overseen link reference or whatever.
…and that coming from an oldschool “black screen editor” guy :wink:

I’m glad, openHAB still supports both textual and GUI configuration, but especially the more fail-proof configuration in the GUI makes really sense for setting up your environment. and since a few releases back, you can also completely backup your installation without any problem and migratre it into a vanilla installation, which was the last thing I needed for abondoning textual configuration as a whole. But yeah - different setups different opinions.

Thanks Frank and Thomas for your argumentation. I decided to go with the Generic MQTT Thing solution. It didn’t take too long. I already had Persistance configured for a year, so I can’t give up on it. Now the “%” numbers have a different scaling, but that can be compensated for in the Grafana dashboard.
I started with file based items and things and really like the fast bulk processing. But the UI solution isn’t too bad either. I have some earlier development in files and newer UI based. Let’s see what happens next.

1 Like

You can also mix file based and GUI based.
Just be aware you can’t change file based in the GUI.

I know that, I did it in parallel for several years.

1 Like

I have my Zigbee devices coming over zigbee2mqtt. Most of the values are coming without unit. A payload might look something like that.

{
  "battery" : 90,
  "humidity" : 53.86,
  "last_seen" : "2025-01-15T17:39:31+00:00",
  "linkquality" : 48,
  "power_outage_count" : 7,
  "pressure" : 1000.7,
  "temperature" : 19.58,
  "voltage" : 2985
}

For that reason I set the unit in the Thing definition. It looks something like that:

Thing topic zigbee2mqtt_kinderzimmer_1_feuchtigkeitssensor "Kinderzimmer 1, Feuchtigkeitssensor" @ "Kinderzimmer 1" [
  availabilityTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor/availability",
  payloadAvailable="online",
  payloadNotAvailable="offline",
  transformationPattern="JSONPATH($.state)"
] {
  Channels:
    Type number : batteryCharge "Verbleibende Batterieladung" [
      stateTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor",
      transformationPattern="JSONPATH:$.battery∩JS:|input + ' %'"
    ]

    Type number : batteryVoltage "Batteriespannung" [
      stateTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor",
      transformationPattern="JSONPATH:$.voltage",
      unit="mV"
    ]

    Type number : humidity "Relative Luftfeuchtigkeit" [
      stateTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor",
      transformationPattern="JSONPATH:$.humidity∩JS:|input + ' %'"
    ]

    Type number : barometricPressure "Luftdruck auf Meereshöhe" [
      stateTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor",
      transformationPattern="JSONPATH:$.pressure",
      unit="hPa"
    ]

    Type number : temperature "Temperatur" [
      stateTopic="zigbee2mqtt/kinderzimmer 1/feuchtigkeitssensor",
      transformationPattern="JSONPATH:$.temperature",
      unit="°C"
    ]
}

Whereas this works for pyhsical units pretty well, the dimensionless do not. For that reason I apply a transformation pattern, that adds a percent sign to the value using this syntax JSONPATH:$.battery∩JS:|input + ' %'. This will concat the two transformations to a single result.

The definition of items is then pretty forward with the exception of the percentage value:

// Equipment Feuchtigkeitssensor
Group gHallway2_eqHumiditySensor "Feuchtigkeitssensor" <if:meteocons:humidity> (gHallway2) [ HVAC, Heating ]

    // Batteriestatus
    Number:Dimensionless gHallway2_eqHumiditySensor_itBatteryCharge "Batterieladung [%.0f %unit%]" <oh:battery> (gHallway2_eqHumiditySensor) [ Measurement, Status ] {
        channel="mqtt:topic:smarthome:zigbee2mqtt_diele_2_feuchtigkeitssensor:batteryCharge",
        unit="%"
    }

    Number:ElectricPotential gHallway2_eqHumiditySensor_itBatteryPotential "Batteriespannung [%.1f %unit%]" <oh:battery> (gHallway2_eqHumiditySensor) [ Measurement, Status ] {
        channel="mqtt:topic:smarthome:zigbee2mqtt_diele_2_feuchtigkeitssensor:batteryVoltage"
    }

    // Luftdruck

    Number:Pressure gHallway2_eqHumiditySensor_itPressure "Luftdruck [%.0f %unit%]" <if:mdi:barometer> (gHallway2_eqHumiditySensor) [ Measurement, Pressure ] {
        channel="mqtt:topic:smarthome:zigbee2mqtt_diele_2_feuchtigkeitssensor:barometricPressure"
    }

    // Luftfeuchtigkeit

    Number:Dimensionless gHallway2_eqHumiditySensor_itHumidity "Luftfeuchtigkeit [%.0f %unit%]" <if:meteocons:humidity> (gHallway2_eqHumiditySensor) [ Measurement, Humidity ] {
        channel="mqtt:topic:smarthome:zigbee2mqtt_diele_2_feuchtigkeitssensor:humidity",
        unit="%"
    }

    // Temperatur

    Number:Temperature gHallway2_eqHumiditySensor_itTemperature "Temperatur [%.1f %unit%]" <oh:temperature> (gHallway2_eqHumiditySensor) [ Measurement, Temperature ] {
        channel="mqtt:topic:smarthome:zigbee2mqtt_diele_2_feuchtigkeitssensor:temperature"
    }

That feels a bit over-engineered! :wink:
It’s only neccessary to define the units at the item and “know” the unit, the mqtt-message sends.
Two examples, starting with the easy one:

mqtt-channel for outside temp:

  - id: WetterstationTempOutside
    channelTypeUID: mqtt:number
    label: Außentemperatur
    configuration:
      postCommand: false
      min: -30
      stateTopic: devices/Wetterstation/tempOutsideC
      max: 60
      unit: °C // optional

bonus: you can optionally define the unit of an mqtt-channel (last line)

the topic sends already Celsius, so I just define at the linked item:


which then leads to:

now for the more “complicated” one, this time a channel of my solar inverter for the current PV power:

  - id: powerDC1
    channelTypeUID: mqtt:number
    label: PV-Erzeugung PV-String1
    description: 260- Power DC1
    configuration:
      stateTopic: devices/Kostal/powerDC1
      unit: W

this time the unit at the channel is important, because I’d like to show it in openHAB as kW, not W.
(I could have changed the unit in my Node-Red flow, which reads out the inverter and sends seperate mqtt-topics for each information, but I leave the unit as is)

on item level you can then simply configure kW:

which in turn leads to using kW as the unit for the item, even if the value is sent unitless, but in W.

and it even works that same way with dimensionless values like percentages of SoC.
channel:

  - id: batteryStatus
    channelTypeUID: mqtt:number
    label: batteryStatus
    description: Fahrzeug Status
    configuration:
      stateTopic: devices/KiaEV6/batteryStatus
      unit: "%"

item: