MQTT Generic - retained flag not working

Hello,

I am having issues with “retained” flag in mqtt. Even if i set retained=true in the config file messages are not send with retain flag.

My Thing:

Thing mqtt:topic:arduinoBoiler "Arduino boiler" (mqtt:broker:local)
{
    Channels:
        Type number : targetTemp "Target temperature" [ stateTopic="arduino-boiler/out/target", commandTopic="arduino-boiler/in/target", formatBeforePublish="%.0f", retained=true ]
}

When I send command to the linked item message gets published - however without the retain bit set.
I have also noticed, that when i display the thing config code in paper ui, the retained flag is set to false. However retainMessage flag is set to true. No idea what that is.

I have tried using retained="true", retained=1 in the config file, but nothing seems to work.
Any ideas what am I doing wrong please?

Seems like restarting whole OpenHab resolved the issue, but that should not be necessary. :confused:

It’s a known limitation with using .things files. OH doesn’t always pick up changes to those files until a restart of OH. There have been some recent changes to the file watcher in OH 4 so perhaps this will be addressed. But as long as you use .things file you’ll have to remember to restart OH to pick up changes. Or use managed Things.

Hello!
My saving of an item only works when the commandTopic topic is changed from the sitemap, but when changed from outside, for example, using my ESP through stateTopic, the element in the sitemap changes, but when the ESP is rebooted, the old state appears in the saved topic. Accordingly, the light on this element is not restored, although it remains on on the sitemap. In the logs, it makes no difference which element changes position

2024-01-15 18:02:33.340 [INFO ] [openhab.event.ItemCommandEvent      ] - Item 'lightHallway' received command ON
2024-01-15 18:02:33.344 [INFO ] [penhab.event.ItemStatePredictedEvent] - Item 'lightHallway' predicted to become ON
2024-01-15 18:02:33.350 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'lightHallway' changed from OFF to ON
2024-01-15 18:02:33.353 [INFO ] [hab.event.GroupItemStateChangedEvent] - Item 'gLightHouse' changed from OFF to ON through lightHallway

openHAB 4.0.4
my settings in the thing file

Type switch : technical "Light technical room" [
            stateTopic = "home/esp/Lolin32main/relay/3",
            commandTopic = "home/esp/Lolin32main/control/3",
            on = "1",
            off = "0",
            retained = true
        ]

Is it possible to somehow write it state to DB mqtt via stateTopic?

It’s unclear what is and is not working here. But it is the message publisher that controls the retained flag. Any message published to the commandTopic from OH will be retained (with the current configuration) which is often not what you want. In order for the messages published to the stateTopic to be retained, you have to configure the ESP device to publish with the retained flag set.

I’m confused about how you’re setting things up, but here’s a summary of how I do it:

  1. The ESP should save its own state, if you want it to restore (e.g. light on/off) after it has rebooted.
  2. The ESP should publish its current state via MQTT. openHAB will receive this and update its Item state accordingly
  3. When openHAB commands the item, it publishes that command to your ESP
  4. Your ESP then changes its state (e.g. turn light on, or off)
  5. Back to step 2

Your openHAB Item should be created with autoupdate=false so that it doesn’t update the item’s state automatically when it commands the item. Instead, the item’s state is only updated when it receives the actual status update from the ESP via MQTT.

In this scenario, retained flag is not needed, nor desired.

Thanks Rich for the quick response! but the retain=true flag
In my esp I use AsyncMqttClient

uint16_t AsyncMqttClient::publish(const char *topic, uint8_t qos, bool retain, const char *payload = (const char *)nullptr, size_t length = 0U, bool dup = false, uint16_t message_id = (uint16_t)0U)

my publication

mqttClient.publish(fullTopic.c_str(), 0, true, (char*) message.c_str());

but when changing my thing using stateTopic, the recording does not happen

Thank you for your implementation option, but it seems to me that it would be more correct to store the states of your things using openhub. Moreover, if both devices are completely rebooted, the ESP will boot up much faster and will have time to perform actions without oenhub knowing, but what should you do in this case?

No no message at all is published? Then there is something going wrong on the ESP preventing it from publishing or there is a typo in the topic that openHAB subscribes to.

Why? The ESP is the device that is the closest to the truth.

You’d still want to keep the messages retained for state updates. The general rule of thumb is states should be retained, commands should not be retained. But everything else in @JimT’s approach is spot on.

A less good alternative would be for the ESP to periodically publish it’s state and an even less good approach would be to implement a command/response message OH would send that causes the ESP to republish it’s current states.

MQTT 5 added a field to set an expiration for retained messages. So if it’s not replenished after the supplied timeout the retained message self destructs. You can see if your MQTT library on the ESP supports that.

everything (almost!)) works fine, I implemented a code in my ESP that when one contact changes, it changes the state of the relay and sends the state to stateTopic.
Algorithm 1 - openhub commandTopic changes this state; everything is recorded and restored when power is applied to both devices.
Algorithm 2 - the esp itself changes the state and informs openhub using stateTopic.
also works, the state in sitemap is turned on, exactly the same actions occur in the log as with algorithm 1
but recovery does not occur on reboot; the ESP returns to the old state, and in sitemap the switch is on :frowning:

A number of things can be happening then. Most likely the ESP isn’t publishing it’s state when it comes back online so there’s no way OH would know about it. Alternatively is’t possible that OH is coming back online after the ESP has already come online but the ESP published it’s new state without the retained flag so OH missed it.

In either case, you’ll need to adjust the ESP to publish it’s state when it comes online for the first time and when it publishes it’s state it needs to publish it as retained.

If the Item isn’t updating in OH, it’s because there’s no MQTT message telling it to.

Rich no, it’s not like that))
I have two relays.
I turn on one relay using openhab, commandTopic sent the ESP and it worked.
I turned on the second relay using the ESP switch itself, and sent stateTopic, everything worked too.
There are the same messages in the logs. the states in the sitemap are set correctly, mapdb is also saved, but when the ESP is rebooted, only switch No. 1 will be restored

Right, meaning when the ESP is rebooted, it doesn’t publish a message with it’s current state.

The ESP returns to it’s old state. It doesn’t publish a message indicating it’s returned to the old state. Therefore OH doesn’t know that it’s returned to the old state.

No. When loading the ESP, all relay are turned off. When ESP connect to the MQTT, I see that it receives all the states, but relay 1 right on and relay 2 does not. although before the reboot ESP it sent and changed states.
My switches are buttons, they have no state. I don’t store states in the ESP and when connected it doesn’t send anything.
MQTT does not save the states of an element modified using stateTopic

It’s completely incomprehensible how this is set up.

But no matter how it’s set up, if the ESP doesn’t send a message, OH isn 't going to change anything. No message, nothing happens.

You’ve stated that the OH does receive messages from the ESP when you activate the device on the ESP.

Therefore:

  1. the ESP isn’t sending a message when it boots to indicate it’s been activated
  2. the ESP is sending a message when it boots but it’s to a different topic

If you want an Item to change state in OH when the ESP boots, there must be a message sent from the ESP to make that happen. No message and OH doesn’t know anything.

If there is no state why do you have any Items in the first place? Use an Event Channel. If you do use an Item, why do you care what the state is if the relay doesn’t have a state? It doesn’t matter is the Item stays ON because that ON state is meaningless anyway. The buttons the Item represents have no state.

You can’t have it both ways.

Keep State No State
The ESP must keep track of state and report all state changes, even during boot No Items at all, use an event Channel, or if you do use Items, ignore the Item’s state, they don’t matter.

If your device doesn’t have a state, you’re never going to keep an Item, whose entire purpose is to represent the state of something, synced with the stateless device. So don’t even try.

we don’t understand each other) please let me try to explain my problem differently.
My goal is for the ESP to restore the relay states correctly relative to the sitemap at the time of connecting to the broker.
at the time of connection in the sitemap, both relays are turned on.
one item was included using the sitemap.
the second using ESP via stateTopic.

When loading again, the ESP receives different states.

OK, so that means the relays do have a state (otherwise what is being restored).

How is OH going to knwo whe the ESP connected to the broker? The only way it can know that is if the ESP sends some sort of message when it connects to the broker that OH can receive and do something about.

The stateTopic is one direction: ESP to openHAB.

The best approach is for the ESP to remember it’s previous state as @JimT suggested. If you make OH do this your relay is going to flicker. And frankly anything else is going to be a hack.

Hack 1:

When the state of the Item changes for any reason you need to publish that as a command to the command topic with the retained flag set. If you change the relay on the device it’s going to publish a state update but not a command. So OH needs to publish the command. Of course this means that the device will get commanded twice and certain types of commands like “toggle” will be impossible. Because the message is retained, the ESP will receive these commands when it boots even if OH isn’t running. If OH comes up later, the Items and the relays could very well be out of sync.

Hack 2:

The ESP publishes some message when it comes online. OH subscribes to that topic and when a message is received, publishes commands representing the current states of the Items. Of course this is limited in that if OH isn’t online when the ESP publishes it’s message, OH will not publish the Item states as commands.

You also need to deal with the situation when openhab is down

I found that in fact the broker saves both topics, but when connected, openhab reads the states from the commandTopic and not from the stateTopic.
In general, I couldn’t think of anything smarter than how to make the state and command topics the same.
my scheme - ESP sends a topic and immediately reads it and moves the relay based on the topic. when the broker is offline, the ESP manages itself on a failover basis, and when the broker comes online again, it sends its states.
Everything seems to work, when connected to the broker, all devices enter the same states, the states are stored by the broker, as I wanted)

in my opinion, it would be more logical if OH read the state from the topic to which it is subscribed and not from which it sends commands

This doesn’t sound right.

Is your item’s autoupdate set to false?