Persistence not available/working after upgrade to OH 4 (not OH4-issue!)

kdmsch · August 17, 2023, 7:25am

I have a strange behaviour of OH after upgrading from well working OH 3.4.4 to OH 4.0.1 and also Snapshot 4.1.0 #3590 (after failled upgrade via config-tool I made a fresh installation from scratch on a RP4 and restored my backup from OH 3.4.4). Thereafter no persitence of data is available -no historical and also no actual data will be recorded. All graphs of items (e.g. temperature, humidity, etc.) are empty, although the database is available on var/lib/openhab/persistence/rrd4j and rrd4j persitence tool is installed! When I install also influxdb - as I have used it in the beginning of using OH3 - there is no change besides warning of increasing number of

…… “Re-queuing xxxx elements, failed to write batch.”

Could it be an effect of changing the unit-handling by OH4 and maybe a wrong implementation in my old OH3 system? Only indication is the warning in the logdata of:

"2023-08-17 08:46:00.805 [WARN ] [penhab.core.library.items.NumberItem] - Failed to update item ‘KNXWetterstationAP257_HelligkeitAussen’ because ‘34611.2 °C’ could not be converted to the item unit ‘cd’

But this is the only warning and I could not correct it up to now.

Another strange behavior and maybe also linked with persistence issue is changing of the titel of cards (location, equipment and property) from German to English on different devices (PC, smartphone, tablet). Sometimes all is German and sometimes partly in English. It changes from time to time as if by an invisible hand w/o any changes made in OH! Obviously this is only on cards where no individual titels used in the card-menue!

It would be great to get a hint for a solution as I’m not an expert and I have currently no idea why I run into these issues and how to solve it!

rlkoshak · August 17, 2023, 2:15pm

Well, that definitely points to something. It’s failing to write to the databases.

No because persistence largely ignores the unit. It just stores the raw number.

This is unrelated to persistence. And the fix is you need to a unit metadata to this Item with a unit compatible with Temperature (°C, °F or °K). cd is the unit indicator for “candela” which is a measure of light intensity.

If this impacts persistence, it would be only for this one Item.

I think there is an issue or perhaps even a PR open for something related to this. But again, it’s not related to persistence.

Do you have .persist files or did you set the persistence strategy on the bindings?

Do you see the rrd4j files under userdata being written to at all (the modified dates should be changing)?

Show the full logs around those error messages.

Put the add-on into debug logging.

kdmsch · August 18, 2023, 7:46am

Thanks first for your reply and valuable hints.
I use the default persitence strategy of OH. There is a file for each item in the folder according to following example:

But date of files is 1.08.2023 which is date of OH4 installation and restoring the backup. After that nothing has happened with these data and the result as described.
Following the log-data of the startup after upgrade to latest snapshot #3592:

[openHAB] Running JSON Database upgrade tool (${OPENHAB_RUNTIME}/bin/upgradetool.jar).
[main] INFO org.openhab.core.tools.internal.Upgrader - Copying item unit from state description to metadata in database '/var/lib/openhab/jsondb/org.openhab.core.items.Item.json'
[main] WARN org.openhab.core.tools.internal.Upgrader - Generic_MQTT_Thing_KGAz_Humidity: Could not determine if channel 'mqtt:topic:37e730e2a4:9bd5cc7fe5:MQTT_01_Humidity' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Thermostatli_Temperatur: Could not determine if channel 'boschshc:thermostat:36ddf8ec80:8bbc46d8e6:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Generic_MQTT_Thing_Pressure: Could not determine if channel 'mqtt:topic:37e730e2a4:9bd5cc7fe5:THP_z2m_01_Pressure' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Thermostatli_Position_Ventilstossel: Could not determine if channel 'boschshc:thermostat:36ddf8ec80:8bbc46d8e6:valve-tappet-position' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Thermostatre_Temperatur: Could not determine if channel 'boschshc:thermostat:36ddf8ec80:181181015d:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - KNXRaumtemperaturreglerUP231302_SollTempKNXEGWohnen: Could not determine if channel 'knx:device:bridge:4e849180dd:SollTemp_KNX_EG_Wohnen' sets a state description. Check if you need to set unit metadata.
[main] INFO org.openhab.core.tools.internal.Upgrader - KNXRaumtemperaturreglerUP231302_SollTempKNXEGWohnen: Wrote 'unit=°C' to metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - HumiTemp04_Temperature: Could not determine if channel 'enocean:temperatureHumiditySensor:bdf9f894:05194A4A:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - THP_z2m_01_Pressure: Could not determine if channel 'mqtt:topic:37e730e2a4:b2896d101b:THP_z2m_01_pressure' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - ThermostatAZ01_FeedTemperature: Could not determine if channel 'enocean:thermostat:bdf9f894:019AA8F9:feedTemperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - THP_z2m_01_Temperature: Could not determine if channel 'mqtt:topic:37e730e2a4:b2896d101b:THP_z2m_01_Temp' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - AstronomischeSonnendaten_TimeLeft: Could not determine if channel 'astro:sun:6705b6d030:season#timeLeft' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - HumiTemp01_Temperature: Could not determine if channel 'enocean:temperatureHumiditySensor:bdf9f894:05820B1D:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - HumiTemp03_Temperature: Could not determine if channel 'enocean:temperatureHumiditySensor:bdf9f894:0582F9CC:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - HumiTemp02_Temperature: Could not determine if channel 'enocean:temperatureHumiditySensor:bdf9f894:05820A68:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - ThermostatAZ01_Temperature: Could not determine if channel 'enocean:thermostat:bdf9f894:019AA8F9:temperature' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - KNX_Wetterstation_AP257_AussenHelligkeit: Could not determine if channel 'knx:device:bridge:436f7cff64:Aussen_Helligkeit' sets a state description. Check if you need to set unit metadata.
[main] INFO org.openhab.core.tools.internal.Upgrader - Generic_MQTT_Thing_Temperature: Could not find thing for channel 'mqtt:topic:37e730e2a4:1a0da29bdd:MQTT_001_Temp'. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - KNXRaumtemperaturreglerUP231302_EGWohnenTemperatur: Could not determine if channel 'knx:device:bridge:4e849180dd:gGF_EG_WE_Temperatur' sets a state description. Check if you need to set unit metadata.
[main] INFO org.openhab.core.tools.internal.Upgrader - KNXRaumtemperaturreglerUP231302_EGWohnenTemperatur: Wrote 'unit=°C' to metadata.[main] WARN org.openhab.core.tools.internal.Upgrader - Plug04_TotalUsage: Could not determine if channel 'enocean:measurementSwitch:bdf9f894:0582F69D:totalusage' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - KNXWetterstationAP257_Aussentemperatur: Could not determine if channel 'knx:device:bridge:436f7cff64:Aussentemperatur' sets a state description. Check if you need to set unit metadata.
[main] INFO org.openhab.core.tools.internal.Upgrader - KNXWetterstationAP257_Aussentemperatur: Wrote 'unit=°C' to metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Thermostatre_Position_Ventilstossel: Could not determine if channel 'boschshc:thermostat:36ddf8ec80:181181015d:valve-tappet-position' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - Generic_MQTT_Thing_Temperature_Az: Could not determine if channel 'mqtt:topic:37e730e2a4:9bd5cc7fe5:MQTT_001' sets a state description. Check if you need to set unit metadata.
[main] WARN org.openhab.core.tools.internal.Upgrader - ThermostatAZ01_SetPoint: Could not determine if channel 'enocean:thermostat:bdf9f894:019AA8F9:temperatureSetPoint' sets a state description. Check if you need to set unit metadata.
[main] INFO org.openhab.core.tools.internal.Upgrader - Upgrading JS profile configuration in database '/var/lib/openhab/jsondb/org.openhab.core.thing.link.ItemChannelLink.json'
[openHAB] JSON Database upgrade completed.

Maybe his put some ligth on the issue!?

rlkoshak · August 18, 2023, 2:30pm

I don’t know what file browser tool you are using but each file has multiple timestamps: date created, date modified, sometimes more. I suspect that GUI is only showing the data created. Assuming this is a Linux machine, the command ls -l will show the last modified timestamp.

This is not related to the upgrade tool.

Either your persistence config is broken, the add-on is broken, or something else is going wrong systematically. Nothing the upgradetool does nor nothing related to UoM has any impact here.

kdmsch · August 22, 2023, 10:39am

… I think there is something broken in my backup zip-file from OH3.4.5.
I have now installed fresh OH 3.4.5 and after restore of the data I have the same issue - no persistence date available, no historical and no actual ones! Have to go deeper into configuration of rrd4j (also doc is not easy to follow for a nonprofessional-but i will learn ). Nevertheless it’s not what I have expected by backup/restore of OH configuration!
Thanks for your help so far!

Regards
Klaus

rlkoshak · August 22, 2023, 2:51pm

Well, that narrows down †he problem at least.

If the problem is caused by the configuration …

Keep in mind that openHAB really is a development platform. The “configuration” is really your code. It’s high level in most cases but it is software code. It’s way more complicated and involved than the term “configuration” gets across.

Above I said

Either your persistence config is broken, the add-on is broken, or something else is going wrong systematically. Nothing the upgradetool does nor nothing related to UoM has any impact here.

So you’ve shown that the add-on is not what’s broken. So now to check the configs. The add-on configs where available and the persistence configs. Putting the relevant add-ons into trace or debug level logging will help too.