4.0.1: Cannot delete or modify things that are provisioned from file

jpalo · August 10, 2023, 9:13am

After upgrading to 4.0.1 I’m having problems with deleting things (that are defined in a .things file) as they remain in the system even if I remove them from the things file. There is no such issue with items, modifying them in .items file reflects to items on the management web site in few seconds. Before on openhab v3 there was no such problem.

Issue is with all bindings/things. Restarting openhab service has no effect.

Even in the CLI, it keeps listing the things even after removing them from the .things file and they cannot be removed (which is obvious as they were provisioned via file).

openhab> openhab:things remove august:account:accountName
Could not delete thing august:account:accountName.

What could I try to get things working again and changes to .things files to reflect to system?

mstormi · August 10, 2023, 12:35pm

Try deleting the cache, too, but I’d rather believe you have another definition for these things in effect.
Another file (do the things show the lock in UI?) or you configured them in the UI, too.

jpalo · August 10, 2023, 12:56pm

Clearing cache did nothing (followed this, except openhab2 → openhab: Clear the Cache - #4 by binderth). I only configure via files, so things show the lock icon. I only have one default.things file.

EDIT: It seems to be loading the default.things file when I modify it, though.

2023-08-10 16:01:09.239 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'default.things'

2023-08-10 16:01:09.265 [INFO ] [el.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model 'default.things', using it anyway:

Provide a thing type ID and a thing ID in this format:

 <thingTypeId> <thingId>

Provide a thing type ID and a thing ID in this format:

 <thingTypeId> <thingId>

==> /var/log/openhab/events.log <==

CONCLUSION: I can add things via file, but things are not removed from the system when they are removed from the things file. Constant repro. Seems like an issue with v4.

Added few things via file (Fercs2 and Fercs3), and removed them from the file - things remain in the system:

Wolfgang_S · August 10, 2023, 2:21pm

That is because validation issues.
Correct it and the thing should be removed.

jpalo · August 10, 2023, 3:39pm

Thanks for the suggestion, but that’s not it. This warning has been forever there also on v3, and new things get added just fine.

jpalo · August 11, 2023, 6:56am

There are no other .things files:

openhabian@openhab:~ $ sudo find / -name *.things
/etc/openhab/things/default.things
openhabian@openhab:~ $

So, to reiterate the repro steps:

comment out a thing in things file
stop/clear cache/start → thing is removed from the system (see web ui)
add thing via things file → thing is added and can be seen in web ui with a lock indicating it is defined in a things file
comment out thing in a things file → thing remains in system (see web ui, thing remains there with a lock indicating it is still defined in a things file)

Only way to remove thing from the system is to do step 1.

Commands for step 1:

openhabian@openhab:~ $ sudo systemctl stop openhab.service
openhabian@openhab:~ $ sudo openhab-cli clean-cache

This command will delete the temporary files within openHAB.
May resolve issues with addon installation and configuration.
The next start of openHAB will take a bit longer.

Okay to Continue? [y/N]: y
openhabian@openhab:~ $ sudo systemctl start openhab.service

Using openHAB 4.0.1 - Release Build. I can add things via web ui just fine, and remove the ones that have been added via web ui just fine.

Lolodomo · August 11, 2023, 7:31am

As you have the “loading file.things” log, that is a good point.
I will check myself but it is more than probable that your problem is simply your errors in the file. Why not starting by fixing them??
Move one or two things to another file and check that there is no validation errors for this file. Now try again to comment in this new file.

jpalo · August 11, 2023, 7:42am

After testing, I can confirm that my finding and repro steps are valid. There is nothing wrong with the things file as even after removing all things from the system and testing with single simple thing definition in a things file and no Provide a thing type ID and a thing ID in this format INFO messages when parsing the file, I can repro the issue.

I can also confirm that after a thing has been added to the system via things file, it can also NOT be modified even if the thing definition is modified in the things file.

2023-08-11 10:38:21.178 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'default.things'

==> /var/log/openhab/events.log <==

2023-08-11 10:38:21.386 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'astro:sun:home' changed from UNINITIALIZED to INITIALIZING

2023-08-11 10:38:21.392 [INFO ] [ab.event.ThingStatusInfoChangedEvent] - Thing 'astro:sun:home' changed from INITIALIZING to ONLINE

==> /var/log/openhab/openhab.log <==

Lolodomo · August 11, 2023, 10:01am

Works perfectly for me, so at least it is not a general problem but a specific problem to your OH setup.
Test on a RPI.

jpalo · August 11, 2023, 10:14am

So the thing is removed from UI after you remove it from the things file?

I am using RPI and issue started after updating to OH4.

What could I try to pinpoint the issue? Is there a more detailed log somewhere from which I could see what happens when things file is parsed and things fail to be removed from the system?

Lolodomo · August 11, 2023, 10:19am

Yes

jpalo · August 11, 2023, 10:28am

Could you please test with these two, the “Fercs” doesn’t get removed for me and is stuck in the system, but the “Astro” does work fine as it gets removed from the system when I remove it from the things file (so a small progress in investigation). Or just test the “Fercs” one as it is an example of a thing causing problems for me.

Thing http:url:fercs "Fercs" [
	baseURL="http://192.168.1.37:8080/static/fercs.json",
	refresh=1] {
		Channels:
			Type number: alarmState "Fercs alarm state" [ stateTransformation="JSONPATH:$.FERCS10_Data_API.alarm.state", mode="READONLY" ]
            Type number: otherDoorsState "Other doors state" [ stateTransformation="JSONPATH:$.FERCS10_Data_API.zones.zone[3].active", mode="READONLY" ]
}

Thing astro:sun:home [ geolocation="63.361874,25.339227000000004", interval=60 ]
{
    Channels:
        Type start : rise#start [
            offset=30
        ]
        Type rangeEvent : rise#event [
            offset=30
        ]

        Type start : set#start [
            offset=-20
        ]
        Type rangeEvent : set#event [
            offset=-20
        ]
}

Lolodomo · August 11, 2023, 10:31am

I tried myself with netatmo things.

So now you identified that it is not working only for particular things.

I have not the http binding installed so I can’t test your precise case.

Is your binding the official binding packaged with OH or is it one coming from the marketplace ?

jpalo · August 11, 2023, 10:35am

Http is probably one of the simplest bindings that comes with OH, but the issue is also on other bindings. In fact, the Astro is the only one I’ve found to be working so far:

Can you @Lolodomo please paste the netatmo thing declaration you used for testing, and I’ll see how it works for me. And thank you for your time in helping me.

Lolodomo · August 11, 2023, 11:09am

You can try to enable the following logs: log:set TRACE org.openhab.core.model.core

jpalo · August 11, 2023, 11:24am

Thanks, unfortunately that log level provides no additional logs when parsing the things file.

Can you please paste your netatmo definition and I can test it on my installation. With this dummy test, the thing gets stuck in my system:

Bridge netatmo:account:myaccount "Netatmo Account" [clientId="xxxxx", clientSecret="yyyy"] {    
    Bridge home myhome "My home" [ id="0123456789abcdef", refreshInterval=150 ] {
        Thing welcome mycam "My camera" [ id="70:aa:bb:cc:dd:ee" ] {
            Channels:
                Type live-stream-url : live#local-stream-url [ quality="high" ]
                Type live-stream-url : live#vpn-stream-url [ quality="low" ]
        }
    }
}

Can someone install the HTTP Binding (it takes one click to install in web ui), and test the simplest thing definition I’ve been able to repro the issue with:

Thing http:url:test "Test" [ baseURL="#" ]

jpalo · August 14, 2023, 9:16am

Any help?

j.hoekstra · August 14, 2023, 11:05am

Alright, I did:

add http.things with above contents for the Ferc.thing
it loaded immediately, stating errors since the endpoint is not available on my network.
thing not editable using web-interface
deleted http.things from the things folder.
thing is gone from things in web-interface.

Setup:
OH_4.0.1 in docker.

This seems expected behaviour?

It gets interesting when:

editing the file and removing multiple things except 1:
- removed things get removed immediately from web-interface
editing the file and introducing an error(ie trailing . after thing):
- all things in the file get removed in web-interface
editing the file and remove all things:
- the following error in std log: ‘Configuration model ‘http.things’ is either empty or cannot be parsed correctly!’
- things still avaiable in web-interface(locked)
- after restart openhab: things gone from web-interface

What would be interesting to know:

What happens when this default.things is only filled with the above definition, or if the above definition is the first in the file.

The given error: ‘Validation issues found in configuration model ‘default.things’, using it anyway:’ always has been a note of best effort in my mind. So as far as the config can be parsed correctly it will be used up until the point it can’t be parsed, after that it’s ignored.

jpalo · August 14, 2023, 12:56pm

Thanks for testing, appreciate it! For some reason, it doesn’t seem to update or remove a thing if it is modified or removed from the things file. Only if you delete the whole things file, it will remove the thing from the system. This has changed since OH3, so hopefully someone could revise the implementation and see if this is a desired new behavior regarding thing files.

I’ve also already updated my things file not to anymore get that validation INFO message, so now it parses without any errors, warnings, info messages. Yet the above described behavior remains.

Lolodomo · August 14, 2023, 3:44pm

This is not true. Please read again my test feedback and those from @j.hoekstra .

The only case is when you remove all things from one file but keeping the file. Rather than displaying a warning, we could decide to remove all things in this case too. I have no doubt I can reproduce and fix this particular case. Maybe @J-N-K had a good reason to not do that?

@jpalo : unfortunately, noone is able to reproduce your generic case.