[SOLVED] Fail to start OpenHAB after upgrade to testing 2.4.0.M5 (or is it M6?)

ncwekjn · November 23, 2018, 1:29pm

Platform information:
- Hardware: Raspberry Pi 3B
- OS: OpenHABian Linux 4.14.79-v7+ #1159 SMP Sun Nov 4 17:50:20 GMT 2018 armv7l
- Java Runtime Environment:

openjdk version "1.8.0_152"
OpenJDK Runtime Environment (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 1.8.0_152-b76)
OpenJDK Client VM (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 25.152-b76, mixed mode, Evaluation)

openHAB version: 2.4.0~M6-1 (Milestone Build)

$ dpkg-query --show openhab2
openhab2	2.4.0~M6-1

After an upgrade to testing I experienced issues with binding installation, for example:

[ERROR] [core.karaf.internal.FeatureInstaller] - Failed installing 'openhab-binding-expire1': Unable to resolve root: missing requirement [root] osgi.identity; osgi.identity=openhab-runtime-base; type=karaf.feature; version="[2.4.0.M5,2.4.0.M5]"; filter:="(&(osgi.identity=openhab-runtime-base)(type=karaf.feature)(version>=2.4.0.M5)(version<=2.4.0.M5))"

Then I’ve cleaned the cache with openhab-cli clean-cache, after that it fails to start entirely:

[ERROR] [ternal.service.BootFeaturesInstaller] - Error installing boot feature repository mvn:org.openhab.distro/distro/2.4.0.M5/xml/features
java.lang.RuntimeException: Error resolving artifact org.openhab.distro:distro:xml:features:2.4.0.M5: [Could not find artifact org.openhab.distro:distro:xml:features:2.4.0.M5 in openhab (https://openhab.jfrog.io/openhab/online-repo-milestone/2.4/)] : mvn:org.openhab.distro/distro/2.4.0.M5/xml/features
        at org.apache.karaf.features.internal.service.RepositoryImpl.load(RepositoryImpl.java:116) ~[11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.RepositoryImpl.<init>(RepositoryImpl.java:50) ~[11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.RepositoryCacheImpl.create(RepositoryCacheImpl.java:51) ~[11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.FeaturesServiceImpl.addRepository(FeaturesServiceImpl.java:372) ~[11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.FeaturesServiceImpl.addRepository(FeaturesServiceImpl.java:367) ~[11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.BootFeaturesInstaller.addRepositories(BootFeaturesInstaller.java:123) [11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.BootFeaturesInstaller.installBootFeatures(BootFeaturesInstaller.java:93) [11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.service.BootFeaturesInstaller.start(BootFeaturesInstaller.java:87) [11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.features.internal.osgi.Activator.doStart(Activator.java:196) [11:org.apache.karaf.features.core:4.2.1]
        at org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:275) [11:org.apache.karaf.features.core:4.2.1]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
        at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: java.io.IOException: Error resolving artifact org.openhab.distro:distro:xml:features:2.4.0.M5: [Could not find artifact org.openhab.distro:distro:xml:features:2.4.0.M5 in openhab (https://openhab.jfrog.io/openhab/online-repo-milestone/2.4/)]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:720) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:659) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:600) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:567) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:557) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.Connection.getInputStream(Connection.java:123) ~[?:?]
        at java.net.URL.openStream(URL.java:1045) ~[?:?]
        at org.apache.karaf.features.internal.service.RepositoryImpl.load(RepositoryImpl.java:113) ~[?:?]
        ... 14 more
        Suppressed: shaded.org.eclipse.aether.transfer.ArtifactNotFoundException: Could not find artifact org.openhab.distro:distro:xml:features:2.4.0.M5 in openhab (https://openhab.jfrog.io/openhab/online-repo-milestone/2.4/)
                at shaded.org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:39) ~[?:?]
                at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355) ~[?:?]
                at shaded.org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67) ~[?:?]
                at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581) ~[?:?]
                at shaded.org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249) ~[?:?]
                at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520) ~[?:?]
                at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421) ~[?:?]
                at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) ~[?:?]
                at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) ~[?:?]
                at shaded.org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:294) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:705) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:659) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:600) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:567) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:557) ~[?:?]
                at org.ops4j.pax.url.mvn.internal.Connection.getInputStream(Connection.java:123) ~[?:?]
                at java.net.URL.openStream(URL.java:1045) ~[?:?]
                at org.apache.karaf.features.internal.service.RepositoryImpl.load(RepositoryImpl.java:113) ~[11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.RepositoryImpl.<init>(RepositoryImpl.java:50) ~[11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.RepositoryCacheImpl.create(RepositoryCacheImpl.java:51) ~[11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.FeaturesServiceImpl.addRepository(FeaturesServiceImpl.java:372) ~[11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.FeaturesServiceImpl.addRepository(FeaturesServiceImpl.java:367) ~[11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.BootFeaturesInstaller.addRepositories(BootFeaturesInstaller.java:123) [11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.BootFeaturesInstaller.installBootFeatures(BootFeaturesInstaller.java:93) [11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.service.BootFeaturesInstaller.start(BootFeaturesInstaller.java:87) [11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.features.internal.osgi.Activator.doStart(Activator.java:196) [11:org.apache.karaf.features.core:4.2.1]
                at org.apache.karaf.util.tracker.BaseActivator.run(BaseActivator.java:275) [11:org.apache.karaf.features.core:4.2.1]
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]
                at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
                at java.lang.Thread.run(Thread.java:748) [?:?]
Caused by: shaded.org.eclipse.aether.resolution.ArtifactResolutionException: Error resolving artifact org.openhab.distro:distro:xml:features:2.4.0.M5
        at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444) ~[?:?]
        at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246) ~[?:?]
        at shaded.org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223) ~[?:?]
        at shaded.org.eclipse.aether.internal.impl.DefaultRepositorySystem.resolveArtifact(DefaultRepositorySystem.java:294) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:705) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:659) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:600) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:567) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.AetherBasedResolver.resolve(AetherBasedResolver.java:557) ~[?:?]
        at org.ops4j.pax.url.mvn.internal.Connection.getInputStream(Connection.java:123) ~[?:?]
        at java.net.URL.openStream(URL.java:1045) ~[?:?]
        at org.apache.karaf.features.internal.service.RepositoryImpl.load(RepositoryImpl.java:113) ~[?:?]
        ... 14 more

How can I fix that? I also don’t understand why it tries to install 2.4.0.M5 whereas it’s supposed to be 2.4.0.M6.

Dim · November 23, 2018, 1:34pm

do you have anything in the addons folder? (maybe the M5 version of the offline kar)?

try to clean again cache&tmp and check the logs

did the upgrade script ask you to overwrite any config files?

ncwekjn · November 23, 2018, 1:36pm

Everything seems to be M5, that’s the thing, whereas dpkg reports that M6 is installed.

Did exactly that and it stopped working, see above

Nope

Dim · November 23, 2018, 1:46pm

anything in the addons folder?

check please:

/var/lib/openhab2/config/org/apache/karaf/features/repos.config
/var/lib/openhab2/config/org/apache/karaf/features.config

for any hardcoded M5 references (shouldn’t be any)

Dim · November 23, 2018, 1:53pm

from which build did you upgrade to M6 from? M5 or something else?

ncwekjn · November 23, 2018, 2:00pm

anything in the addons folder?

Do you mean this?

$ ls -l /var/lib/openhab2/tmp/mvn/org/openhab/distro/openhab-addons/
total 4
drwxrwxrwx 2 openhab openhabian 4096 Nov 22 20:30 2.4.0.M5

Dim:

check please:
/var/lib/openhab2/config/org/apache/karaf/features/repos.config
/var/lib/openhab2/config/org/apache/karaf/features.config
for any hardcoded M5 references (shouldn’t be any)

$ grep M5 /var/lib/openhab2/config/org/apache/karaf/features/repos.config
$ grep M5 /var/lib/openhab2/config/org/apache/karaf/features.config
featuresRepositories="mvn:org.openhab.distro/distro/2.4.0.M5/xml/features,\ mvn:org.openhab.distro/openhab-addons/2.4.0.M5/xml/features,\ mvn:org.apache.karaf.features/framework/4.2.1/xml/features,\ mvn:org.apache.karaf.features/standard/4.2.1/xml/features"

So there are indeed M5 references, how do I go about that?

from which build did you upgrade to M6 from? M5 or something else?

Not 100% sure, I think it was M5.

Dim · November 23, 2018, 2:03pm

no, the /usr/share/openhab2/addons/

most likely, you have the M5 offline addon (*.kar) there

it extacts automagically into userdata/tmp

remove it from the addons folder and clear cache (again )
you should be fine after that (no bets)

ncwekjn · November 23, 2018, 2:04pm

Nope, it’s empty:

$ ls -l /usr/share/openhab2/addons/
total 4
-rw-rw-r-- 1 openhab openhabian 70 Nov 20 13:13 README

Dim · November 23, 2018, 2:05pm

ok, trying to reproduce this now… i just installed M5 and going to upgrade it to M6 to see what the heck is going on

Dim · November 23, 2018, 2:06pm

one quick check: does openhab-cli clean-cache really removes tmp & cache subfolders in userdata ?

ncwekjn · November 23, 2018, 2:06pm

Thanks. By the way:

$ find / -name '*.kar' -type f 2>/dev/null

Nothing found.

ncwekjn · November 23, 2018, 2:06pm

Yes

Dim · November 23, 2018, 2:16pm

I cannot reproduce this with an M5->M6 upgrade

grep M /var/lib/openhab2/config/org/apache/karaf/features.config
featuresRepositories="mvn:org.openhab.distro/distro/2.4.0.M6/xml/features,\ mvn:org.openhab.distro/openhab-addons/2.4.0.M6/xml/features,\ mvn:org.apache.karaf.features/framework/4.2.1/xml/features,\ mvn:org.apache.karaf.features/standard/4.2.1/xml/features"

it has been correctly updated, all bindings in PaperUI show M6 and I don’t have in tmp any remains from M5…

ls -al /var/lib/openhab2/tmp/mvn/org/openhab/
total 20
drwxr-xr-x 5 openhab openhab 4096 16:11 23-Nov-2018 .
drwxr-xr-x 4 openhab openhab 4096 16:11 23-Nov-2018 ..
drwxr-xr-x 3 openhab openhab 4096 16:11 23-Nov-2018 binding
drwxr-xr-x 5 openhab openhab 4096 16:11 23-Nov-2018 core
drwxr-xr-x 4 openhab openhab 4096 16:11 23-Nov-2018 ui

can you try to redeploy M6 ?

ncwekjn · November 23, 2018, 2:19pm

What’s the best way to do that?

Dim · November 23, 2018, 2:22pm

apt-get install --reinstall openhab2

ncwekjn · November 23, 2018, 2:30pm

Hmmm, did that, but still

$ grep M /var/lib/openhab2/config/org/apache/karaf/features.config
featuresRepositories="mvn:org.openhab.distro/distro/2.4.0.M5/xml/features,\ mvn:org.openhab.distro/openhab-addons/2.4.0.M5/xml/features,\ mvn:org.apache.karaf.features/framework/4.2.1/xml/features,\ mvn:org.apache.karaf.features/standard/4.2.1/xml/features"

Is there any way to get rid of that?

Dim · November 23, 2018, 2:31pm

try to edit the file and change M5 to M6…
on the other hand… this is not normal and shouldn’t happen

ncwekjn · November 23, 2018, 2:51pm

Ah. My bad, after restarting OpenHAB the versions changed to M6. And if I clean the cache now, everything comes back online nicely. Thanks a lot!

For the recap:

sudo openhab-cli backup --full   # just in case
sudo systemctl stop openhab2.service
sudo apt-get install --reinstall openhab2
sudo openhab-cli clean-cache
sudo systemctl start openhab2.service

yves · November 24, 2018, 11:27am

I’m trying to understand what originally went wrong .
Do I understand correctly that openhab was running when you originally cleared the cache ?

(the more the community understands what you did, the better we know what to warn people for)

y

ncwekjn · November 24, 2018, 8:38pm

I’m trying to recollect now, I cannot say for sure if it was running.

However if it isn’t supposed to be running while clearing the cache then maybe openhab-cli must enforce that?

What I did was:

Install OpenHABian from the image
Switch to testing using openhabian-config
Upgrade to the latest release
Recover /etc/openhab2 and /var/lib/openhab2 from my own backup (not using openhabian-config).

And somewhere along the way it went wrong I guess.