Optional dependency not available

lujop · October 2, 2021, 2:35pm

For a DBQuery addon, I have optional dependencies with jdbc drivers in the same way the jdbc persistence add-on does.

This works correctly in my local dev instance that is downloaded from nightly builds but not in my ‘production’ instance that is running 3.2M2.

The problem is that my addon is loaded and activated correctly, but the optional dependency for PostgreSQL isn’t enabled for the addon, despite it’s present and with the correct version:

Result of bundle:headers on the addon:

But the PostgreSQL is correctly installed:

and extract from bundle:headers for the postgresql driver:

openhab> bundle:headers 279

PostgreSQL JDBC Driver (279)
----------------------------
Automatic-Module-Name = org.postgresql.jdbc
Implementation-Title = PostgreSQL JDBC Driver
Implementation-Vendor = PostgreSQL Global Development Group
Implementation-Vendor-Id = org.postgresql
Implementation-Version = 42.2.18
Main-Class = org.postgresql.util.PGJDBCMain
Manifest-Version = 1.0
Specification-Title = JDBC
Specification-Vendor = Oracle Corporation
Specification-Version = 4.2
Tool = Bnd-4.3.1.201911131708

Bundle-Activator = org.postgresql.osgi.PGBundleActivator
Bundle-Copyright = Copyright (c) 2003-2020, PostgreSQL Global Development Group
Bundle-Descriptiona = Java JDBC driver for PostgreSQL database
Bundle-DocURL = https://jdbc.postgresql.org/
Bundle-License = BSD-2-Clause
Bundle-ManifestVersion = 2
Bundle-Name = PostgreSQL JDBC Driver
Bundle-SymbolicName = org.postgresql.jdbc
Bundle-Vendor = PostgreSQL Global Development Group
Bundle-Version = 42.2.18

Provide-Capability =
        osgi.service;effective:=active;objectClass=org.osgi.service.jdbc.DataSourceFactory
Require-Capability =
        osgi.ee;filter:=(&(|(osgi.ee=J2SE)(osgi.ee=JavaSE))(version>=1.8))

Export-Package =
        org.postgresql;

The curious is that the same works in my dev machine.

Any help or advice to diagnose and find a solution will be appreciated.

J-N-K · October 2, 2021, 4:56pm

What is the result of diag on the dbquery bundle?

lujop · October 2, 2021, 5:22pm

The result from bundle:diag is blank, because the addon starts correctly because the driver is an optional dependency. The problem is that it’s not available in the classpath to establish a connection to de database.

What I’ve tried is to remove the optional and then that’s the output from diag:

openHAB Add-ons :: Bundles :: DBQuery Binding (278)
---------------------------------------------------
Status: Installed
Unsatisfied Requirements:
osgi.wiring.package; filter:="(&(osgi.wiring.package=com.mysql.jdbc)(&(version>=8.0.0)(!(version>=9.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.h2)(&(version>=1.4.0)(!(version>=2.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.h2.jdbcx)(&(version>=1.4.0)(!(version>=2.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.hsqldb)"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.hsqldb.jdbc)"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.mariadb.jdbc)"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.postgresql)(&(version>=42.2.0)(!(version>=43.0.0))))"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.sqlite)(&(version>=3.16.0)(!(version>=4.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.sqlite.jdbc4)(&(version>=3.16.0)(!(version>=4.0.0))))"; resolution:="optional"
Declarative Services

But I still don’t know why it isn’t satisfied by the PostgreSQL driver that it’s already available with that:

openhab> bundle:headers 279

PostgreSQL JDBC Driver (279)
----------------------------
Automatic-Module-Name = org.postgresql.jdbc
Implementation-Title = PostgreSQL JDBC Driver
Implementation-Vendor = PostgreSQL Global Development Group
Implementation-Vendor-Id = org.postgresql
Implementation-Version = 42.2.18
Main-Class = org.postgresql.util.PGJDBCMain
Manifest-Version = 1.0
Specification-Title = JDBC
Specification-Vendor = Oracle Corporation
Specification-Version = 4.2
Tool = Bnd-4.3.1.201911131708

Bundle-Activator = org.postgresql.osgi.PGBundleActivator
Bundle-Copyright = Copyright (c) 2003-2020, PostgreSQL Global Development Group
Bundle-Descriptiona = Java JDBC driver for PostgreSQL database
Bundle-DocURL = https://jdbc.postgresql.org/
Bundle-License = BSD-2-Clause
Bundle-ManifestVersion = 2
Bundle-Name = PostgreSQL JDBC Driver
Bundle-SymbolicName = org.postgresql.jdbc
Bundle-Vendor = PostgreSQL Global Development Group
Bundle-Version = 42.2.18

Provide-Capability =
        osgi.service;effective:=active;objectClass=org.osgi.service.jdbc.DataSourceFactory
Require-Capability =
        osgi.ee;filter:=(&(|(osgi.ee=J2SE)(osgi.ee=JavaSE))(version>=1.8))

Export-Package =
        org.postgresql;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.copy,
                        org.postgresql.fastpath,
                        org.postgresql.jdbc,
                        org.postgresql.largeobject,
                        org.postgresql.replication,
                        org.postgresql.util";
                version=42.2.18,
        org.postgresql.copy;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql,
                        org.postgresql.core,
                        org.postgresql.util";
                version=42.2.18,
        org.postgresql.core;
                uses:="javax.net,
                        javax.net.ssl,
                        org.checkerframework.checker.index.qual,
                        org.checkerframework.checker.nullness.qual,
                        org.checkerframework.dataflow.qual,
                        org.ietf.jgss,
                        org.postgresql,
                        org.postgresql.copy,
                        org.postgresql.core.v3,
                        org.postgresql.jdbc,
                        org.postgresql.replication,
                        org.postgresql.replication.fluent.logical,
                        org.postgresql.replication.fluent.physical,
                        org.postgresql.util,
                        org.postgresql.xml";
                version=42.2.18,
        org.postgresql.core.v3;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.copy,
                        org.postgresql.core,
                        org.postgresql.jdbc,
                        org.postgresql.util";
                version=42.2.18,
        org.postgresql.core.v3.replication;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.copy,
                        org.postgresql.core,
                        org.postgresql.replication,
                        org.postgresql.replication.fluent.logical,
                        org.postgresql.replication.fluent.physical";
                version=42.2.18,
        org.postgresql.ds;
                uses:="javax.naming,
                        javax.sql,
                        org.checkerframework.checker.nullness.qual,
                        org.postgresql.ds.common";
                version=42.2.18,
        org.postgresql.ds.common;
                uses:="javax.naming,
                        javax.naming.spi,
                        javax.sql,
                        org.checkerframework.checker.nullness.qual,
                        org.postgresql,
                        org.postgresql.jdbc";
                version=42.2.18,
        org.postgresql.fastpath;uses:="org.checkerframework.checker.nullness.qual,org.postgresql.core";version=42.2.18,
        org.postgresql.geometric;uses:="org.checkerframework.checker.nullness.qual,org.postgresql.util";version=42.2.18,
        org.postgresql.gss;
                uses:="javax.security.auth.callback,
                        org.checkerframework.checker.nullness.qual,
                        org.ietf.jgss,
                        org.postgresql.core,
                        org.postgresql.util";
                version=42.2.18,
        org.postgresql.hostchooser;uses:="org.checkerframework.checker.nullness.qual,org.postgresql.util";version=42.2.18,
        org.postgresql.jdbc;
                uses:="javax.xml.transform,
                        org.checkerframework.checker.index.qual,
                        org.checkerframework.checker.lock.qual,
                        org.checkerframework.checker.nullness.qual,
                        org.checkerframework.dataflow.qual,
                        org.postgresql,
                        org.postgresql.copy,
                        org.postgresql.core,
                        org.postgresql.fastpath,
                        org.postgresql.jdbc2,
                        org.postgresql.largeobject,
                        org.postgresql.replication,
                        org.postgresql.util,
                        org.postgresql.xml";
                version=42.2.18,
        org.postgresql.jdbc2;uses:=org.checkerframework.checker.nullness.qual;version=42.2.18,
        org.postgresql.jdbc2.optional;uses:=org.postgresql.ds;version=42.2.18,
        org.postgresql.jdbc3;uses:=org.postgresql.ds;version=42.2.18,
        org.postgresql.jre7.sasl;uses:="org.postgresql.core,org.postgresql.util";version=42.2.18,
        org.postgresql.largeobject;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.fastpath";
                version=42.2.18,
        org.postgresql.osgi;uses:="javax.sql,org.osgi.framework,org.osgi.service.jdbc";version=42.2.18,
        org.postgresql.replication;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.replication.fluent";
                version=42.2.18,
        org.postgresql.replication.fluent;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.replication,
                        org.postgresql.replication.fluent.logical,
                        org.postgresql.replication.fluent.physical";
                version=42.2.18,
        org.postgresql.replication.fluent.logical;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.replication,
                        org.postgresql.replication.fluent";
                version=42.2.18,
        org.postgresql.replication.fluent.physical;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.replication,
                        org.postgresql.replication.fluent";
                version=42.2.18,
        org.postgresql.ssl;
                uses:="javax.net.ssl,
                        javax.security.auth.callback,
                        org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.util";
                version=42.2.18,
        org.postgresql.ssl.jdbc4;uses:="javax.net.ssl,org.postgresql.ssl,org.postgresql.util";version=42.2.18,
        org.postgresql.sspi;uses:="com.sun.jna,org.checkerframework.checker.nullness.qual,org.postgresql.core";version=42.2.18,
        org.postgresql.translation;version=42.2.18,
        org.postgresql.util;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.checkerframework.dataflow.qual,
                        org.postgresql.core";
                version=42.2.18,
        org.postgresql.util.internal;
                uses:="org.checkerframework.checker.nullness.qual,
                        org.checkerframework.dataflow.qual";
                version=42.2.18,
        org.postgresql.xa;
                uses:="javax.naming,
                        javax.sql,
                        javax.transaction.xa,
                        org.checkerframework.checker.nullness.qual,
                        org.postgresql.core,
                        org.postgresql.ds,
                        org.postgresql.ds.common";
                version=42.2.18,
        org.postgresql.xml;
                uses:="javax.xml.parsers,
                        javax.xml.stream,
                        javax.xml.transform,
                        javax.xml.transform.sax,
                        org.checkerframework.checker.nullness.qual,
                        org.xml.sax";
                version=42.2.18
Import-Package =
        javax.sql,
        javax.transaction.xa,
        javax.naming,
        javax.security.sasl;resolution:=optional,
        com.sun.jna;resolution:=optional,
        com.sun.jna.platform.win32;resolution:=optional,
        com.sun.jna.ptr;resolution:=optional,
        com.sun.jna.win32;resolution:=optional,
        javax.crypto;resolution:=optional,
        javax.crypto.spec;resolution:=optional,
        javax.naming.ldap;resolution:=optional,
        javax.naming.spi;resolution:=optional,
        javax.net;resolution:=optional,
        javax.net.ssl;resolution:=optional,
        javax.security.auth;resolution:=optional,
        javax.security.auth.callback;resolution:=optional,
        javax.security.auth.login;resolution:=optional,
        javax.security.auth.x500;resolution:=optional,
        javax.xml.parsers;resolution:=optional,
        javax.xml.stream;resolution:=optional,
        javax.xml.transform;resolution:=optional,
        javax.xml.transform.dom;resolution:=optional,
        javax.xml.transform.sax;resolution:=optional,
        javax.xml.transform.stax;resolution:=optional,
        javax.xml.transform.stream;resolution:=optional,
        org.checkerframework.checker.index.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.checker.initialization.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.checker.lock.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.checker.nullness.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.checker.regex.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.common.value.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.dataflow.qual;resolution:=optional;version="[3.5,4)",
        org.checkerframework.framework.qual;resolution:=optional;version="[3.5,4)",
        org.ietf.jgss;resolution:=optional,
        org.osgi.framework;resolution:=optional;version="[1.6,2)",
        org.osgi.service.jdbc;resolution:=optional;version="[1.0,2)",
        org.postgresql;resolution:=optional,
        org.postgresql.copy;resolution:=optional,
        org.postgresql.core;resolution:=optional,
        org.postgresql.core.v3;resolution:=optional,
        org.postgresql.core.v3.replication;resolution:=optional,
        org.postgresql.ds;resolution:=optional,
        org.postgresql.ds.common;resolution:=optional,
        org.postgresql.fastpath;resolution:=optional,
        org.postgresql.geometric;resolution:=optional,
        org.postgresql.gss;resolution:=optional,
        org.postgresql.hostchooser;resolution:=optional,
        org.postgresql.jdbc;resolution:=optional,
        org.postgresql.jdbc2;resolution:=optional,
        org.postgresql.jdbc2.optional;resolution:=optional,
        org.postgresql.largeobject;resolution:=optional,
        org.postgresql.replication;resolution:=optional,
        org.postgresql.replication.fluent;resolution:=optional,
        org.postgresql.replication.fluent.logical;resolution:=optional,
        org.postgresql.replication.fluent.physical;resolution:=optional,
        org.postgresql.ssl;resolution:=optional,
        org.postgresql.sspi;resolution:=optional,
        org.postgresql.util;resolution:=optional,
        org.postgresql.util.internal;resolution:=optional,
        org.postgresql.xa;resolution:=optional,
        org.postgresql.xml;resolution:=optional,
        org.w3c.dom;resolution:=optional,
        org.xml.sax;resolution:=optional,
        org.xml.sax.helpers;resolution:=optional,
        waffle.windows.auth;resolution:=optional,
        waffle.windows.auth.impl;resolution:=optional
Private-Package =
        org.postgresql.shaded.com.ongres.saslprep,
        org.postgresql.shaded.com.ongres.scram.client,
        org.postgresql.shaded.com.ongres.scram.common,
        org.postgresql.shaded.com.ongres.scram.common.bouncycastle.base64,
        org.postgresql.shaded.com.ongres.scram.common.bouncycastle.pbkdf2,
        org.postgresql.shaded.com.ongres.scram.common.exception,
        org.postgresql.shaded.com.ongres.scram.common.gssapi,
        org.postgresql.shaded.com.ongres.scram.common.message,
        org.postgresql.shaded.com.ongres.scram.common.stringprep,
        org.postgresql.shaded.com.ongres.scram.common.util,
        org.postgresql.shaded.com.ongres.stringprep

J-N-K · October 2, 2021, 5:59pm

Maybe tree-show <bundlename> on both machines shows a difference.

lujop · October 2, 2021, 8:43pm

Thanks @J-N-K , I tried, but I don’t see any relevant difference. There are some because one is a nighty build and another M2, and also for other installed addons but in what is relevant to me are equal.

It’s possible a silly mistake, but by now, I’m not able to see anything. It’s very strange the diag fails with the mandatory postgresql dependency, but it seems to be present:

openhab> bundle:diag 280
openHAB Add-ons :: Bundles :: DBQuery Binding (280)
---------------------------------------------------
Status: Installed
Unsatisfied Requirements:
osgi.wiring.package; filter:="(&(osgi.wiring.package=com.mysql.jdbc)(&(version>=8.0.0)(!(version>=9.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.h2)(&(version>=1.4.0)(!(version>=2.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.h2.jdbcx)(&(version>=1.4.0)(!(version>=2.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.hsqldb)"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.hsqldb.jdbc)"; resolution:="optional"
osgi.wiring.package; filter:="(osgi.wiring.package=org.mariadb.jdbc)"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.postgresql)(&(version>=42.2.0)(!(version>=43.0.0))))"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.sqlite)(&(version>=3.16.0)(!(version>=4.0.0))))"; resolution:="optional"
osgi.wiring.package; filter:="(&(osgi.wiring.package=org.sqlite.jdbc4)(&(version>=3.16.0)(!(version>=4.0.0))))"; resolution:="optional"
Declarative Services


openhab> bundle:list |grep -i postgresql
279 │ Active    │  80 │ 42.2.18               │ PostgreSQL JDBC Driver
openhab> exports |grep org.postgresql
org.postgresql.copy                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.core.v3.replication                                     │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.core.v3                                                 │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.core                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.ds.common                                               │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.ds                                                      │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.fastpath                                                │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.geometric                                               │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.gss                                                     │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.hostchooser                                             │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.jdbc2.optional                                          │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.jdbc2                                                   │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.jdbc3                                                   │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.jdbc                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.jre7.sasl                                               │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.largeobject                                             │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.osgi                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.replication.fluent.logical                              │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.replication.fluent.physical                             │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.replication.fluent                                      │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.replication                                             │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.ssl.jdbc4                                               │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.ssl                                                     │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.sspi                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.translation                                             │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.util.internal                                           │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.util                                                    │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.xa                                                      │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql.xml                                                     │ 42.2.18               │ 279 │ org.postgresql.jdbc
org.postgresql                                                         │ 42.2.18               │ 279 │ org.postgresql.jdbc

lujop · October 2, 2021, 9:09pm

At the end uninstalling an installing again the postgresql dependency did the trick.
But I don’t know because I did the same way using bundle:install https://repo1.maven.org/maven2/org/postgresql/postgresql/42.2.18/postgresql-42.2.18.jar

lujop · January 15, 2022, 2:06pm

It happened to me again after updating to OH 3.3.
In the end, it worked but it was a little nightmare of trial and error, it seems that there is some race condition or install/activation time dependency between the driver bundle and the addon one.

Looking at that answer in SO it seems that the correct way to be able to instantiate the driver should be using DynamicImport-Package.
But it doesn’t seem to be supported by a pom property and only org.openhab.automation.jsscripting seems to use it in a bnd.bnd file that I suppose it overwrites the properties in pom.xml, doesn’t do?

I suppose that in persistence addon when the package is installed by feature as the driver is started from the persistence addon it works in a reproducible way.
But I don’t know how to test it, how can I install a concrete feature of an addon I build locally?

@cweitkamp Did you experience any of that problems while developing the JDBC persistence addon?

splatch · January 15, 2022, 3:32pm

Optional dependencies require “refreshing” of imports to take them into account.

If you have that many optional dependencies then best is to pair your dbquery addon with jdbc drivers. By this way you have a base feature named for example dbquery with addon itself and then companion features such as dbquery-mysql which install JDBC driver and base feature. By this way you will solve problem with refreshing of dbquery module just to pick installed driver.
Installation of second feature ie. dbquery-postgresql while dbquery-msql is still installed should trigger refresh of dbquery addon installed earlier too.

Best,
Łukasz

lujop · January 15, 2022, 5:23pm

Thanks you very much for replying @splatch.
In theory I did what you say replicating the feature structure of jdbc persistence addon:
bundles/org.openhab.binding.dbquery/src/main/feature/feature.xml

The problem is that I don’t know how to test this using my home build version. Now, what I do, is leave it in the addons folder. Is there any way I can install one of the concrete features trying to reproduce how the final user install addons?

And about what you explain about refreshing, do you mean bundle:refresh command? Because I tried it several times on addon and driver and it didn’t work. I think that as the driver isn’t used as an OSGI service, as is instantiated dynamically there something more obscure is you don’t have the DyamicImport

splatch · January 15, 2022, 10:37pm

There are few things to explain. I’ll start from feature definition. The prerequisite attribute means that given feature must be fully resolved prior installing anything from your feature. This is especially useful if you have a specific URI (ie. wrap:mvn:somedriver) or other low level requirement which should be there when your binding/addon is activated.
Making openhab runtime a prerequisite=false doesn’t change anything because that is default behavior. This means that your addon+openhab-runtime can be installed and resolved in one shot.

On the refresh command - yes, I mean it. it is there to trigger re-wiring of bundle. Normally when you do bundle:restart the bundle imports are not re-evaluated. This is a case especially when environment is intact (there is nothing new nor gone), and all caches are up to date. The refresh command on other hand cause re-computation of imports/exports and so on so it allows to release imports which are gone (cause exporting bundle is gone or was simply updated), release optional dependencies (which might again be gone due to removal). When you bring a new jdbc driver into runtime you normally should do refresh to let your addon bundle catch it. If you do update it does the same, but additionally it tries to fetch latest version of addon itself.
Reason why refresh might not give reliable results is difficult to catch. I’d say that any static caches on other dependencies you use could be a reason for that. If you change jdbc driver, refresh your addon which (supposedly) also use dbcp, then dbcp will still hold references to the old driver classes.

	<feature name="openhab-binding-dbquery" description="DBQuery Binding" version="${project.version}" hidden=true">
		<feature>openhab-runtime-base</feature>
		<bundle>mvn:org.openhab.addons.bundles/org.openhab.binding.dbquery/${project.version}</bundle>
	</feature>

	<feature name="derby" description="Derby driver" version="${project.version}">
		<bundle>mvn:org.apache.derby/derby/10.12.1.1</bundle>
	</feature>

	<feature name="openhab-binding-dbquery-derby" description="DBQuery Binding Derby" version="${project.version}">
		<feature prerequisite="true">derby</bundle>
		<feature>openhab-binding-dbquery</feature>
	</feature>

Last point, you can’t or at least shouldn’t use Class.forName within your code without passing a class loader argument. This is no-go within OSGi and dynamic import package is a workaround for situations when third party tries to do something very, very odd you can’t amend.
The semantics of Class.forName call is unreliable and might work differently depending when or how call stack look a like. For example a call from refresh command might work differently from a launch of system or restart of the system. This is because Class.forName uses current Thread class loader (so called ContextClassLoader) which is different in each of above situations. Just to give you an idea - you could end up with shell executor, framework launch thread or async launcher of service component (it handles @Component annotations). None of above is suitable to find your JDBC driver. For that reason you can use following constructions:

Class<?> loadClass(String name) throws ClassNotFoundException {
  // correct way to locate own module class loader
  /* 1. */ ClassLoader myLoader = getClass().getClassLoader();
  /* 2. */ ClassLoader myLoader = getBundle().adapt(BundleWiring.class).getClassLoader();
  // correct way to load class from own dependency chains
  /* 1. */ return Class.forName(name, false, myLoader);
  /* 2. */ return myLoader.loadClass(name);
  /* 3. */
  // necessary if you rely on Class.forName
  ClassLoader ctx = Thread.currentThread().getContextClassLoader();
  try {
    Thread.currentThread().setContextClassLoader(myLoader);
    return Class.forName(name);
  } finally {
    // only with class.forName
    Thread.currentThread().setContextClassLoader(ctx);
  }
}

Hope that above helps. Also if you wish to work with jdbc you might have a look on org.ops4j.pax.jdbc, it should take some of troubles from you at the cost of extra dependency.
Best,
Łukasz

lujop · January 17, 2022, 8:54pm

Thank you very much for your detailed explanation it makes a lot of sense for me and I will try it.
Althought I don’t know why the jdbc persistence add-on doesn’t seem to need it.

splatch · January 17, 2022, 9:54pm

If you ask me, then I can assure you that jdbc persistence does that, but over the third party dependencies it embeds. If you would have a look on what org.openhab.persistence.jdbc jar have inside, you would find there three similar things - Yank (a jdbc query wrapper), Hikari (connection pool provider) and DBCP-util (another chunk of different connection provider). That’s the reason why it doesn’t have such code, because class resolving is made by Hikari and probably by DBCP-util (I am not sure now when and why). Have a look on this piece of code:

github.com

brettwooldridge/HikariCP/blob/HikariCP-2.4.7/src/main/java/com/zaxxer/hikari/HikariConfig.java#L315L325

    
      
          public void setDataSourceProperties(Properties dsProperties)
          {
             dataSourceProperties.putAll(dsProperties);
          }
          
          
public String getDriverClassName()
          {
             return driverClassName;
          }
          
          
public void setDriverClassName(String driverClassName)
          {
             try {
                Class<?> driverClass = this.getClass().getClassLoader().loadClass(driverClassName);
                driverClass.newInstance();
                this.driverClassName = driverClassName;
             }
             catch (Exception e) {
                throw new RuntimeException("Failed to load class of driverClassName " + driverClassName, e);
             }
          }

Because Hikari is embedded in org.openhab.persistence.jdbc, its classloader has access to all drivers listed in manifest, only if they are installed.

lujop · January 23, 2022, 9:47pm

A lot of thanks for your explanations @splatch
I applied your suggestions but I still have problems with my production machine.
And I think that the problem is not the way I load the class, because when I do a bundle:headers the optional PostgreSQL dependency appears as red:

Refreshing the dbquery bundle, nor the driver one helps in any way and the result of bundle:headers always shows the postgres dependency in red.
The same running in my develop environment works correctly.

I attach some collected information:

277 │ Active │  80 │ 9.4.1212              │ PostgreSQL JDBC Driver JDBC42
281 │ Active │  80 │ 42.2.18               │ PostgreSQL JDBC Driver
282 │ Active │  80 │ 3.3.0.202201232027    │ openHAB Add-ons :: Bundles :: DBQuery Binding

package:imports -b 282

org.openhab.core.types                 │         │            │ 282 │ org.openhab.binding.dbquery
org.postgresql                         │         │ resolved   │ 282 │ org.openhab.binding.dbquery
org.slf4j                              │         │            │ 282 │ org.openhab.binding.dbquery
org.sqlite.jdbc4                       │         │ unresolved │ 282 │ org.openhab.binding.dbquery
org.sqlite                             │         │ unresolved │ 282 │ org.openhab.binding.dbquery
sun.misc                               │         │            │ 282 │ org.openhab.binding.dbquery

openhab> package:exports -b 281
Package Name                               │ Version │ ID  │ Bundle Name
───────────────────────────────────────────┼─────────┼─────┼────────────────────
org.postgresql.copy                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.core.v3.replication         │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.core.v3                     │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.core                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.ds.common                   │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.ds                          │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.fastpath                    │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.geometric                   │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.gss                         │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.hostchooser                 │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.jdbc2.optional              │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.jdbc2                       │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.jdbc3                       │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.jdbc                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.jre7.sasl                   │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.largeobject                 │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.osgi                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.replication.fluent.logical  │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.replication.fluent.physical │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.replication.fluent          │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.replication                 │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.ssl.jdbc4                   │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.ssl                         │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.sspi                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.translation                 │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.util.internal               │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.util                        │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.xa                          │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql.xml                         │ 42.2.18 │ 281 │ org.postgresql.jdbc
org.postgresql                             │ 42.2.18 │ 281 │ org.postgresql.jdbc

All except the bundle:headers seems correct, but I really know where to look.
I tried start/stop/refresh several times a system:shutdown --clean-caches,…

The only thing that seems to work in my production machine is to use an old version of the driver like the JDBC addon. But I don’t know if the real reason is that it’s already had been resolved by the persistence addon.

splatch · January 24, 2022, 12:36am

You have conflicting results since imports shows that postgresql is resolved while headers command show opposite. Have a look on postgresql driver exports cause coloring of headers verify package name and its version. Maybe for some weird reason, which I can’t tell now, there is a mismatch. Looking at headers and imports command sources I have impression that both do the same!

github.com

apache/karaf/blob/karaf-4.3.6/bundle/core/src/main/java/org/apache/karaf/bundle/command/Headers.java#L313L330


      
                  String sym = bundles[i].getSymbolicName();
                  if ((sym != null) && sym.equals(bundleName)) {
                      if (vr.contains(bundles[i].getVersion())) {
                          return true;
                      }
                  }
              }
              return false;
          }
          
          private boolean checkPackage(String packageName, String version) {
              VersionRange range = VersionRange.parseVersionRange(version);
              Bundle[] bundles = bundleContext.getBundles();
              for (int i = 0; (bundles != null) && (i < bundles.length); i++) {
                  BundleWiring wiring = bundles[i].adapt(BundleWiring.class);
                  List<BundleCapability> caps = wiring != null ? wiring.getCapabilities(BundleRevision.PACKAGE_NAMESPACE) : null;
                  if (caps != null) {
                      for (BundleCapability cap : caps) {
                          String n = getAttribute(cap, BundleRevision.PACKAGE_NAMESPACE);
                          String v = getAttribute(cap, Constants.VERSION_ATTRIBUTE);
                          if (packageName.equals(n) && range.contains(VersionTable.getVersion(v))) {

github.com

apache/karaf/blob/karaf-4.3.6/package/src/main/java/org/apache/karaf/packages/core/internal/PackageServiceImpl.java#L43L62


      
          import org.osgi.framework.wiring.BundleRevision;

          

          public class PackageServiceImpl implements PackageService {

          

              private final BundleContext bundleContext;

          

              public PackageServiceImpl(BundleContext bundleContext) {

                  this.bundleContext = bundleContext;

              }

          

              public List<PackageVersion> getExports() {

                  Bundle[] bundles = bundleContext.getBundles();

                  SortedMap<String, PackageVersion> packageVersionMap = new TreeMap<>();

                  for (Bundle bundle : bundles) {

                      BundleRevision rev = bundle.adapt(BundleRevision.class);

                      if (rev != null) {

                          List<BundleCapability> caps = rev.getDeclaredCapabilities(BundleRevision.PACKAGE_NAMESPACE);

                          for (BundleCapability cap : caps) {

                              Map<String, Object> attr = cap.getAttributes();

                              String packageName = (String)attr.get(BundleRevision.PACKAGE_NAMESPACE);

                              Version version = (Version)attr.get("version");