Binding with complex dependencies

Sonic · July 27, 2021, 11:16pm

Hello everyone!

I am developing a next version of Danfoss binding and i am using a library, which has own set of dependencies:

org.openhab.binding.danfoss
|
+org.opensdg.java
 |
 + eu.neilalexander.jnacl
 + com.google.protobuf.protobuf-java

And i am struggling with building a deployable JAR. From what i know, when building a binding, all the dependencies, except those provided by the environment, are bundled in the JAR. At least direct ones.

Initially in my case opensdg was bundled in, but not jnacl and protobuf. I’ve used best of my google-fu and in a couple of hours i came up with my own maven-dependency-plugin configuration. Everything builds fine and i can run the result via app.bndrun.

But a drastic failure happened when i tried to deploy this JAR:

org.osgi.framework.BundleException: Could not resolve module: org.openhab.binding.danfoss [252]
  Unresolved requirement: Import-Package: io.swagger.annotations; version="[1.5.0,2.0.0)"; resolution:="optional"
  Unresolved requirement: Import-Package: com.google.protobuf; version="[3.15.0,4.0.0)"

        at org.eclipse.osgi.container.Module.start(Module.java:444) ~[org.eclipse.osgi-3.12.100.jar:?]
        at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:383) ~[org.eclipse.osgi-3.12.100.jar:?]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundle(DirectoryWatcher.java:1260) [bundleFile:3.6.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startBundles(DirectoryWatcher.java:1233) [bundleFile:3.6.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.startAllBundles(DirectoryWatcher.java:1221) [bundleFile:3.6.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.doProcess(DirectoryWatcher.java:515) [bundleFile:3.6.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.process(DirectoryWatcher.java:365) [bundleFile:3.6.4]
        at org.apache.felix.fileinstall.internal.DirectoryWatcher.run(DirectoryWatcher.java:316) [bundleFile:3.6.4]
2021-07-28 01:55:33.877 [WARN ] [org.apache.felix.fileinstall        ] - Error while starting bundle: file:/usr/share/openhab2/addons/org.openhab.binding.danfoss-2.5.9-SNAPSHOT.jar

What i am missing? I am examining manifest in META_INF/MANIFEST.MF and i see that the whole contents of opensdg is listed in “Private-Package:”, but opensdg’s dependencies are still in “Import-Package:” . What am i missing ? Or am i doing something badly wrong? I see that in .kar file, provided by OpenHAB, things are significantly different, and all the possible dependencies are not embedded in JARs but are placed in the KAR file.

Source codes are here:
Binding: GitHub - Sonic-Amiga/org.openhab.binding.devireg at opensdg-java
Library: GitHub - Sonic-Amiga/opensdg-java: Free and open SecureDeviceGrid protocol client side implementation in Java

Can anyone help ?

ssalonen · July 28, 2021, 9:39am

Hi,

prior read that helped me to wrap my head around the topic: 3rd party Maven dependencies in openhab core

In DynamoDB I documented the logic for specifying dependencies in pom.xml (for my own sake as maintainer as well): openhab-addons/pom.xml at main · openhab/openhab-addons · GitHub . Perhaps you find this illuminating.

You need to decide which ones are compiled in (dependencies less likely to be used by other bundles), and which are referred as OSGi-imports.

When coming up with the right pom.xml and feature.xml statements, I tested using these bash one-liners that you might find useful

Build addon kar by using the karaf:kar: ( cd openhab-addons && mvn --also-make-dependents --also-make --fail-fast --pl bundles/org.openhab.persistence.dynamodb clean install ) && ( cd openhab-addons/bundles/org.openhab.persistence.dynamodb && mvn karaf:kar )

Note how kar is used for testing – jar does not really capture these OSGi definitions and dependencies.

Installing it to clean SNAPSHOT openHAB: ( rm -rf /tmp/oh-test && cd /tmp && mkdir oh-test && cd oh-test && cp ~/src/openhab-main/git/openhab-distro/distributions/openhab/target/openhab-3.1.0-SNAPSHOT.tar.gz ./ && tar xzvf openhab-3.1.0-SNAPSHOT.tar.gz && cp ~/src/openhab-main/git/openhab-addons/bundles/org.openhab.persistence.dynamodb/target/org.openhab.persistence.dynamodb-3.1.0-SNAPSHOT.kar addons/ && ./start.sh ) . After this you need to go to MainUI and install the addon. Monitor logs for errors.

ssalonen · July 28, 2021, 9:42am

I checked quickly your github repo and see you have implemented “your own way” of compiling in the dependencies.

Similar thing is already done automatically:

github.com

openhab/openhab-addons/blob/main/bundles/pom.xml#L532-L574

    
      
          <!-- embed compile time dependencies by unpacking -->
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
            <version>3.1.1</version>
            <executions>
              <execution>
                <id>embed-dependencies</id>
                <goals>
                  <goal>unpack-dependencies</goal>
                </goals>
                <configuration>
                  <includeScope>runtime</includeScope>
                  <includeTypes>jar</includeTypes>
                  <excludeGroupIds>javax.activation,org.apache.karaf.features,org.lastnpe.eea</excludeGroupIds>
                  <excludeArtifactIds>${dep.noembedding}</excludeArtifactIds>
                  <outputDirectory>${project.build.directory}/classes</outputDirectory>
                  <overWriteReleases>true</overWriteReleases>
                  <overWriteSnapshots>true</overWriteSnapshots>
                  <excludeTransitive>true</excludeTransitive>

This file has been truncated. show original

As you have noted, this is not done for transitive dependencies, but this is not a big issue: many addons just list all the dependencies, including transitive, as ‘dependencies’.

EDIT: corrected formatting

J-N-K · July 28, 2021, 3:53pm

Why don’t you add the other dependencies to the dependency section of the pom? Usually there is no need to change the plugin configuration.

Sonic · July 28, 2021, 5:50pm

Yes, i’ve seen this; and the two deps i need are transitive.

Sorry, what does it mean: “as ´´” ? Do you mean list them in , like @J-N-K is suggesting ?

@J-N-K This indeed helps to build (almost) correct manifest. All the packages are listed correctly, but there’s one more catch. protobuf imports sun.misc as optional, and ";resolution:=“optional” " gets lost in the resulting manifest. And sun.misc is excluded from OpenHAB environment, so i get:

! Failed to start bundle org.openhab.binding.danfoss-2.5.9.202107281741, exception Could not resolve module: org.openhab.binding.danfoss [135]
  Unresolved requirement: Import-Package: io.swagger.annotations; version="[1.5.0,2.0.0)"; resolution:="optional"
  Unresolved requirement: Import-Package: sun.misc

org.osgi.framework.BundleException: Could not resolve module: org.openhab.binding.danfoss [135]

J-N-K · July 28, 2021, 5:54pm

Add

<properties>
    <bnd.importpackage>sun.misc;resolution:=optional</bnd.importpackage>
</properties>

to the pom.

Sonic · July 28, 2021, 5:58pm

@J-N-K Thank you very much, it works!

Sonic · July 28, 2021, 6:18pm

Just for some learning i decided to leave maven-dependency-plugin and try to understand why my configuration is broken:

I’ve noticed that i had “package”, while OpenHAB’s own config doesn’t have it. So i’ve removed it and now all three deps are unpacked in one shot; and MANIFEST.MF is generated correctly. It’s not overwritten any more.I guess by default the plugin runs in some more appropriate phase. compile? prepare? This Java stuff is so badly documented…
I keep as @J-N-K suggested.

This also works.

J-N-K · July 28, 2021, 6:28pm

The documentation is here: Build System | openHAB and there it reads:

Optional parameters available for importing/exporting packages (e.g. foo.bar.*;resolution:="optional") are available, too. Packages can be excluded from import/export by prepending ! in front of the name (e.g. <bnd.importpackage>!foo.bar.*</bnd.importpackage> would prevent all packages starting with foo.bar from being imported).

Sonic · July 28, 2021, 7:10pm

Finally deployed the new binding on my home system. After some testing i’ll proceed with the release