~That~ Did you already try it with the check disabled or even the complete dependency to checks gone, that could just be a PR?
A mvn dependency tree reveals some small conflicts but none look really interesting:
[11:52:21] : [Step 1/4] [INFO] | | | - (org.slf4j:slf4j-api:jar:1.6.4:provided - omitted for conflict with 1.7.12)
[11:52:21] : [Step 1/4] [INFO] | | - (jline:jline:jar:2.7:provided - omitted for conflict with 2.14.1)
Some code was just merged to improve the compatibility of the static-code-analysis project with older maven version.
Tried it with the static code analysis tool completely being removed from the pom.xml - no difference.
This plan for openhab-core just succeeded: https://openhab.ci.cloudbees.com/job/openHAB-Core/
It also runs the Karaf verification (which fails on the other builds).
Did that and spent many hours for it. All repos should now be nicely decoupled from each other. Unfortunately there is no improvement on Jenkins 
Just released a new version of the static analysis tool with this change, but it does not make a difference (as expected after aboves observation that even without that tool it fails).
I removed our plan and renamed the new build plan and changed it to deploy the artifacts AFTER a successful build to Artifactory (this change cannot possibly have any relevance) - executed it after the renaming and it swiftly failed as well
- Loaded Jenkins config from disk - no effect
- Restarted Jenkins - no effect
ā¦
Again very much out of ideas 
Sorry for thatā¦
Do we fetch the data from the same artifactory as we publish it to? Can it be that we publish (corrupt) distributables which can not be loaded back in properly?
While running on my buildserver it fetches this one remotely:
[19:06:49][Step 1/3] Downloading: https://repo.eclipse.org/content/repositories/snapshots/org/openhab/io/org.openhab.io.transport.feed/2.1.0-SNAPSHOT/maven-metadata.xml
[19:06:49][Step 1/3] Downloading: JFrog
[19:06:51][Step 1/3] Downloaded: JFrog (2 KB at 1.0 KB/sec)
I have compared the log files and found a difference in the maven parameters:
The successful build used these parameters:
Executing Maven: -B -f /scratch/jenkins/workspace/test-zd47405/pom.xml -U clean install -DskipChecks=true -P verify-features
The in the failed jobs the profile āverify-featuresā was missing:
Executing Maven: -B -f /scratch/jenkins/workspace/openHAB2-Bundles/pom.xml -U clean install -DskipChecks=true
Another difference was this message that only appeared in the failed builds (several times):
[INFO] Using Groovy-Eclipse compiler to compile both Java and Groovy files
Can you change the job to create a verbose log of the Karaf feature generation using e.g.
(cd openhab-core/features/openhab-core; mvn -X org.apache.karaf.tooling:karaf-maven-plugin:4.0.8:features-generate-descriptor)
Hm, AFAIK it is part of āaether-apiā
@Kai did you also check if Jenkins is not using some kind of mixed up Maven version? I can reproduce exactly the same Exception when I copy /lib/aether-*.jar from a Maven 3.0.5 to a Maven 3.3.3 /lib installation folder.
Hm, AFAIK it is part of āaether-apiā
From looking onto these jars, I can confirm that.
Additionally, there is a āswitchā in karafās DependencyHelperFactory which either uses RepositorySystem from the sonatype namespace or the eclipse one. From looking at the code, it seems as if it decides for the sonatype one because the plexus thinks it is available, but actually itās not in the classpath.
I donāt really get why it is not reproducible outside of cloudbees. There must be something in the way how maven is started there which is leads to this difference. Nevertheless, I created a build with debug logging and exception stacktraces (i.e. -e -X active) here. Unfortunately it fails too early to give us (me?) any additional useful info and also the effectively used aether versions cannot be seen.
Any further ideas anybody?
Hm, AFAIK it is part of āaether-apiā
Yes, you are right. I probably got a bit confused by the two different libraries with similar names from Sonatype and eclipse.org.
Can you create a Job that executes
(cd openhab-core/features/openhab-core; mvn -X org.apache.karaf.tooling:karaf-maven-plugin:4.0.8:features-generate-descriptor)
only?
So, only the stuff that is really necessary for this goal is in the classpath?
You could then change 4.0.8 to the most recent plugin version 4.1.1 for this goal only.
Yes, I will do that.
Can you create a Job that executes [ā¦] only?
This will work as running the full openhab-core project also succeeds: https://openhab.ci.cloudbees.com/job/openHAB-Core/87/
The problem only starts once it is put together into a reactor with the other repos through openhab-bundles.
Is there any fix where you think it will solve our problem? The āthis goal onlyā part is important - we must make sure that the distro wonāt suddenly use 4.1.x.
Just a quick update: I created an exact copy of the build job (okay, admittedly without the publishing and email post-build-stepsā¦) and reducing the complexity stepwise:
The full build as we know it (using karaf-maven-plugin 4.0.8):
-f ./pom.xml -U clean install -DskipChecks=true
log
=> FAILS
Only doing the karaf feature generation (using karaf-maven-plugin 4.1.1) but still for the whole project:
-f ./pom.xml -U clean org.apache.karaf.tooling:karaf-maven-plugin:4.1.1:features-generate-descriptor -DskipChecks=true -e -X
log
=> FAILS
So the karaf version does not seem to matter. Also the
Only running the karaf feature generation (using karaf-maven-plugin 4.1.1) only for one subproject - interestingly it succeeds:
-f ./openhab-core/features/openhab-core/pom.xml -U clean org.apache.karaf.tooling:karaf-maven-plugin:4.1.1:features-generate-descriptor -DskipChecks=true -X
log
=> SUCCEEDS
Ok, I have some news! The Cloudbees support continued to look into it (I am amazed that they are so dedicated to even support non-paying open source projects so eagerly 
) and hereās what they found:
we discovered that the problem is related to the Deploy artifacts to Artifactory post-build action. Removing this post build action everything seems to work fine.
Why this is happening now and not before is because of the upgrade of the Maven plugin to the latest version on March 31 (this was a massive upgrade in all DAC instance because of a security issue).
We tried to fix the problem upgrading the Artifactory plugin but it didnāt help. Iām still working on it in order to find a solution.
So after all, the problem started through some change on the Cloudbees infrastructure - letās hope that they can find a solution for it; otherwise I might change the builds to deploy the artifacts in some other way. Iāll keep you posted.
7 Likes
Short update: Cloudbees will look into fixing the Artifactory plugin, but this can take many weeks as it isnāt trivial. Iāll therefore go for a workaround and see if I either use a local Maven repo on Cloudbees for passing the artifacts from the bundle build to the distro build or to do a āmvn deployā to Artifactory without using the plugin. As I am on holiday this week, my time is pretty limited - but Iāll make sure to resolve the issue over Easter, so that we have up-to-date snapshot builds again in a few days from now. Will keep you posted.
7 Likes
Ok, first success: I have removed the use of the Artifactory plugin in the bundles and the distro build and instead the artifacts are deployed by the normal Maven deploy mechanism. Both builds are green again, which means that you can again get the current snapshots from https://openhab.ci.cloudbees.com/job/openHAB-Distribution/.
One last thing I need to do is to also populate the online repo from the build. Currently, this still holds old snapshots, so that you will not get the latest bindings, if you do not also download/install the addons package for offline usage. Will work on this next (hopefully tonight).
2 Likes
Hi all,
I just updated my OH2 to the latest snapshot.
But now I get NPEs in zwave binding and the complete zwave control is not working anymore. So I guess there is an incompatibility from latest OH2 snapshot to older binding snapshots?
Just to inform you all.
CHeers MArkus
What are the NPEs? There havenāt been many changes to the master branch of the zwave binding for a month or so - just 3 commits and I donāt think they should change much, but it would be interesting to see what the NPE is.