Recompiling and testing openhab-core: guidance please

Apologies for a newbie question. I am trying to troubleshoot issues related to nrjavaserial lockfiles under FreeBSD that are currently killing the OH Java runttime, and, possibly, failing RFC2217. Having forked/cloned the openhab-core, I have successfully changed the dependencies to use a newer version of the nrjavaserial, and I have managed to successfully build openhab-core@2.5.0.

Unfortunately, now I am lost in a sea of artefacts, many in my ~/.m2 directory. I do not know what to copy from the new build over the existing installation of OH2 to see if it worked. I could not find any references to that in the guides. Unfortunately, I do not develop in Java (normally).

Any pointers would be much appreciated, thank you.

To test in a live environment, build the package that you updated the dependencies for. You can then stop OH, replace the file under /runtime/ and clear the cache, or uninstall through the console and copy the jar into addons.

Thank you, @5iver. I will rsync ~/.m2/repositories/ over /libexec/openhab2/runtime/system/ then empty OH tmp and cache, start and see what happens.

Thank you, again, @5iver, your suggestion has (almost) worked. I got the newly recompiled runtime working, but it was still seeing the older version of nrjavaserial, even though the new one was meticulously listed in the poms and karaf’s xml. However, once I moved the also just-recompiled nrjavaserial-5.0.0.jar to addons, and loaded it in Karaf, it loaded it. I am now getting different nrjavaserial-related errors, still crashing the runtime, but at least I can see I have made some progress. :slight_smile:

1 Like

May I ask for a little more help, please @5iver or @chris? Since I have managed to help the maintainer of the nrjavaserial library to fix the issues that were preventing it from working with RFC2217 ports, and possibly with some other issues that affected FreeBSD only, I would like to give the new version a good try under openhab-core.

What I do not understand is what is a good way to reconfigure the core to use my locally compiled version of nrjavaserial. For now, I have changed the three pom.xml files that I found in the openhab-core tree (compile, runtime, and karaf) to require the new nrjavaserial from its namespace/group. But even after I recompiled openhab-core I can see that Karaf is still trying to load the old one. I put the new jar in addons, then I try to bundle:install and I get an error, but when I clear the caches it somehow picks it up. It all seems very haphazard.

What is the correct way to:

  1. Reconfigure openhab-core to use my recompiled version of nrjavaserial with all the fixes
  2. Make sure it is loaded instead of the broken one that shipped with OH 2.5
  3. Ensure that zwave binding is using the correct one?

Many thanks for any help. OpenHAB development side is daunting, and I appreciate any suggestions you can make to help me along the way.

Simple - all you need to do is uninstall the old version, and install the new version. You don’t need to recompile the core or anything like that - just the single file.

To uninstall the old file, use karaf console, use the list command to find the bundle number, then use the uninstall command to remove it. You can then put the new file in the addons folder and it should run.

To make it a little simpler, just run these in the console


bundle:uninstall com.neuronrobotics.nrjavaserial
bundle:install -s file:///path/to/my/file.jar
```
This allows you to just clear the OH cache to put things back to stock. Sometimes it is easier to test in a live install rather than debugging in the IDE.

You probably can’t use nrjavaserial 5.x without recompiling everything because bnd will have added imports for packages with a [3.15,4) version range. It uses semantic versioning so it assumes 4.x and 5.x are incompatible. I have taught bnd for OH3 core what is compatible with:

Thank you @5iver and @wborn. I will try both approaches and report. Out of interest @wborn, if I understand it, OH3 core requires Java 11: would the 2.5 bindings work with it? By the way, in case you have not been following the other discussions about nrjavaserial, 5.1.1+ will have, at long last, a functioning rfc2217—the implementation that sat there for a long time was behind inaccssible code


No they won’t work. Not because of Java 11, but because all Eclipse SmartHome packages were renamed from org.eclipse.smarthome to org.openhab.core. We do have a Jenkins build job for porting all OH2 add-ons to OH3 so we can test the add-ons with OH3.

So you can also try the OH3 snapshot build and update the nrjavaserial bundle in it and then every add-on using the serial transport should already work. I.e. not one of those we have to update as listed in this issue. I’ll probably do another PR for adding a version range to the imports of the add-ons still using gnu.io so they work with 3.12-5.x. That way everyone can use their favorite nrjavaserial version. :slight_smile:

The new UI is still a bit buggy though. You sometimes have to manually refresh the page.

Thank you for the suggestions. I had a quick look at the master branch of openhab-addons and it looks like only a small number of bindings made it there—but I can see pretty much all of them in the artifacts of that Jenkins job, except for the http binding (which is still OH1 I believe). It is an intriguing idea to try the new build instead of OH2, though I am a little unsure about its overall stability at this stage—if things do work, I would like them to run my home just as 1.8.3 has been happily running it for a few years :+1: I wonder what is your take on the current OH3 stability vs OH2.5 in “production”—the UI does not concern me?

My current workflow is: I am (just) recompiling OH2.5 with the 5.1.1 of nrjavaserial (which does not have the rfc2217 but is the newest in Maven repos) by having manually edited these to depend on it instead of the OH2 variant that you built:

modified:   bom/compile/pom.xml
modified:   bom/runtime/pom.xml
modified:   features/karaf/openhab-tp/src/main/feature/feature.xml

this way I will have the right imports. Then I will

rsync -a ~/.m2/repository /usr/local/libexec/openhab2/runtime/system

and then I manually

cp ~/nrjavaserial/build/libs/nrjavaserial-5.1.2.jar /usr/local/libexec/openhab2/addons/

then I will do what @5iver suggested, ie. clear the OH caches, tmp, and use Karaf to

bundle:uninstall com.neuronrobotics.nrjavaserial
bundle:install -s file:///path/to/my/file.jar

and see if that works. Does it seem like a reasonable approach?

I wouldn’t recommend using OH3 for production only use it for testing. :slight_smile:

Since you’ve already recompiled the serial transport with a 5.x nrjavaserial version, bnd will have added a [5.x,6) version range to the manifest. So you no longer need to recompile the core if you just want to test new nrjavaserial 5.x versions.

Personally I would only recompile and update the bundles in an existing OH 2.5.x release. But that requires knowing exactly what bundles to update.

1 Like

Ah, you just made me wonder: do I also need to recompile the 2.5.5 zwave binding to take advantage of the new nrjavaserial? Or is its functionality encapsulated through an openhab-core abstraction?

It’s using the serial transport so that’s an advantage this time around. :wink: So it shouldn’t be recompiled.

1 Like

Thank you, @wborn, very much indeed. Without your guidance I would not be able to proceed. I see the core has finished compiling—funny, it got quicker, only 17 mins (but with -DskipTests=true). Fingers crossed I get to the top of what is affecting nrjavaserial under OH today. From my tests of nrjavaserial using a hello world app yesterday I have, at long last, seen it not crash under FreeBSD. I will look for the crash pid file if it does.

1 Like

For an even faster build, try a parallel build, disable static code analysis, tests and Karaf feature validation

mvn clean install -DskipChecks=true -DskipTests -Dfeatures.verify.skip=true -T 1C

Builds in less than 1 minute on my aging quad core i7-6700K CPU @ 4.00GHz :slight_smile:

There’s now also a PR for updating the version ranges of the add-ons still depending on gnu.io:

1 Like

Hmm
 I am still not doing it right. Even though I cleared the system of all variants of nrjavaserial, and cleared tmp and cache, having then reinstalled in Karaf from the file carefully placed in addons, I get this as soon as the ZWave binding tries to open the port (this is trying with rfc2217, I will try the direct route in a few moments):

2020-05-10 10:00:57.309 [INFO ] [zwave.handler.ZWaveControllerHandler] - Attempting to add listener when controller is null
Native Library /var/db/openhab2/userdata/tmp/libNRJavaSerial_openhab_0/libNRJavaSerial.so already loaded in another classloader
java.lang.UnsatisfiedLinkError: Native Library /var/db/openhab2/userdata/tmp/libNRJavaSerial_openhab_0/libNRJavaSerial.so already loaded in another classloader
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1900)

which is followed later by:

java.lang.ExceptionInInitializerError thrown while loading gnu.io.RXTXCommDriver
java.lang.NoClassDefFoundError: Could not initialize class gnu.io.RXTXCommDriver thrown while loading gnu.io.RXTXCommDriver

I suppose I am missing a step in Karaf.

I have tried once again, making sure the only things I have in addons are:

-rw-r--r--  1 root  wheel   713K 10 May 09:23 nrjavaserial-5.1.1.jar
-rw-r--r--  1 root  wheel   185M 21 Apr 16:25 openhab-addons-2.5.4.kar
-rw-r--r--  1 root  wheel   3.1M  4 May 12:09 org.openhab.binding.zwave-2.5.5-20200504.110626-5.jar
-r-xr-xr-x  1 root  wheel    70B 19 Apr 14:08 README

Cleared again cache and tmp, then bundle:uninstall and install of the above nrjavaserial. This time, I can see both the RFXCOM binding and ZWave started. Unfortunately—and unlike when running nrjavaserial without OH in place—I get the Abort trap:

RXTX uucp_lock() /var/spool/lock/LK.255.000.242 is there
Abort trap

I searched for

sudo find / -name "*hs_err*" 

and there is nothing on the system. Do I need to start OH with some flags to tell it to generate the hs_err file when Java aborts? Also, how can I be sure OH is defnitely using the new library—this is what I did in Karaf, but is that sufficient?

list -s | grep serial
214 │ Active    │  80 │ 5.1.1                   │ nrjavaserial

Also, when I tried over rfc2217 I got:

check_lock_status: device is locked by another application
check_lock_status: device is locked by another application
check_lock_status: device is locked by another application

and no attempt to actually connect (whilst monitoring the remote ser2net USB server).

It all feels as if I were not using the new version of nrjavaserial but something older


Debugging OpenHAB is hard. :confused:

If it doesn’t crash the JVM and only logs errors or throws exceptions it won’t generate the file. You can also configure where it will be created using a JVM startup option. IIRC it’s generated by default in the openHAB userdata dir.

Usually I update the nrjavaserial bundle, then stop openHAB, remove the userdata/tmp/libNRJavaSerial_openhab_0 dir and restart openHAB to be 100% sure the right library is used.

After that you’ll see it will recreate the dir and extract a new .so file into it. You can then run md5sum on the extracted .so file to see if the MD5 matches that of the .so file you’d expect it to be.

Does it work properly if you only use one serial connection type? I.e. not both a physically connected device and a rfc2217 connection type? I have never used rfc2217 serial connections or a combination of both.

I cannot be sure. In my case I have both RFXCOM (which works perfectly) and ZWave (which is causing all these issues). The difference is that RFXCOM directly handles the port over TCP (as a bridge) while ZWave expects the openhab-core+nrjavaserial to do it. I have only been able to test (leading to that Abort trap) the RFXCOM + ZWave over a “local” socat’ed port.

I have not been able to successfully test ZWave over rfc2217, despite confirming, yesterday, that I can get connected using a plain Hello World nrjavaserial-5.1.1+ app, because there is another issue: ZWave binding tries to .getSerialPortIdentifier of the rfc2217 port, but for some reason (which I tried to debug) that call goes down the wrong abstraction tree and ends in RxTxPortProvider.java instead of RFC2217PortProvider.java, which then ends up causing “no such port”
 Please see my findings on this GH issue, perhaps you can see something that could get this further.

Unfortunately, I think I have no other option but to give up on trying to fix rfc2217 in openhab. The core’s use of it is incomplete and likely has more issues. I am not competent enough to do it on my own, and even with your extremely helpful hints I would need more support from an existing OH developer to get properly set-up. I realise you and others have other things you need to be focusing on, so with much regret, I think I have to give up. :disappointed: