Sorry for the delay, I had to resurrect the old machine to get the command line option I used.
After compiling my own copy of libNRJavaSerial.so I added this line to the java $prog_args in the startup script:
-DlibNRJavaSerial.userlib=/opt/nrjs/libNRJavaSerial.so \
where /opt/nrjs was just where I had the library sitting to be used.
Like I said, not sure this will help, but in my case it was a version mismatch with what was available to me, and what was being requested by OpenHAB.