How to run available tests from IDE?

Development
, ,
1

Hi.
Can anyone help me with how can I run the tests that are available with the bindings from the Eclipse IDE?
I tried various things but nothing worked. It is possible that I don’t run it right or my environment is not complete or they stopped working.
Appreciate your help,
Pawel

2

This is a known issue (see here) and is currently worked on (see here).

3

I also have issues running the tests - both in IDE and in Maven in the openhab-core repo. I am currently working on #628 but cannot get the necessary tests green - even on master. It’s a clean checkout. ANy ideas?

[INFO] openHAB Core :: Integration Tests :: Binding XML Tests FAILURE [01:22 min]
[INFO] openHAB Core :: Integration Tests :: 1.x Compatibility Layer Tests SUCCESS [ 33.985 s]
[INFO] openHAB Core :: Integration Tests :: Configuration Core Tests SUCCESS [  4.908 s]
[INFO] openHAB Core :: Integration Tests :: Configuration mDNS Discovery Tests SUCCESS [ 11.605 s]
[INFO] openHAB Core :: Integration Tests :: Configuration Discovery Tests SUCCESS [ 18.401 s]
[INFO] openHAB Core :: Integration Tests :: Network I/O Tests SUCCESS [  5.408 s]
[INFO] openHAB Core :: Integration Tests :: IO REST Core Tests SUCCESS [ 12.121 s]
[INFO] openHAB Core :: Integration Tests :: Model Thing Test Support SUCCESS [  0.565 s]
[INFO] openHAB Core :: Integration Tests :: Model Thing Tests SUCCESS [ 19.722 s]
[INFO] openHAB Core :: Integration Tests :: Storage JSON Tests SUCCESS [  4.300 s]
[INFO] openHAB Core :: Integration Tests :: Thing Tests ... FAILURE [  1.775 s]
[INFO] openHAB Core :: Integration Tests :: Thing XML Tests FAILURE [03:54 min]
[INFO] openHAB Core :: Integration Tests :: Transform Test  SUCCESS [  5.924 s]
[INFO] openHAB Core :: Integration Tests :: UI Tests ...... SUCCESS [ 14.148 s]
[INFO] openHAB Core :: Integration Tests :: UI Icon Tests . SUCCESS [  8.750 s]
[INFO] openHAB Core :: Integration Tests :: Voice Test .... SUCCESS [  6.241 s]
4

Thanks so much for working on that issue @afuechsel! :+1:
I think it’s currently the most frustrating issue.

The current master builds just fine on my machine as well as:

What might help is:

  • Cleaning your local Maven repository
  • Removing generated files from your Git clone
5

Unfortunately this does not help. Removed the local m2 repo, cleaned the git. Again several tests failed in 3 different projects:

[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ org.openhab.core.voice.tests ---
[INFO] Installing D:\afuec\workspaces\openhab-master\git\openhab-core\itests\org.openhab.core.voice.tests\target\org.openhab.core.voice.tests-2.5.0-SNAPSHOT.jar to C:\Users\afuec\.m2\repository\org\openhab\core\itests\org.openhab.core.voice.tests\2.5.0-SNAPSHOT\org.openhab.core.voice.tests-2.5.0-SNAPSHOT.jar
[INFO] Installing D:\afuec\workspaces\openhab-master\git\openhab-core\itests\org.openhab.core.voice.tests\pom.xml to C:\Users\afuec\.m2\repository\org\openhab\core\itests\org.openhab.core.voice.tests\2.5.0-SNAPSHOT\org.openhab.core.voice.tests-2.5.0-SNAPSHOT.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for openHAB Core :: Integration Tests :: Binding XML Tests 2.5.0-SNAPSHOT:
[INFO]
[INFO] openHAB Core :: Integration Tests :: Binding XML Tests FAILURE [01:18 min]
[INFO] openHAB Core :: Integration Tests :: 1.x Compatibility Layer Tests SUCCESS [ 45.393 s]
[INFO] openHAB Core :: Integration Tests :: Configuration Core Tests SUCCESS [  5.385 s]
[INFO] openHAB Core :: Integration Tests :: Configuration mDNS Discovery Tests SUCCESS [ 11.836 s]
[INFO] openHAB Core :: Integration Tests :: Configuration Discovery Tests SUCCESS [ 19.376 s]
[INFO] openHAB Core :: Integration Tests :: Network I/O Tests SUCCESS [  6.680 s]
[INFO] openHAB Core :: Integration Tests :: IO REST Core Tests SUCCESS [ 13.483 s]
[INFO] openHAB Core :: Integration Tests :: Model Thing Test Support SUCCESS [  0.875 s]
[INFO] openHAB Core :: Integration Tests :: Model Thing Tests SUCCESS [ 19.251 s]
[INFO] openHAB Core :: Integration Tests :: Storage JSON Tests SUCCESS [  4.898 s]
[INFO] openHAB Core :: Integration Tests :: Thing Tests ... FAILURE [01:21 min]
[INFO] openHAB Core :: Integration Tests :: Thing XML Tests FAILURE [03:39 min]
[INFO] openHAB Core :: Integration Tests :: Transform Test  SUCCESS [  6.059 s]
[INFO] openHAB Core :: Integration Tests :: UI Tests ...... SUCCESS [ 14.017 s]
[INFO] openHAB Core :: Integration Tests :: UI Icon Tests . SUCCESS [  8.751 s]
[INFO] openHAB Core :: Integration Tests :: Voice Test .... SUCCESS [  6.759 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  09:04 min
[INFO] Finished at: 2019-03-06T10:43:21+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal biz.aQute.bnd:bnd-testing-maven-plugin:4.1.0:testing (default) on project org.openhab.core.binding.xml.tests: 1 errors found -> [Help 1]
[ERROR] Failed to execute goal biz.aQute.bnd:bnd-testing-maven-plugin:4.1.0:testing (default) on project org.openhab.core.thing.tests: 1 errors found -> [Help 1]
[ERROR] Failed to execute goal biz.aQute.bnd:bnd-testing-maven-plugin:4.1.0:testing (default) on project org.openhab.core.thing.xml.tests: 1 errors found -> [Help 1]
[ERROR]
6

Found the possible reason: all tests are green on MacOS, several tests fail on Windows. All are somehow related to the SynthethicBundleInstaller.

7

It is indeed a very hidden bug in the SyntheticBundleInstaller. I will provide a PR: https://github.com/openhab/openhab-core/pull/633

1 Like
8

OK, but my original question remains valid. What is the proper way of running tests under IDE?
Let me say what I do and please correct me:
In Eclipse, I go to homematic binding source, right click on homematic.test.launch and choose Run As->Run org.openhab.binding.homematic.test

What I get is first a warning, that there are no constraints on homematic and homematic.test dependencies. If I click continue, it fails right away with unresolved homematic bundle.
If I try to “add required plugins” from the run configurations plugins tab, it fixes something and show other unresolved dependencies on javax.xml.bind and openhab.core.test

Any hints? I have never ever run any tests in openHAB, so please help me.

9

Good question, I am also not able to run the tests properly in the IDE. What you describe is the normal way to run tests in a PDE environment, as it was with ESH and with openhab formerly. Now it has been changed to bndtools and AFAIK you need to run the itest.bndrun file (right click, run as BND OSGi test launcher (JUnit) ) and pray. Pray because very often it does not work, e.g. I am not able to run the openhab.core.thing tests in the IDE. I do not know whether or not the itest.bndrun needs to be resolved first (and how to handle the the displayed optional dependencies). A clear description of how to run tests inside the ide would be helpful. Additionally I would like to know, how I can run specific tests inside the IDE and in maven, because the usual -Dtest=testclass property is not working for bnd-based tests.

10

Thanks for your response. This is all new to me. I can see two bindings (astro, avmfritz) moved tests to a new itests directory and itest.bndrun exists for them, but not under IDE. But the bindings disappeared from the repo at the same time. No idea what it means and where it goes. I’ll try to search for some PRs and discussions about it to learn.
What I’m really trying to do is to write some tests for my binding. Any hints where should I start at with the current situation?

11

There are two kinds of tests: Unit tests that checks classes, methods for correctness and integration tests, that need part of the framework to run in the background. The latter ones are a bit harder to start from the IDE at the moment until everybody is familiar with bndtools, all tools are at the right version etc.

If your binding is already using the new buildsystem (which it should, because new binding pull requests are not accepted anymore for the old build system), your unit tests go into the src/java/tests or src/java/test directory (not sure, please have a look at the astro binding). Eclipse provides a JUnit->Run tests option in the context menu when right clicking on the project in the project explorer.

Cheers, David

12

David, plain unit tests are not an issue, integration tests are. They are running fine in maven, but are a nightmare in the IDE. This is related to bnd tools. There should be some short HowTo somewhere, if the transition to bnd is finished.

13

There is no complete documentation yet, but this description may help:
https://github.com/openhab/openhab2-addons/pull/5050#issuecomment-470459971