Addon development

Development
1

Hey developer community,

we need to find a solution for an easy to setup binding development.

Status quo

The current distro uses the Demo-App approach that Markus established and that is also referred to from the openhab2-addons readme and developer documentation.

The problem with this Demo-App is, that it is not based on Karaf and therefore it’s not really related to the openHAB distro. Especially you can’t manually install features in your debug session, that you may need for testing your binding.

Starting Karaf from Eclipse for bundle development is not really supported (actually no IDE is really supported for this usecase by Karaf).

Start Karaf separately

What Karaf developers recommend instead is to just start a Karaf instance in debug mode (which allows a connection via JDWP - Java developer wire protocol).

With our bnd build we lost a good part of native Eclipse integration. We manually need to maven build the binding-under-development, push it to karaf and make sure that a connection via JDWP is established.

Thanks to bndtools however that changed recently. See https://github.com/mrulli/com.flairbit.examples.karaf-debug.

If we’d preinstall

mvn:biz.aQute.bnd/biz.aQute.remote.agent/4.2.0

with the distro, we would allow bndtools to “control” the OSGi container.

Our development workflow would need to be changed to:

  1. Start a karaf openhab instance
  2. Clicking on “Debug” in the IDE does the following automatically: builds the bundle, pushes it to karaf and establishes a remote connection.

The bndrun file that we ship with such a developer-maven project for Eclipse would look like this:

# Override the default bnd launcher
-runpath: biz.aQute.remote.launcher

# Contact the remote agent on Karaf
-runremote: test;\
	shell   =   -1; \
	jdb     =   5005; \
	host    =   localhost; \ # A developer could change that to his testing or production system
	agent   =   29998; \ # Default port of the agent
	timeout =   10000

We only need to solve how to start and stop a karaf instance. Maybe instruct a developer to manually start it together with Eclipse, if the developer does not want to experiment with his production system. But even that is possible with this setup.

This is actually a huge advantage of OH in contrast to HomeAssistant et al, we can develop and test bindings on production systems if we want to, without restarting anything etc. How cool is that.

And: This solution should also work from Intellij and Visual Studio Code.

WDYT?

Terminology used in this post

Jar files: Compiled java classes bundled together with meta data in a zip file results in a jar file.
OSGi bundle: A jar file, but your java classes are prepared to be started/stopped by an OSGi framework.
OSGi framework: Multiple bundles orchestrated by OSGi standards+services.
Allows to dynamically add/remove jars + start/stop services.
Karaf: A solution that embeds an OSGi framework and provides a ton of extra features starting with a decent dependency system (“Karaf features”), Logging, Monitoring, Authentication, Database connections, Persistence support, Docker support, web management console, text console. openHAB uses Karaf because of its dependency management and its text console commands (we don’t really use more than that atm, which is another topic)

Alternative

Drop Karaf and only use the karaf feature service in our own OSGi framework compilation in the openHAB distro as well as the binding development.

Cheers, David

6 Likes
2

Would and eclipse ide, set up for openHAB be something that could be done in docker? Or a vm appliance type thing?

3

In theory we can bundle all OH dependencies including the maven cache and Eclipse into a docker.

But Eclipse would not “run” within that docker container, only Karaf itself. We would expose the Eclipse install path to the host for manually starting Eclipse. With a bundled .bat and .sh start file this is a one-click solution, no-additional-download-time solution. Microsoft is working hard on docker file support. So yes might be viable for all supported operating systems in the future.

A VM instance is not that cool. Cannot be automated that much and someone would need to manually craft the VM instance and maintain it.

1 Like
4

It could be automated with Vagrant which is a pretty standard approach for development in my industry, which probably means it’s considered old and clunky in the wider development world.

1 Like
5

True. Actually what ever is more convenient should win. As both docker and vm images contain a full operating system, it doesn’t even matter size wise. Maybe a bit, the docker image would be a command line only stripped down linux while in a VM we need the full GUI stack.

6

Sorry guys but thinking about a VM for writing ad addons in Java looks crazy to me.

I think many developers will just be fine if a couple of mvn commands are needed the first time a newly created bundle is added:

  1. skeleton commands to create the binding
  2. some mvn commands to the openhab-demo and openhab2-addons project to have the binding included in the build process and dependencies supported

and the all the rest is done from Eclipse using the usual debug framework.

I personally almost managed to have an IDE setup with just openhab2-addons and openhab-demo checked out, with a /lib jar and with 2 features-bundles dependencies , finally after many tries I was able to debug the new binding with Eclipse + felix.
I am not expert at all on OSGi, mvn or bndtools so the setup has some hiccups, for example only one of the 2 dependencies is loaded, but I think here the goal should be to struggle all together (with the support of the real mvn and bndtools experts) to have a full Eclipse+mvn-only setup.

Any other solution too much complicated will make addons developers go away to other systems (maybe learning a simple language like python, if not already known…).

As a side note, I think that was really really “brave” (to say the least…) to decide in 1 move to drop PDE and the previous dev env, without having first all clear (including working examples) about the new setup supporting all development use cases.
It’s not a complain (because it’s clear every maintainer is putting an incredible amount of effort in this), but from a sw enigeering pov that was really a brave move…

7

I’m not arguing in favor of a VM by any means, but I just don’t understand this. The whole point of a VM or Docker based config is to get a full development environment with IDE and OSGI container and everything else present and working you would:

  1. Install prerequisites (type2 hypervisor like VirtualBox, install config engine Vagrant)
  2. Download the Vagrant script
  3. Run Vagrant
  4. Start development

There will never be an “almost managed to” or “hickups” during setup. It’s all already set up and ready to use.

Why should the goal be to all struggle together when the stuff that people are having difficulty setting up can all be automated? You should be able to focus your time on developing bindings, not struggling with build tools or IDE configs.

2 Likes
8

correct, but this “struggle” has to be done only once now to complete the migration from Tyco to bndtools, that was promising many enhancements over the previous situation.

My feeling is that with some good help from the bndtools and mvn experts, the Eclipse IDE setup can be finalised, without having to disturb VMs, which is going around the problems and not solving them.

9

It is automated with an Eclipse setup file. But in that setup file there are many options, so each permutation would require a different image (addons, core, webui, zigbee, zwave, etc.). And the setup file is updated every couple months, like it was two days ago, which would mean updating all of the images. Someone would need to put in a lot of effort to maintain IDE VMs. You also need a good bit of processing power and RAM for an IDE, and a VM adds the needs of the host system and slows things down, so the hardware requirements go up.

10

The other problem is that maven downloads half of the internet in a very inefficient way and a docker image or VM would allow us to ship a pre-populated maven cache.

There are probably other ways, but maven itself is dying (IMO) and you find less and less people with expert knowledge. The next step after tycho->bnd would actually be maven->gradle to stay up to date with current java development patterns. Also bnd works way better with gradle (the bnd project itself is build with gradle).

Gradle development and maintenance is driven by a commercial company (Goggle!) whereas maven belongs to the Apache foundation with no commercial backing anymore (no progress, even the most simple things must be solved with plugins, most of the time outdated).

That’s why we struggle so much at the moment to get a working IDE setup that doesn’t take ages to be usable. Because m2e (maven to Eclipse, yes Eclipse does not speak maven natively!) is not really in harmony with bnd. Also for Gradle there is buildship, a user-interface to start and monitor Gradle tasks (like “check licenses”):

buildship_open_build_scans
buildship_open_build_scans.png1295×590 77 KB

11

I disagree. Maven is old, but it’s still widely used. Development activity is at least as strong as within openHAB.

I think there are still more Maven than Gradle experts out there.

I agree. But who really wants to take the effort for such a migration now? This is another big change and we should carefully weigh if it’s really worth it. “gradle is newer” and “bnd works way better with gradle” is too abstract to justify this migration in my eyes. We really should be sure what will be the exact advantages, especially how the IDE-integrated runtime and debug use case will be covered, because that was the inital issue that you made the thread for.

I’m not sure if this should be a major argument. Be honest, how many people here in the Community really like it to work with Eclipse? IntelliJ is the de-facto preferred solution for most developers meanwhile, we have to accept that. Of course, as a FOSS project we should also support a FOSS development environment. But I don’t think that we should change our build tool just because of flaws within the m2e Eclipse Plugin.

1 Like
12

Yes I agree that we should support both IDEs as best as possible. Fact is, that bnd developers use gradle themselves. See for example https://github.com/openhab/openhab2-addons/issues/5554 where we run into an endless build loop in Eclipse. And the linked source code in the maven bnd m2e code is a big TODO (“will fix later”) with no activity for 2 years.

We will not get any support from bnd for using Eclipse with maven and why should we spend resources here if gradle is the official and supported way. Remember that Google is supporting Gradle while maven has no big company sponsoring it anymore.

I think there are still more Maven than Gradle experts out there.

There are more Android developers (“gradle”) than enterprise (“maven”) java developers.

13

We could go back to OH1.8. Boom. Problem solved.

14

Not really. Try to make p2 repositories and pde work in visual studio code and intellij or netbeans. Eclipse IDE is not what most new developers are feeling comfortable with.

15

FWIW, I’ve never worked on OH in Eclipse or any other IDE (partly because I could never get a working build environment in Eclipse). But developing bindings was incredibly easy using just a text editor and maven.

16

That hasn’t changed. Maven builds fine on command line. This thread is about Eclipse IDE.

17

@David_Graeff
Not to side track this discussion - do you have a guide on how to setup VSCode for OH development. I’d love to switch over to it - but am still waiting until things settle down…

Thanks,
Tim

18

I haven’t put a guide together yet. For actually being not a pain in the ass we need to … ehm… migrate our maven buildsystem to Gradle… (better don’t tell anyone), because that is what VS Code can handle out of the box.

Basically follow the setup here (you need two extensions):

You need to create a launch file where you specify that project should be build, uploaded to a running karaf (openHAB) instance (must be started in debug mode), and that you want to use remote debugging.

Search in the same article for “attach” and enter hostname and port (5005).

Maybe you can try such a setup and report back if it works for you and what pitfalls are to be expected so that we can document the workflow.

1 Like
19

Just started the process (and documenting each step). I’ll report back when I get it done with the steps (and will likely post a new thread for any questions I have along the way).

20

Care to share your launch file to save me some effort? :wink: