Using Eclipse with openhab-webui

Having openhab-webui imported as a project in Eclipse tends to go into a refresh/build loop at times, and it can be very frustrating to try to “break the loop”. It also leads to long build times whenever something has been changed in the MainUI bundle (org.openhab.ui).

I got frustrated after waiting more than an hour for this to finish today, and tried to find some information online. It’s not a “general problem” with Eclipse, it seems to be related to M2E (Maven for Eclipse) and projects that doesn’t “properly follow the Maven structure”. In short, it is expected that everything produced by a build ends up in the target folder. Changes to this folder doesn’t trigger a workspace refresh and a new rebuild.

Somthing about org.openhab.ui isn’t organized like that, and I suspect that the problem is that as a part of the Maven build, npm ci is executed. This doesn’t just update the npm dependencies, but actually does a “clean” first and then installs everything. This happens in a folder that Eclipse considers a source folder, so a refresh is queued after the build is finished. Something probably changes during npm ci, like a timestamp for the last update or similar, which leads to the following Eclipse refresh to “find changes” - triggering a new build. And so the story goes…

What I did, that sees promising this far, is that I navigated to the web folder under org.openhab.ui, right-clicked the folder and selected “properties”, and enabled Derived. I think that signals to Eclipse that it’s an “output folder”, so that it shouldn’t be watched for changes. It seems to have resolved the “build loop”, at least results have been good this far.

edit: The above did not work, I’m not sure why it appeared to help initially, but it’s clear now that the problem is much deeper. Actually, I’m not sure it has to do with Eclipse at all, it seems to rather be about conflicting Maven and npm configurations.

I see now that there are multiple issues causing this.

As of now, I’ve identified:

  • web/www This folder is deleted and recreated every build. As it’s in a “source folder”, it will trigger a rebuild.
  • web/node_modules This folder is “cleaned” (as far as I understand) by npm ci and repopulated for every build. Also a “source folder” → trigger rebuild.
  • web/src/assets/build-info.js This file is rewritten with “build info” each build. Contains version and commit hash, will trigger a rebuild if either has changed, during rebase for example.

Does anybody know what build-info.js is actually used for? Its location would indicate to me that it might be used by MainUI itself (in the “about page” or similar), but a full grep of the folders can’t find any references to it.

edit: I think I’ve found it in app.vue. It seems to use the version property only, which is used as the “F7 version”:

      f7params: {
        id: 'org.openhab.ui', // App bundle ID
        name: 'openHAB', // App name
        version: buildInfo.version, // App version

It would be really beneficial to find another way to “attach” this info, as a build should never write to the source folders.

I get little to no response when reporting problems using OH with Eclipse. I don’t quite understand why.

Eclipse is the documented way to contribute to OH. It’s also a brilliant IDE for Java (not for other things IMO) once you get it working. A would-be contributor following the documentation will probably end up facing the same issues. In fact I’ve tried to set up OH’s “Eclipse dev environment” in the past (years) and given up because it doesn’t actually work. This time around I was persistent and didn’t give up, and it’s now mostly working for me, but I’ve “moved quite far” from the documented setup.

How can new contributors be expected to arise when such hurdles exists before they even get to the actual development? Why doesn’t anybody care?

Since I don’t get any response, could somebody please at least give me a hint of why?

It would be helpful to know if it’s in the general direction of any of these:

  • Nobody cares, people have found their own tools and don’t mind if it doesn’t work for others.
  • Nobody knows the answers/reasons and thus have nothing useful to say.
  • The people that do know don’t read the forum (or GitHub issues it seems).
  • The way I present things are unsuitable and put people off.

I don‘t have a real answer for you, but would like to share some experience.
Yes, Eclipse is the recommended IDE for development, but it is not necessary for MainUI development.
I started to contribute to MainUI some weeks ago, nothing committed right now, but there is no need for Eclipse at all to do so.
If you look at the contribution guide un the webUI folder, you can see that all you need is a working openHAB installation and npm/node on the same machine. Furthermore I use VSC (code server) with the VueJS addon.
Having set up all three as services in my VM, I can edit the code in VSC and npm will pick up changes on the fly presenting them through a proxy on port 8081.

I do the same. I do use Eclipse for everything Java, but I don’t do it for mainUI development. I use VSC code for that. I therefore don’t open the mainUI project in Eclipse and avoid the compilation issues.
I also try to avoid having the core model projects open if I can. The XTEXT compilation often has a similar problem with recompilation. If I need to work on it, I switch automatic compilation off.

Sometimes I work in core and mainUI development at the same time. You can perfectly run OH from Eclipse and connect to that from VSCode for the UI part. That makes sure you have a changed core without having to create a full build. I can then use the debugger in Eclipse for the Java core part and the browser VSCode or browser debugger for the UI part.

1 Like

Thank you both for replying.

I too use vscode for JavaScript/Vue/CSS/HTML, so that’s not where Eclipse comes in. I have had the “WebUI” project closed to avoid this issue at times, but unfortunately Eclipse doesn’t even let you view the (Java) source files or resource files with the project closed - so I’ve had to open it at times (working both with Core and MainUI in tandem). If I’ve kept it closed and then open it, you can pretty much guarantee that it will take “forever” before things are up and working in Eclipse again (rebuild, update Maven, refresh +++).

My reason for raising this topic isn’t that I can’t manage, it’s that I don’t understand why it’s left in this state. If you follow the documentation and checks out everything (the default AFAICR), it will include the UIs and it will take “forever” to complete the initial build. I let mine run for two days before aborting it one time… there’s not even a warning in the documentation to keep certain projects closed, turn of “automatic build” etc. Turning off automatic build doesn’t help much though, as the “bnd tools” will start to build everything that’s not built when you try to launch.

So, this is kind of “left up to every individual to figure out”, which is what I find “strange” and counterproductive.

From what I’ve found when “investigating” this so far, it should be compeltely possible to fix. It’s not really an Eclipse issue, it’s a Maven issue. Maven doesn’t expect source folders to change as the result of a build, and Maven itself will also rebuild when it’s not necessary as a result of this. It’s just that it doesn’t do it automatically, but it’s still wasted time and energy if you run Maven without clean (with clean it will delete everything first, so it will always build everything). It’s all about how things are organized. As long as it’s a Maven project, the “Maven rules” should be obeyed, which means that all “build output” should end up somewhere in the target folder structure.

I don’t know much about Node and the build tools around it, I’ve looked a bit at some of it, and it seems that moving the web/www folder to target/www works just fine from my testing this far. It has only been the matter of changing a few configuration files.

When it comes to node_modules I have no knowledge of how the things that need them find them, but I would imagine that it should be possible to “reroute” that too to the target folder. That only leaves build-info.js that I’m aware of. It might be something similar for tests that I haven’t look at, but generally, looking in the .gitignore files gives a pretty good indication of what’s in the source tree that shouldn’t be.

Regarding build-info.js I’m trying to figure out a way to make that information available from a location within the target folder. I see that the “static” resources are routed to target/www, so that might be a way for MainUI to “find” that file…?

Imagine how much easier it would be for everybody if these issues didn’t exist.

I’ve noticed, and have had similar struggles there. I too keep them closed if I can, but I think chances are that this has a similar cause, some files from the build that end up in a source folder. I’m not exactly sure how the whole “model” thing works, but from what I’ve understood it does produce Java source files from “declarative files”. If so, it’s probably reasonable that they have to end up in the source somewhere, so that they are build and used, but even if this is “the problem”, there should be a way to “tell” Maven that this is “an output folder”. It might be a trickier problem to solve, but given that Eclipse is behind both the IDE and EMF, you would think that this “problem” has a solution.

That’s the reason I don’t check in the UI when setting up a new Eclipse instance. Only core and addons are imported into my workspace.

You have to know that though. It suggests including the UI AFAICR. Also, there’s a similar problem with “model”, which you get when you check out core. I ended up importing nothing during installation (I think), and then I imported the projects manually in Eclipse. That was better anyway, since i already had to repo’s cloned in a different location than “the installer thought”. When importing manually, it’s also possible to disable “Build Automatically” before doing the import, so that it’s possible to close all the projects before the madness starts. I then opened only the projects “of interest” and turned on automatic building - and it has mostly worked, although the number of opened projects have grown over time.

It takes quite a lot of time and effort to end up with a procedure this different from the documented one though. I’m not claiming that this is “the best way” either, I just found a way that worked for me. The only thing I can say is that the documented way failed every time I tried it. I also had to disable a lot of the Oomph stuff that kept breaking stuff on startup, but I don’t remember the details.

I think I’ve found a way that improves the situation significantly:

 bundles/org.openhab.ui/pom.xml | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/bundles/org.openhab.ui/pom.xml b/bundles/org.openhab.ui/pom.xml
index ce13438d..39da6257 100644
--- a/bundles/org.openhab.ui/pom.xml
+++ b/bundles/org.openhab.ui/pom.xml
@@ -52,15 +52,26 @@
           </execution>
 
           <execution>
-            <id>npm ci</id>
+            <id>npm clean install</id>
             <goals>
               <goal>npm</goal>
             </goals>
+            <phase>clean</phase>
             <configuration>
               <arguments>ci</arguments>
             </configuration>
           </execution>
 
+          <execution>
+            <id>npm install</id>
+            <goals>
+              <goal>npm</goal>
+            </goals>
+            <configuration>
+              <arguments>install --save false</arguments>
+            </configuration>
+          </execution>
+
           <execution>
             <id>npm run build-prod</id>
             <goals>
@@ -84,7 +95,7 @@
             <configuration>
               <resources>
                 <resource>
-                  <directory>web/www</directory>
+                  <directory>target/www</directory>
                   <targetPath>app</targetPath>
                 </resource>
               </resources>
 bundles/org.openhab.ui/web/build/webpack.config.js | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/bundles/org.openhab.ui/web/build/webpack.config.js b/bundles/org.openhab.ui/web/build/webpack.config.js
index 99646abb..fcb85ad0 100644
--- a/bundles/org.openhab.ui/web/build/webpack.config.js
+++ b/bundles/org.openhab.ui/web/build/webpack.config.js
@@ -31,7 +31,7 @@ module.exports = {
     './src/js/app.js'
   ],
   output: {
-    path: resolvePath('www'),
+    path: resolvePath('../target/www'),
     filename: 'js/app.[contenthash].js',
     publicPath: '/',
     hotUpdateChunkFilename: 'hot/[id].[fullhash].hot-update.js',
@@ -65,7 +65,7 @@ module.exports = {
       }
     },
     static: [
-      path.resolve(__dirname, 'www'),
+      path.resolve(__dirname, '../target/www'),
     ],
     allowedHosts: "all",
     historyApiFallback: true,
@@ -263,15 +263,15 @@ module.exports = {
       patterns: [
         {
           from: resolvePath('src/res'),
-          to: resolvePath('www/res')
+          to: resolvePath('../target/www/res')
         },
         {
           from: resolvePath('src/manifest.json'),
-          to: resolvePath('www/manifest.json')
+          to: resolvePath('../target/www/manifest.json')
         },
         {
           from: resolvePath('src/robots.txt'),
-          to: resolvePath('www/robots.txt')
+          to: resolvePath('../target/www/robots.txt')
         }
       ]
     }),

While webpack is still terribly slow, it seems like the recompilation cycle is gone. Hopefully it will also trigger less often.

I know almost nothing about npm, but I replaced npm ci with npm install --save false. npm ci does a “clean install”, which means that it deletes the node_modules folder and then installs all the dependencies again. This is done every time a Maven build is performed, and even if the files are cached locally, my node_modules consists of 52922 files in 6245 folders. It takes time to recreate the folder structure even if the source is local. npm ci also doesn’t touch package-lock.json (doesn’t change any package versions) unlike npm install. As far as I understand, npm install --save false doesn’t touch the package-lock either, but in addition it doesn’t delete the folder first, so it just checks that all the dependencies are there, installing as necessary. This should both be quicker and lead to fewer changes in the folder structure.

To preserve the ability to do a “full cleaning” of node_modules, I’ve bound npm ci to Maven’s clean phase instead. That means that if you run Maven with clean, the result will be the same, but if you omit clean, you won’t have to reinstall all the dependencies.

In addition to that, I’ve moved the webpack “output folder” to under target. This is generated with every build anyway, so I can’t see how it can hurt that it isn’t “among the sources”. Running the local dev server seems to work just as fine, and inspecting the resulting .jar file it also seems like the app content is the same.

I’ll have to test this over some time, but I think this could be a potential solution to this issue.

3 Likes

I’d just like to add that it seems like the NPM team have some severe problems understanding how the real world works, and that many have switched to yarn to get out of this mess.

There are two files the govern the dependencies, package.json and package-lock.json. The former is created/managed by humans/developers, while he latter is automatically generated. The former can have “version ranges”, while the latter only have exact versions. npm install reads package.json, resolves what it considers the “best dependency versions” from the specified ranges and records the resolution result to package-lock.json.

To have a 100% reproducable build, you’d want to install the exact dependencies in package-lock.json. There is however no flag to tell npm install to do that. You can tell it to not update package-lock.json, but as far as I understand, that still doesn’t mean that it respects package-lock.json when installing dependencies. The only way to install the exact versions in package-lock.json is to use npm ci, but it is very wasteful in that it deletes all dependencies first, run build scripts etc again, thus taking a lot of time for large dependency trees.

The NPM team seems to have a very hard problem understanding this, since they live in a world where npm ci is only used with automated build systems/CI, so that “it doesn’t matter” if this takes extra time, and since the CI instance is likely to be “empty” anyway, everything must be built/installed regardless.

Unfortunately, the real world isn’t like that. Developers also need a reproducable build, and it’s not always possible to differentiate the options between when it’s being run “automated” and not, like in OH’s case where the same Maven build is being used. So, they’re kind of forcing people to “accept” that you will just have to live with waiting for minutes to do npm ci, which takes seconds using npm install, because… they don’t see the problem.

For those interested, some of the argumentation can be found here:

What I’m trying to figure out, is if it’s “safe” to use npm install also when building for a release. There was a change in behavior in NPM 7, MainUI is currently using NPM10. After NPM 7, a new “version” of the lock files exists, “version 2”. I have verified, and the lockfile in use here is v2.

While it’s very difficult to find the exact “rules” for how NPM behaves, the lock files are “respected more” if they are v2. So, with v2, npm install is supposed to work “more like” npm ci, although not completely. If there is a mismatch between the two files, npm ci will throw an error and abort, while npm install will ignore the lockfile for the dependencies where there’s a mismatch. But, as long as these files are “in sync”, the lock file should be resected from what I understand.

Since both files are commited to Git, they should be in sync. When a developer updates dependencies, he/she does so by updating the version ranges in package.json and then run “a normal” npm install, which overwrites/updates the lockfile. As long as these are committed together, they should stay in sync and things should work correctly. It would just have been so much easier if there were an option to tell npm install to only respect the lock file for builds.

TL/DR: I think what I have doen is “safe”, but it’s very, very hard to be absolutely sure.

It is used by Main UI to embed the build version and the build commit hash into the about page.
Main UI needs this info so the UI‘s code that is cached by the service worker recognizes an openHAB update and you don‘t have to clear cache and reload the UI to get the latest UI in your browser.

You cannot put the build-info.js into the Java target folder, because when developing Main UI without caring about the Java code around (which is the case when using VS Code), you possibly don‘t have the Java target folder, because you don‘t run a Java build.

I feel like more and more openHAB devs use IntelliJ and not Eclipse anymore …
Wrt to Main UI, we should make it very clear in the documentation that Eclipse very likely is no good choice for Main UI. You better go with VS Code, WebStorm or any other IDE for the web.

Having those web dependencies outside of the web project’s root dir (i.e. where the package.json is) feels wrong and is against the rules IMO.

It is because it does a production build … for development one should use the webpack dev server.

Your changes to the Maven build wrt to the npm clean install look great BTW, would be even nices if mvn clean would ONLY clean, i.e. just delete the node_modules folder and not install. Install should then be part of the build step only.

The change to the target should be fine as well I think, but I would make it configurable with default behaviour to use www inside the web project to allow building it without needing Maven‘s target folder.

Yarn also uses the package.json with version ranges and has its additional lockfile, same as pnpm …

It only behaves like this is there is no lockfile. If you have a lockfile in place, the lockfile controls what is installed as long as there is no issue between the lockfile and the package file. Normally, you are fine with running npm install, but e.g. if you switched versions, there might be something old left inside node_modules, so it is safer to use clean install.

I would say yes. To be absolutely safe, we could run Maven‘s clean goal when building a release, so we will continue to have the same behavious in the pipeline.

True — but we can make CI run clean install so it will enforce that there are no version mismatches, keep in mind that Main UI PRs not only have a Maven build CI workflow but an additional CI workflow where we can do whatever we cannot do as part of the Maven build.

I wrote some of these things as I was “investigating”, so I’ve since figured out how it’s used and why it’s there. I think it might have been possible to reroute it, but I’m not sure it’s necessary. I’ve concluded that the best thing might be to a fs.readFileSync before writing, and comparing the content (if the file exists). If it were only written if the file didn’t exist or if the content had changed, it would eliminate the problem. The problem when rewriting the file with identical content is that an “OS event” for the modification is still generated, and the timestamp is updated.

I haven’t attempted to do that, as I’m not sure if that’s within my “skill set” (I’m sure I could figure out how to do the read and compare, I’m more worried about proper error handling).

As explained above, I’ve kind of “moved away” from that idea. But, in general, even if you work on it without using Maven (Java isn’t really involved here), wouldn’t you still have the file structure of the project intact? The target folder could just be created if it didn’t exist, that’s what Maven does - I’m just wondering if there is a scenario where somebody would work just with the web folder, without org.openhab.ui as the parent. Beause, as long as org.openhab.ui is there, target could be created in the “correct” place.

I understand that, and I think it’s sad. IntelliJ is a commercial, proprietary product, Eclipse is open-source and a brilliant IDE for Java. “Everybody” used Eclipse until Google decided to go with IntelliJ instead. I just don’t understand why it matters to the rest of us, especially open source developers, what Google does. That said, I do understand that if you’re used to one tool in your day job, you might prefer to work with the same tool as the one you’re used to. Still, let’s not forget what a powerful and fantastic tool Eclipse is (for Java). In my opinion, it’s the best IDE I’ve ever used for any language, as long as it’s used with Java.

So, I think it should be made sure that things keep working with Eclipse.

Yes, people will probably be very frustrated if they attempt to work with JS/CSS/HTML from Eclipse. It doesn’t do those things well at all. As said, I use VS Code for this, but Eclipse is still in the picture, as I use it for “the rest of OH”. MainUI is a bundle of openhab-webui, so it will exist in Eclipse as a result of that. All that’s needed is that it doesn’t trigger a Maven build in Eclipse “constantly”, which I think comes down to mixing “output” and “input” folder roles.

I assume you’re talking about the “rules” of NPM or WebPack or similar, or do you just mean that “it’s not how you’re used to”? I think I found ways to “resolve” this to any location when being read by WebPack, the problem however is that NPM seems to be very inflexible in this manner and simply insists on creating this folder wherever package.json is located. Either that, or it’s the frontend-maven-plugin that has this inflexibility. In either case, I gave up on getting it created in another location.

But, the “fix” is the same here. As long as npm install is used, and nothing else has changed, the file structure won’t change when it’s run, since everything is “in order”. Thus, no file system changes trigger another rebuild in Eclipse. This already seems to be working with my current “solution”.

Yes, but that’s also part of the problem here, because a “production build” is launced for every tiny change in development, because the Maven build is only doing “production build”. This could probably be mitigated by using Maven profiles, but it sounds like it could get complicated quickly, and I think trying to make sure nothing is written when it doesn’t have to is “the best compromise”.

Also, production build or not, for the WebUI to work from Eclipse, the app “folder” with working content would have to end up in the JAR. I don’t think that’s how the “dev server” works - it probably runs directly from disk, while Eclipse runs the JAR.

I’ve already made this change in my “local experiment” - it seems to be working fine:

 bundles/org.openhab.ui/pom.xml | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/bundles/org.openhab.ui/pom.xml b/bundles/org.openhab.ui/pom.xml
index 98d73cec..9e3da699 100644
--- a/bundles/org.openhab.ui/pom.xml
+++ b/bundles/org.openhab.ui/pom.xml
@@ -32,6 +32,17 @@
 
   <build>
     <plugins>
+      <plugin>
+        <groupId>org.apache.maven.plugins</groupId>
+        <artifactId>maven-clean-plugin</artifactId>
+        <configuration>
+          <filesets>
+            <fileset>
+              <directory>${basedir}/web/node_modules</directory>
+            </fileset>
+          </filesets>
+        </configuration>
+      </plugin>
       <plugin>
         <groupId>com.github.eirslett</groupId>
         <artifactId>frontend-maven-plugin</artifactId>
@@ -52,12 +63,12 @@
           </execution>
 
           <execution>
-            <id>npm ci</id>
+            <id>npm install</id>
             <goals>
               <goal>npm</goal>
             </goals>
             <configuration>
-              <arguments>ci</arguments>
+              <arguments>install --save false</arguments>
             </configuration>
           </execution>
 
@@ -84,7 +95,7 @@
             <configuration>
               <resources>
                 <resource>
-                  <directory>web/www</directory>
+                  <directory>target/www</directory>
                   <targetPath>app</targetPath>
                 </resource>
               </resources>
 bundles/org.openhab.ui/web/build/webpack.config.js | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/bundles/org.openhab.ui/web/build/webpack.config.js b/bundles/org.openhab.ui/web/build/webpack.config.js
index 99646abb..fcb85ad0 100644
--- a/bundles/org.openhab.ui/web/build/webpack.config.js
+++ b/bundles/org.openhab.ui/web/build/webpack.config.js
@@ -31,7 +31,7 @@ module.exports = {
     './src/js/app.js'
   ],
   output: {
-    path: resolvePath('www'),
+    path: resolvePath('../target/www'),
     filename: 'js/app.[contenthash].js',
     publicPath: '/',
     hotUpdateChunkFilename: 'hot/[id].[fullhash].hot-update.js',
@@ -65,7 +65,7 @@ module.exports = {
       }
     },
     static: [
-      path.resolve(__dirname, 'www'),
+      path.resolve(__dirname, '../target/www'),
     ],
     allowedHosts: "all",
     historyApiFallback: true,
@@ -263,15 +263,15 @@ module.exports = {
       patterns: [
         {
           from: resolvePath('src/res'),
-          to: resolvePath('www/res')
+          to: resolvePath('../target/www/res')
         },
         {
           from: resolvePath('src/manifest.json'),
-          to: resolvePath('www/manifest.json')
+          to: resolvePath('../target/www/manifest.json')
         },
         {
           from: resolvePath('src/robots.txt'),
-          to: resolvePath('www/robots.txt')
+          to: resolvePath('../target/www/robots.txt')
         }
       ]
     }),

As said above, couldn’t (or wouldn’t) the target folder just be created if it’s not there? As long as the parent folder is org.openhab.ui it shouldn’t matter if you use Maven or not.

I’m no expert with this build tools, in fact I’m quite to opposite. But, I’ve done quite some reading trying to figure this out, and what I’m left with is that while they all have lock files, NPM doesn’t “properly respect” that lock file, unlike Yarn. That is what has frustrated people and moved them. npm ci completely respects the lock file, but npm install does not. So, it’s hard to actually know if it will install what’s in the lock file or go by what it finds in package.json, which is what has frustrated people. Some have resorted to specifying only exact versions in package.json to try to mitigate this, but it doesn’t really help, since many of the dependencies have “fuzzy versions” and the problem remains.

But, as I understand it, after a lot of criticism and frustration, a change was made in NPM 7, which made its behavior closer to that of others, and the whole concept of a lock file (a lock file is frankly useless if you can’t use it to “lock”). So, my understanding is that post NPM 7 it will only ignore the lock file if package.json and package-lock.json mismatches - that is, specify different versions. Since both are committed, it’s probably safe to assume that they don’t mismatch.

As said above, this didn’t use to be the case until quite recently (2021/22) somewhere. But, it seems to have improved now. Still, it would be better if it did like npm ci does and abort with an error if there’s a mismatch, instead of silently ignoring the lock file when there is one.

This is normal across many different systems/languages, which is why Maven has the clean phase and life-cycle. The whole idea is that you only “clean” when you want it to, typically if you have checked out a different commit or when doing a CI build (if a cache is used). That’s why it would be very useful if npm ci could skip the folder deletion for some situations, but it seems that the NPM authors are incapable of understanding this scenario.

I’m pretty sure clean is used when building a release for other reasons already. You “always” use clean when making a release as a “general principle”.

I’m not familiar with this, I haven’t looked at the GitHub runners if that’s where this is done. The “worry” with npm install’s behavior isn’t that there are remnants in the folder, but that if for some reason choose to disregard the version in the lock file and “resolve” a version from package.json instead. But, I think that’s a thing of the past as long as the files are “in sync”.

I have no idea what “Relative CI” does, but the things found here looks to me as they could be done with Maven if that was desired.

Running the linter and the tests certainly isn’t difficult to do from Maven, and you could argue that it should already be done as part of a Maven build. You could make the webpack report too, but I’m not sure how that information is used, so it might be pointless to do depending on that.

To me it seems likely that this “disconnect” is partly caused by this being set up by people that aren’t that familiary with Maven.

I don’t know if the purpose of the linter run is to abort the build or if it’s just for reporting, but it could probably be configued as a “check” if it’s desirable that it blocks the build. The tests should probably be run during Maven’s test phase, so that whether or not tests are run depend on the “normal” Maven options.

I have just checked if that would help and it would not help because the build-info will be updated on every Maven build as it contains a build timestamp.

Related question: Is Eclipse not able to ignore files in the src? The build-info is gitignored … IntelliJ is able to ignore stuff in any folder you like to ignore stuff.

IMO everything required to build Main UI should be part of the web folder. It feels wrong to put stuff related to the web project outside of its folder, no matter whether that’s technically possible or not. I consider it very bad style and won’t allow a change introducing such bad style.

The project should open with Eclipse, yes … but IMO development experience for the actual web JS and Vue code is low priority as I don’t consider Eclipse a web IDE.

node_modules is always in the same folder as package.json, I have never seen this different. This is just how things are with this tooling and we won’t change that. The same way you try to change things to get the project working in Eclipse web devs will come and say what weird stuff we do with node_modules in a different location etc.

It reads the actual source from disk, exactly.

This “disconnect” is because Main UI is mainly a web project and thus we use web tooling for it and try not to squeeze everything into Maven. Unneccassary complex to do so and with overhead we don’t want. So I would say its not desired to to that in Maven, just because Main UI is shipped through Java we will still use the standard tooling for the web.

Ok, let me make absolutely sure if this is one of the things that trigger a rebuild then.

There is a way where you can right click and mark files as “Derived”, but it’s something that each and every developer must do and I’ve experienced that it can “disappear”. I’m not aware of any other such mechanism, even though I can’t say for sure that none exists.

The idea is rather that you don’t “mix” things. Eclipse doesn’t “watch” every folder, only folders defined as “source folders”. And, the issue with the build info might not actually be an issue, because after I moved the www folder to target, Eclipse no longer shows web a “source folder”. It’s hard to know exactly how things work without testing, I currently have another “unmodified” (without my “experiment”) MainUI branch checked out so I can’t test it right now.

Eclipse itself differentiates between a “folder” and a “source folder”, so it isn’t usually that hard to set things up. But, there’s another factor here, M2E, the Eclipse plugin for Maven. To automate things, it makes some of these “decisions”, and I think that might be the reason why things aren’t always so easy to get exactly like you want it.

Ok - that’s a “philosophical issue”, but you can’t build MainUI without the Java part - the servlet that serves it. As I see it, bundles/org.openhab.ui is MainUI, not bundles/org.openhab.ui/web. But, it all depends on what you “consider MainUI” I guess.

Absolutely, I think the best thing here is to merely document that Eclipse isn’t suited for that.

That makes sense, because it seems to me like NPM has “hardcoded it” this way. But IMO it’s very bad practice to mix dependencies and source in this way. It reminds me of how things used to be before we had Git and build tools etc, where you’d typically “dump” the source of your dependencies somewhere inside your own source. Either way, it doesn’t seem like there is anything we can do about that.

Sure, that’s fine - I only said that because you said “what Maven can’t do”. I think it’s generally better to use one build system, but it’s nothing I think is that important. That said, my suggestion of e.g running the tests during Maven’s test phase would only mean to launch NPM for testing, and then picking up the result as “pass” or “fail”, so it wouldn’t prevent you from running the tests “manually” using just NPM.

That’s unfortunate that it makes it even more complicated …

We could do this additionally in Maven, doesn’t really matter if we run the tests twice, and it will make sure they aren’t forgotten when only building with Maven. But I wouldn’t want to miss them running without Maven around, you get feedback quicker and have less command line output around the actual important information. Just keep in mind this increases build time a bit, but one could use -DskipTests and -DskipChecks …

Changing build-info.js definitely triggers a rebuild, also with “my changes”, but for some reason I haven’t figured out yet, this rebuild goes pretty fast. Right at this moment I’m mostly confused as to exactly what triggers what and why. I mean, I know that the trigger is changes in the file system, but further than that, I’m not sure exactly what’s what.

When just changing a single letter in build-info.js manually, the following is triggered in the Maven log:

10.03.2025, 20:38:44 CET: [INFO] Skipping npm install as package.json unchanged
10.03.2025, 20:38:44 CET: [INFO] Skipping npm install as package.json unchanged
10.03.2025, 20:38:45 CET: [INFO] Copying 0 resource from src\main\resources to target\classes
10.03.2025, 20:38:45 CET: [INFO] Copying 0 resource from target\www to target\classes\app
10.03.2025, 20:38:45 CET: [INFO] skip non existing resourceDirectory C:\Repos\Java\openHAB\openhab-webui\bundles\org.openhab.ui\src\test\resources

So it is triggered, but it doesn’t actually do anything, which is why it’s relatively quick. The decisions not to actually do anything are taken by different Maven plugins as far as I understnad, the decisions to not run npm install is taken by frontend-maven-plugin as far as I can tell, while the others are probably taken by maven-dependency-plugin.

This is the when the www folder has been moved, if it’s in the original position, things might or might not behave differently. Potentially, build-info.js isn’t really a part of the problem, except that it does trigger a queued refresh and rebuild when any build is running, which obviously isn’t ideal, but isn’t that problematic either as long as the cycle doesn’t repeat and it is finished in a few seconds.

1 Like

All in all, npm install instead of npm ci seems to work fine from my end. It takes around 6 seconds compared to around 60 seconds for npm ci. That’s still just a part of the total build time, WebPack takes a long time. But, that’s also possible to live with as long as the cycle doesn’t repeat itself, which it has a tendency to do with the current setup (it doesn’t always repeat, and I’m not sure what factor that decides this).

So, as I see it now, the main problem to tacle is the www folder. It doesn’t trigger a rebuild when it’s under target, but since that’s not considered a “good” solution, I’m not sure what the alternative is.

Maybe we could make some parameter that triggered one or the other behavior, so that only the Maven build would build to target/www. The one thing that really makes the current www location “break” is that for some reason, M2E is automatically making the folder a “source folder”. I’m not exactly sure why, but I think it has to do with build-helper-maven-plugin. It’s the only place I can find the folder “mentioned” in the Maven config:

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <executions>
          <execution>
            <goals>
              <goal>add-resource</goal>
            </goals>
            <phase>generate-resources</phase>
            <configuration>
              <resources>
                <resource>
                  <directory>target/www</directory>
                  <targetPath>app</targetPath>
                </resource>
              </resources>
            </configuration>
          </execution>
        </executions>
      </plugin>

It takes the whole www folder and packs it into a “folder” called app inside the JAR file. Nowhere in this configuration does it explicitly state that this folder should be considered a “source”, but I assume that somewhere somebody has hardcoded that a “resource directory” is a “source folder”. I can’t find any configuration that mentions this.

Maybe a different plugin could be found that does this without “marking” this as a source folder, but this is just speculation, as I haven’t really found the mechanism that does this. I think that if it wasn’t “marked” as a source, it would stay where it is. I don’t know if there are others plugins that can do this task, or if it would make a difference though.