Heads up: JSR223 is broken in recent snapshots (starting 1310)

I’m pretty sure this went into build 1310, and appears to have mangled JSR223…

Welp, that sucks. I knew about the issue and have been following github closely but apt upgraded today accidentally, which installed the latest snapshot. Looks like you can’t even roll back to a working version because the archive doesn’t go back that far. so i’m stuck with 2.3.0 for now :expressionless:

That’s one of the reasons I run a manual install.

1 Like

rolling back to 2.3.0 doesn’t really seem to work, so i’m stuck with a broken system right now. why isn’t there some kind of archive with older snapshot releases, it’s what… a couple MBs? i’ll see if i can find some useful deb files in the apt cache archive of my last sdcard image

I whole heartedly agree with you. This was not just a breaking change. It was a very large change that, from my understanding of the ESH discussions, was only partially baked and known to break automation. It looks like people are scrambling to get it pieced back together, so hopefully there will be a snapshot soon with a fix. Surely there are more factors that we aren’t aware of, as to why it was merged into the OH snapshots. IMO, snapshots should be fully functional, with all known issues for new features, especially from ESH, resolved before being merged into master. @kai, could you please comment on this?

I’ve noticed recently that Cloudbees does not appear to have the manual installs for previous snapshots, which it used to. I had also checked in JFrog, but couldn’t find anything. Archived snapshots are possibly (likely) there, I’m just not at all familiar with JFrog or working with the apt-get install. So, you might still be able to rollback, but I’m no help with that.

Got it working again, so it’s all good :blush:

reminds me of opensource projects i have been working on, the larger it gets with more and more contributors working in parallel it can get pretty hard to maintain and to keep an overview. Should be avoidable with proper branching but sometimes shit just happens. It’s just annoying when it goes unnoticed/unfixed for a longer time and when there are no recent working build backups

same, i’m glad i could find the .deb in the apt cache from a recent sd-backup

A snapshot isn’t a release build, nor a beta build. They generally offer no guarantee of quality past the fact the code managed to successfully compile.

true but shouldn’t a massive change like this have its own branch in the repo and not affect master? i dunno, it’s probably easier to merge this way

You left out the important part… :wink:

I expect there will be issues. That’s part of why I use the snapshots in order to contribute to the testing of new builds. But I would not expect that anything would be merged that was only partially implemented and break a good chunk of functionality, at least without some sort of communication. I was aware that ESH was working on the automation changes and was watching their progress closely, but I would not have expected these changes would have been merged into the OH master before the known issues were resolved.

I’m not flaming here… I’m putting on my release manager hat and questioning how we determine completeness of a PR and its readiness for a merge, as well as the communication of builds with known issues. The ESH changes, which are not being continuously integrated into the OH master, already have some checks in place to buffer the downstream affects those changes could have on OH. But what happened this time and how do we harden the process to prevent it from occurring in the future?

1 Like

I’d prefer to merge this soon and tackle possible bugs as follow up PRs. After all, the current implementation is buggy as well, so waiting with the merge until everything that might come up is fixed would endlessly delay the merge. Separate PRs will also allow us to work on different issues in parallel.

That was a month ago. IMO it should have been put on a different branch considering how many people are using snapshot builds, they probably didn’t expect it to be this much breakage. Too bad i’ve avoided Java in my 30 years coding experience because now i’d love to help out with ESH. JSR223 is the most interesting part of Openhab imo, there should be a much heavier focus on it as it is so powerful and versatile and it’s where the actual AUTOMATION happens. (Xtext DSP .rules are a joke - there i said it :smirk:)

Also btw, does anyone have the .deb of 1309 or any other recent build before the “event” ? Thanks!

Personally I wouldn’t consider the master branch (the codebase the snapshots are built from) stable - changes are developed in their own branch until the author is happy at which point they submit the pull request with the changes. The changes are reviewed and if the reviewer is happy these changes then get merged back into master. There’s always the chance that there are regression issues once merged, there are issues not picked in the unit tests or the unit tests don’t cover all scenarios.

snapshots should be fully functional, with all known issues for new features, especially from ESH, resolved before being merged into master.
@kai, could you please comment on this?

How do you define fully functional? No PR is merged without ALL unit tests passing. The unit tests are the safety net to make sure that refactorings do not break anything crucial.

The issue at hand is: There are no unit tests for the jsr223 support. There isn’t any documentation on how scripts should be written. There is no maintainer that cares about jsr223 related issues in the ESH issue tracker and who would check the implications on jsr223 support of bigger rule engine refactorings that might not be covered by unit tests.

So imho the right question to ask is: Why was jsr223 support ever merged without being covered by tests and documentation? Well, the hope was that this would be added soon after it had been merged, but unfortunately, nothing happened anymore. If nobody steps up as an active maintainer of this code, I do not see it ever become a standard (meaning stable) feature of openHAB. Note that at the moment the whole new rule engine is still marked as “experimental”, so everyone using an undocumented experimental feature in a developer snapshot build should not be too surprised if things change or break.

I did an analysis of the issue last night and came up with this PR, which made me create JS rules that were executed correctly. Note that the required syntax has slightly changed, so existing rules will have to be adapted.

2 Likes

Why a developer would choose Xtext DSL rules over JavaScript or python is beyond me, JSR223 really deserves more attention and should be taken more seriously as it lifts the product on a new level imho, @Kai , it’s so powerful, it’s like OpenHAB on steroids. I started using OH only 2 Months ago, spending so much effort on Xtext rules, despite following design patterns and trying to keep my code as small and efficient as possible i was unhappy with rule performance, readability of code and it all felt more like a hack that could break at any given moment.
So i decided to give JSR223 a chance, i opted for JS, used https://github.com/lewie/openhab2-javascript as base and started translating my collection of rules to JavaScript, which took me around 5 days inclusive bug fixing, it was a bit of work but the result was very elegant, readable code that executes much faster, loads faster, compiles faster und seems to use less memory too, it weighs in around 2000 lines now instead of 3600+ and i could drop a couple bindings like http, exec, mqtt, expire, snmp, owntracks and more, as well as a few translations like json xml etc as this can be done in JS much faster and with simpler code.

I regret wasting so much time on Xtext DSL and i’m not sure why it isn’t being phased out and why JSR223 isn’t the standard yet, Sure it needs to be documented more but the web is full of documentation for JS and Python, there aren’t that many OpenHAB specific parts in JSR223 and it’s actually simpler to learn than Xtext DSL rules syntax (even for non-coders) . So i’m not sure why new users are being forced to dive into the complex, outdated, buggy, hackish mess that is the Xtext Rules DSL when Javascript or Python are scripting standards for all kinds of applications all over the world, well documented and tested. And as for the simple user with no dev background, they should expect some kind of learning curve with applications like this, and judging by the posts in this forum most of them seem pretty patient and like to learn new stuff, but why not learn some simple,basic scripting in javascript/python, something that is actually useful in the real world, instead of Xtext which seems to be very restrictive and only seems to exist in the Eclipse/OpenHAB context (and which i’ve never heard of in my 30 years dev experience prior to OpenHAB)

Just my two cents and i hope i didn’t come across too harsh, i’m not trying to flame here, i just think OpenHAB has much more potential and would attract a lot more people if it would use a more popular approach in scripting.

edited to add: I plan to opensource my set of rules, maybe it helps others and there’s some pretty cool stuff. A JSR223 tag in the forum would be helpful too and i might update Lewie’s set of basic js rules / triggers / helpers. If my Java skills were better i would love to help with the Codebase but i kind of suck at Java and don’t really have much time to learn atm between work and family

3 Likes

FTR, I have updated the included js demo rule in the distro, so tonight’s build should serve as a reference that jsr223 is working again.

@illnesse I fully agree with you that the jsr223 is really really powerful, so don’t get me wrong: I definitely do not want to remove it and I very much believe that it should be further evolved and maintained - that’s why I try to encourage people to step up and help with it (@smerschjo and @lewie being definitely excellent candidates, but as they both seem to be pretty busy, any other volunteers are very welcome!). A good starting point could be joining the discussion about open jsr223 issues.

One feature where I think the DSL rules are much better for beginners than jsr223 rules is the IDE support: By having LSP-support for them, there can be syntax-validation on-the-fly as well as profound content-assist. Especially beginners can easily struggle to figure out what methods are available on an object, etc., so having content assist is imho very helpful. Also the validation at editing or startup time is imho much preferred over runtime errors only when a rule is executed.

I don’t know how much of such IDE goodies is possible for the jsr223 rules as well - but this could also be something to look at by adding support in the openHAB VS Code extension (for this, you do not even have to know Java, so @illnesse you might want to check with @kubawolanin what might be done there). And definitely another great thing to help on is the documentation page, for which no Java skills are required either.

3 Likes

nashorn will be deprecated in jdk11 :frowning:

1 Like

Yes, I am aware of that, but this does not affect jsr223 in general - so Jython, Groovy etc. will still work. And I do not see openHAB requiring jdk >=12 any time soon, so running it on Java8/9/10/11 with Nashorn should be fine for the foreseeable future.

1 Like

I’m working now using OH snapshot 1319. I’ve created a demo.py, submitted a PR for the OH documentation, updated the ESH Wiiki, and submitted a PR for the OH2-Jython-Scripters readme.md. I’m working on a PR for updating the OH2-Jython-Scripters modules.

You may be referring specifically to the Javascript JSR223 documentation, but there is actually documentation for Jython JSR223 at https://www.openhab.org/docs/configuration/jsr223-jython.html.

While it doesn’t provide all the features you list, I’ve had good results using the free community edition of JetBrains PyCharm IDE with Jython. This IDE provides code syntax checks and other edit-time code “inspections” and fast access to Python documentation. For advanced users, it’s possible to run snippets of code against OH bundles directly in the IDE or to write unit tests using Python’s standard unit test libraries.

1 Like

For anyone using the openhab2-Jython triggers.py, I’ve updated it to include the new Trigger constructor. I’m waiting for another PR to be merged. but until then you can find it here.

@illnesse, I’m not sure if you use this module. If you do, this change will let you use the latest snapshots.

I’m currently having issues saving rules after the changes, see https://github.com/eclipse/smarthome/issues/5939#issuecomment-408590292

VS Code (or sometimes PHPStorm) for JS Rules here, which does the usual syntax validation and suggestions.