I’ve never written a binding before so am probably talking from ignorance. Most of my rules are designed to work around binding changing their behaviour. Or alternatively trying to perform the function of bindings using items/rules in conjunction with execommand and javascript/python scripts.
I know the next version of openhab will put the ngre in centre place. So I was wondering if it was possible or there was much thought placed on having a consistent language/rules system across binding and rules.
A good example is say the milight binding. Just fundamentally the way it works if you want to turn the lights off, first you need to send a command to select the light, which turns them on before you send the off command. The mighlight lights are quite unreliable, so say you turn the lights off, you have to send that command twice with a delay, or twice over two different bridges. But obviously if things work perfectly they will turn off, then to max brightness and then off again. It would be great if a rule I could override this behaviour. So say I want smooth dimming that’s not implemented in the binding there is no need to select the lights for every command, I select the the device and just need to send the next command. I don’t think it’s reasonable to expect the binding author to add lots of code to their binding to cover this edge case that only I care about. So if I could override certain function that would be great.
The other point is that since the items/rules are completely different to how I would write a binding I kind of have to start from scratch can can’t use any of the rules/items I written. If bindings/items/rules shares a conman language that would make it much easier.
From what you describe this is a limitation of the milights. There is nothing that could be done from a Rule or a Binding to change this. It’s just how they work (or don’t work in this case).
That doesn’t sound like much of an edge case and it seems something perfectly reasonable to implement in the binding.
The whole purpose of the bindings is to adapt the interface to the specific technology to the standardized OH interface. Rules are designed and should continue to operate on the OH interface (Items, Things, Channels, Links). This is where the power of OH comes from. This is what allows us to write Rules that work with any device from any technology for which there is a binding.
I have real reservations about fuzzing this line between binding and Rules.
Well, bindings don’t use Items or Things or Channels to interact with the technology. Things and Channels are how the binding exposes the devices to the rest of OH.
I don’t see how we can ever reach a place where bindings are written in the same languages as Rules. All of the 350+ currently existing bindings would have to be scrapped and rewritten in a brand new programming language. We can’t even get developers to upgrade their 1.x bindings to 2.x. Telling them they need to rewrite everything in python or some other language would do nothing but create a mass exodus of developers from the project.
Similarly, forcing users to have to write Rules using Java would similarly be a huge burden on the users. Java is overly verbose and not very concise. It’s also not interpreted so users would have to compile their Rules. I just can’t see this as being acceptable.
What might be possible is the ability to write some binding like stuff using the JSR223 Rules. These support Jython, JavaScript, and Groovy and have the power to build something binding like. However, there is no way to distribute these as a binding install able from PaperUI (for example) which would greatly limit their audience.
I agree with everything in your post, but I feel the need to qualify this statement. I believe it is possible to do this… it just hasn’t been done yet. There is also the question of whether the maintainers would allow something like this in the official distro, but they could be kept in something like the marketplace. JSR223 can be used for much more than creating rules, or being used in them.
Because I am not really involved with development, I could caveat all of my postings at all times with “at this time”. As with any software system, all things are possible. Some may be more probable than others. But I focus on the now out of a lack of knowledge and because in most cases (not all) what could be possible is less important to the questioner than what could be possible. In this case what could be possible is probably more important.
Honestly, depending on what it looks like, the way we distribute Rule Templates may be exactly what is needed, but of course then we have this fuzzing between bindings and Rules which I’m uncomfortable with.
My post is mainly about what “could” be possible and thinking about long term direction rather than just what’s currently on the books. You’re right that java probably isn’t the right language for rules. If thinking about the next generation binding for v3/5 or 4, maybe jython/js support would then mean you have 100x as many people capable of upgrading those old gen 1 bindings. Also like the NGRE it wouldn’t make all the old rules and bindings obsolete.
At the moment you have various setups that should be bindings being run thought execcommand with various supporting items/rules. They are messy/buggy and work differently on windows/linux. That’s probably a bad thing, so I wanted to open up the discussion about what can be done to make it easier to move those kind of setups to binding.
I believe Scott is working really hard to bring the current Xtend Rules to the NGRE. So those Rules won’t become obsolete. But there will be strong pressure from me and others to no encourage new users to use it. There are very significant problems with the Xtend based Rules DSL and with all the work Scott and others have done on the JSR223 support libraries, the Xtend based Rules DSL is really no longer easier for new users to pick up and be quickly productive in.
I have no doubt that the Rules DSL will be around for many years to come, but it will be legacy. Of course, this is just my own opinion. But I do know I will be pushing users hard to not use it the moment the PRs are accepted that make Jython and the helper libraries a default part of the OH distribution.
Well, like I said above, the hard part that isn’t solved yet is distribution. Right now bindings are distributed as OSGi bundles which, if I’m not mistaken, pretty much requires that they be written in Java.
There is a to be implemented(?) concept for distributing Rule Templates through the IoT Marketplace.
Concept 1: Instead of trying to write bindings this way, just write Rule Templates.
Advantages:
We already have a planned distribution approach
Disadvantages:
They lack the same sorts of service contracts that OH proper bindings follow that allows configuration and management of them using the REST API (and therefore through the UIs).
Questions:
What about Rule Templates that require third party libraries? In order for this to work the dependencies also need to be distributed.
Concept 2: Figure out how to bundle a JSR223 binding as an OSGi bundle
Advantages:
We can use the same distribution mechanisms as current bindings
Disadvantages:
There will have to be a skeleton built and maintained for each supported language binding developers can build upon.
There sill might need to be some Java wrapper to let us turn it into an OSGi bundle.
Questions:
Is this even possible?
Concept 3: Create a whole new parallel plugin approach that supports JSR223 bindings.
Advantages:
Binding developers don’t have to mess with OSGi bundles and can remain in a more native packaging approach.
Disadvantages:
Now we have two almost completely separate and parallel add-on management systems (perhaps 4 if each language needs it’s own) which needs to be maintained and doubles the amount users need to learn to install and manage bindings.
Personally, I find none of these options to be acceptable. They all have significant limitations that make them hard to support.
I’m very open to other approach ideas or discussion to refine these. But we all need to realize that it’s more than just creating APIs for the various JSR223 languages. That’s the easy part (and if I understand correctly largely already done). The hard part is bundling and distribution in such a way that average joe users do not know and don’t need to know that this binding is Java and that binding is Python. We also need to be cognizant of the fact that some developer(s) will need to actually build this mechanism and maintain it.
I have developed an extension service for using npmjs for providing additional webuis. That can be extended very easily to support any JavaScript package. (Npm is for JavaScript projects only afaik). We just need to agree on install paths and how to adapt the jsr223 library search path.
For python scripts we would probably use the registry that is used by pip with another extension service.
Every new package, tagged with openHAB, would appear automatically like eclipse marketplace extensions.
If scripts can indeed be used as bindings, that would allow a very low entrance barrier.
Well for me I developed a javascript wrapper for tuyapi. It was the first time I’d ever written anything in javascript and it even required users to perform a mitm attack to get the keys for every device to it used. So I never expected anyone else to use it but too my surprise quite a few people are. I’m now contributing and helping out the main tuyapi project but even still porting tuyapi(node js) over to java is probably way too much for me.
I found a way to use the cloud to do some of what we want in a simple easy way and can probably make a binding for that, it seems like it should be a POP. It would be cloud only, and lose lots of the control and functionality of the tuyapi (lan) based scripts. But if I could just write a good js binding that calls tuyapi(node js) then that’s simple and easy. It’s kind of perfect for what I want to do.
If you look at spotify control, that was done via a python script for at least a year before a binding was made available and lots and lots of people were using that.
@rlkoshak So what I’m trying to say is what is the idea system and setup in 2 years time in an idea world, ignoring any of legacy stuff. If that vision is good enough then it seems to make sense to considering and then start worrying about the existing setup and implications, to see if it’s viable. I have no delusions that it takes time and lots of effort to do that, I’m just asking the question is that what we should be moving to.
Maybe we actually want to have a compatible binding systems with other home automation systems? I can make a single extension now that works on firefox, chrome opera, and as a bookmarklet on ie. Should that actually be the next goal to make home automation better? Should you be able to make a binding in node and just have nodeRed as the universal binding system between everything. Sure that’s ridiculous now, but my question is what do we want in the future? What’s the best thing for HA in general. My view has always been I want open system to be able to pick and choose bits and pieces, I love how openhab works but if there’s a home assistant binging that works better or actually exists I would like to use that with openahb. I know that kind of impossible at the moment but should that be our 5/10 year goal?
What makes you think it would be any easier to rewrite your script as a JavaScript OH binding? to be a binding the code has a number of things that it must do and it must do it in a certain way. At a minimum you would need to drastically refactor your script to become a binding.
If all you want is something that calls your existing script then we already have that in the Exec binding and executeCommandLine Action.
We will never be in a situation where anyone can write any old script and be able to call it a binding. Now it might be less work to port a JS script to a JS binding, but it isn’t going to be just a little bit of work.
Ideal for whom? The users or the developer?
I’m very concerned about repeating one of the biggest mistakes that was made in OH 2; there’s more than one independent way to do things.
I see this very easily going down the same path we have with text configs and REST API based configs. Two+ separate binding ecosystems. Two+ separate ways to manage and install bindings. And very confused and upset users because of the confusion this will generate.
I don’t care what language bindings are written in. But from a user’s perspective, if they have to know and care what language a binding is written in in order to use it we’ve failed them and have managed to make the same mistake all over again with the current two independent ways to configure OH. If they have to use npm to install the UI, pip to install this binding because it was written in Python, IoT Marketplace to install that binding because it was written in Groovy, and whatever replaces PaperUI to install this third binding we’ve not only failed the users, we’ve failed them in exactly the same way we’ve already failed them before, only more so because we are now looking at half a dozen completely different ways and systems to do the exact same thing. And we should know better by now.
A binding is a binding and that’s all the users should need to know. No matter what it is written in, it should be found, installed, managed, and documented in the same way.
If I’ve learned one thing in past years working for and with FOSS projects, it doesn’t much matter what we want to do in the future. What matters is what a developer is willing to work on and make happen. It doesn’t help to have plans and goals and great ideas if there is no developer willing to actually implement it. So stuff like this tends to bubble up from the bottom, not imposed from the top.
In this case, I have no doubt that David and Scott will indeed work to pursue this idea. So this idea probably has a lot more legs than most.
But I’ve laid out my concerns. And my concerns have nothing to do with whether or not we should support bindings written in other languages. I think that would be great and obviously see the advantages. My concerns are strictly about the usability of it. If the devs come up with an approach that avoids remaking the same mistakes we made with the bifurcated config we have in OH 2 I’ll jump on board and maybe even write my own bindings sometime. But I can’t say that I support setting up N separate and independent add-on management approaches, even if N is only 2.
I think setting up separate management systems for bindings would be a hugely bad idea.
The problem being addressed here is already a problem and it’s already a problem that developers are working to solve. Just look at the proliferation of <Technology>2Mqtt bridges out there. Zigbee, Insteon, dht22, etc. Heck, I’ve even written one in sensorReporter. MQTT is certainly becoming the langua franka of home automation. If you support that you will work with any home automation system. Just implement the Homie convention.
But, it comes at a cost. Users have to find and install the software on their own and they are all different in how they are used and configured. But that is what we are talking about adding in Concept 3 anyway. If we are going to impose that cost on users in the first place, why go through the effort of making it integrated with OH in the first place?
You raise lots of good point, and often it’s the practical point that are more important that than theoretical ideals. I felt that I wanted to raise my point of view for discussion, it may be impractical, or something we want to move towards.
What makes you think it would be any easier to rewrite your script as a JavaScript OH binding? to be a binding the code has a number of things that it must do and it must do it in a certain way. At a minimum you would need to drastically refactor your script to become a binding.
I think your point about mqtt was quite interesting and I’ve seen it raised elsewhere, should that be a core binding or feature of openhab? Rather than make users install it should that be supported automatically? I know someone took my script to use as a base for a mqtt scritpt, is that the way we should be going?
Well I’m just saying it’s impossible for me to write a java binding since my wrapper javascript is simple a wrapper script around a nodejs script tuyapi. To try and port that script and every underlying script is crazy. So it would actually be possible to write a javascript binding, whereas it’s pretty much going to be impossible for me to write a java binding doing the same thing.
I’m only considering writing a binding because I found an alternative cloud method that doesn’t rely on another nodsjs script. And in fact the cloud implementation is based off an HA system written in python. So I’m effectively porting a HA python script to java a language I’ve never even written a hello world in.
Your right that maybe this is a choice for the devs, but actually if we make the home automation systems for interoperable, they will be able to focus on a specific aspect rather than the ecosystem in general? I don’t have the time or capability to contribute at the moment so you are correct, backseat driving is easy and maybe counter productive, but I felt it was worth raising. I’m not criticising what being done or currently planned just felt it was a thought worth raising to be considered.
It’s always worth having the conversation and raising the issues. But at some point someone needs to take up the effort to make it happen. That’s the key part that many forget. It could be the best idea in the world but with out a developer to implement it, all we have is a nice (and sometimes not so nice) discussion that heads nowhere. That’s all I mean. This is a case where I think there are two developers interested in pursuing it so this conversation is more productive than most actually.
But I’d like to see more “this is what OH needs to do and this is how I’m going to implement it” as opposed to “this is what OH needs to do and someone needs to implement it.”
@David_Graeff I like your suggestion and if you have a fork or something in place I’d be happy to test it out with either or both python/javascript scripts that I use all the time.
I will make a pull request for my extensionservice for webuis probably next weekend. I haven’t thought about javascript bundle requirements though. npm fortunately forces a developer already to state the author, website, license, version, tags and dependencies (same with pip for python).
But we need a few more meta data like required openhab runtime version, required openhab extensions, breaking changes over different versions etc. That could probably be done in a json called “openhab.json” or so. Maybe you and also others want to think about a file structure and write a white paper / specification.
The second part we need to specify is where to store those automatically downloaded/synced libraries so that they don’t collide with user scripts.
I’m more than happy to have a play and write up what I find or think would be useful. I’ll probably need a play around to see how it works and try various ways to get things working, but sure.
