while reading the thread about the direction of openhab 3 I recognized the problem supporting oh1.x bindings with less ressources.
I walked through some bindings and would like to categorize them as following:
simple Interfaces via any standard protocol (i.e. SOAP, REST-API)
individual Interfaces via individual or unusual protocols
complex logic (like astro)
combined types
To reduce the mentioned problem, I wonder if anybody tried to create a Generic Binding Generator? At first for the simplest interfaces to substitute easily as many bindings. At the end there would be only individual Interfaces, where there effort is too high.
Does anybody has any eperience? Or is it worth for me to try this step on my own?
If anybody tried it or has suggestions, feel free?
How I would start?
I would take an easy OH1.x-Binding, design a describing XML-File for this binding. In a next step I would try to generate the files of the equivalent OH2.x-Binding via own code out of the XML-File.
My only concern would be testing. How do you know whether something broke if you donât have the devices or access to the API the binding interacts with?
One of the biggest challenges is the OH 2.x paradigm is fundamentally different from the 1.x paradigm. The way that itâs configured and used can be radically different between the two. Compare the simplest example I can think of, Exec. Look how dramatically different they are configured and used. Iâm skeptical that this can be captured in a generic manner because what makes sense will largely depend on the nature of the technology one is interacting with.
Does it support automatic discovery?
Does the config file need to be replaced with a Bridge Thing?
etc.
I like the idea and anything that can make it easier to migrate OH 1.x bindings would be quite welcome. Iâm skeptical that there is a generic way to do it though.
There is a new PR that is about interacting with UDP based protocols by transforming the binary data stream into JSON. The idea is that all UDP based OH1 bindings can be re-implemented easier that way.
So others picked that idea up to make it easier.
But in general a generic binding generator is not feasible in my opinion. The oh1compat could maybe modified to map OH1 item configurations to thinks and channels in a heuristic way.
Thanks a lot for your feedback.
Okay, maybe I am too enthusiastic.
Regarding your concerns Rich:
Testing
At first you are right. I can only test, what I have. In a second step testing become easier, because you are able to generate the test cases out of the xml-description of the binding. So I see following possibilities for testing:
Interface Openhab - Binding:
Should be easy and generic.
Procedures within binding and the Interface to the protocols
In an easy way the test cases could generate easily for bidirectional channels (state - command - changed state)
In further steps you could work with mocks and test the mocks against the origin binding and than test the generated binding against the mocks. But this is hard stuff.
Paradigm Change from OH1 to OH2
I would use an existing OH2 binding as an standard setup, of course. The new features like discovery wouldnât be implemented at first. In further steps you could implement it, so that the number of individual bindings could decrease. But letâs do small steps.
Approach
I should make clear, that the generator wouldnât be intelligent. He wouldnât analyse old OH1-bindings and generate the description file. This would be a job. The benefit would come later.
David, thank you for you for mentioning the new PR.
I like you, bringing me back to the ground of the earth. Thank you.
Sorry for wasting your time.
To bring a small idea into this project behind the generator:
From my architecture view it would be helpful to reduce the inteligence of real bindings. Theese should only be interfaces between things and 3rd party systems. This would help to test them. In further steps you could exchange or transfer bindings from other open source systems. As you can imagine the description xml-file is not openhab specific. The point is not, that I want to support other systems, but if it is easier for system provider to support a binding, they would do it without support by the community I assume.
Further it would help to create a binding layer for complex logics like Astro or expire, similar to the rule templates Kai introduced in his video 2017. The advantage would be, that the inteligence, which is now within a binding could be recycled easilyer.
From my perspective, I donât have anything to loose without time. So, maybe, I come to the point that I try the a first draft. Then we can discuss on that basis again.
Iâve been working on some kind of binding generator. This is based on the OH2 thing definition xml files. Currently the thing xml files are not of any help for the developer, since there at source code level no dependency between the xml file and the source code. So any change in the xml doesnât trigger anything for the developer something should change. Itâs all runtime based.
However these xml files contain a lot of information that could be used to help the developer. I started working on a generator that would create a set of interface/classes based on these xml files. I have some very early minimal code because I wanted to reuse the parsing of the xml files from the core. My implementation takes the parsed output and generates some interface/classes using javapoet. However the xml files are parsed at runtime, not compile time. So this parsing is only available through an osgi service. To get access to the parser at compile time these classes need to be available to a binding at source code level, which is not the case. However, because the core has been moved to a pure maven build this should become easier I think, and I kind of hold off to wait for this maven build to become available to see how that could be used.
My idea is to first write out how a generated a binding would look like and from that build the generator.
I havenât completely thought out how it should look, but my idea was to generate for each thing a class and an interface. The interface contains methods for each defined channel both refresh/onHandle calls and a class that calls the interface. The developer than should implement the interface and handle the binding specific logic in there. As mentioned, this is a rough idea how it might work. I havenât completely thought out everything.
This is all based on what is currently possible. The next level might be to research common patterns and see if it would be possible to come to some kind of xml definition from which either code is generated or at runtime certain functionality becomes available. Think of discovery support, or like all bindings have background scheduled tasks, is this something that could be generalized? I donât know what the possibilities are, or if it would even make sense because itâs so diverse it doesnât add any help the developer because there would be to much exceptions.
(To relate to OH1: to make it work these thing xml files should be created, which would still take some effort).
I was once going the other direction, by generating the xml based on class annotations. We discussed that at the Smarthome day '18.
I think your approach is superior, because this idea is extendable by offering some kind of WEB UI to create the XML files which in turn create a full binding skeleton with all Thing and Channel handlers.
You are right. I mentioned that this is a job and will take some effort. I see less effort in the further support of a binding and hopefully the migration, if it is a simple binding which aligns to solved pattern.
Please donât start dreaming. Letâs do small steps.
From my position the first milestone would be to create a generator for a first binding containing following:
XML-description file with a understandable documentation
Skeleton of a standard binding mentioned by Hilbrand, so that you can start the compiler without errors
First Connection to external system to migrate the first binding
Generate Testcases for the commands
After this you can discuss, whether this way is feasible or waste time.
Please letâs stay at the ground of the earth. As I recognise by reading some bindings I saw simple bindings like Tankerkönig (sorry, I donât want to hurt somebody) and several complex ones where I thought, that not everything can be generated.
I think we are talking about the current XML files, used to describe a binding, things and channels. They have a full xsl schema file, so having a web UI for those is not far fetched (those exist already).
At the moment a binding developer has to repeat himself 3 times. In XML, in java and in the documentation. If everything is generated out of the XML files, it will be very easy to create binding skeletons. But that has nothing to do with migrating or porting anything.
I missed some information regarding the code (protocol etc.), so I decided to create my own binding.xml and create the thing-type.xml out of it. Therefore both are very similar. My one will be a small extention.
At this stage I created all files, like they would be created by the skeleton. I created testcases to check to check all files against the skeleton-script. (I only have minor problems with Java-Files because of the format, which I will solve). The solution based on templatefiles with placeholders and Generatorclasses for each produced file, so it is easy to change in future. I read the updates on OH3. To create the testcase for the OH3 skeleton would be easy, too.
You mentioned, how I would start now designing the prototype binding. I would like to create one handler for polling status-updates. The outer procedure should be equal over every protocol. Than there is a inner part for choosing the right protocol and a further part for extracting the right information.
Further as you mentioned there should be the handler for the commands - as you mentioned per thing one class and interface. Further these commands of the channels are translated to commands for the specific protocol procedure.
Do you already have something I can look at? Iâd be interested in working with it.
This sounds like it requires to run a command/script. My idea is to add compile level annotations that start generating code when added. The idea is also that itâs an integral part of the system. Meaning it has a clear separation between the generated and handcoded part. Generated files canât be edited and changing a xml file will automagically update the generated files. So itâs not like an extended skeleton script that would generate certain code for certain use cases and than the developer would use that generated code as the starting point for further development. From your description I could not deduce what your approach is?
While I see points of having that I would like to point you and @hilbrand to one article which is on annotations: The case against annotations. Annotations were introduced to Java in⊠| by Adam Warski | SoftwareMill Tech Blog.
Main point is - annotation checks are passive, you are very limited in what you can express there. You can use only constants and enums there making it impossible to extend by others to their cases. Writing annotation validation logic or code generator requires you stick with annotation processing API in Java which has bad renown and it has it for a reason.
Given current nature of OH2/ESH XML definitions which are not that basic but not yet as complicated as JPA mappings I would say that having kind of âdefinition builderâ would be a good start. Annotation processing API can still rely on it. Thanks to that you keep also annotation processing logic away from core business which is - building a model representation.
Iâm not to deep into model types but if we could have model + builders alone and possible to be used by various maven/gradle plugins we could improve tooling also by adding model validators on top of it without getting dirty of xml schema and schematron.
After testing is now fine I only clean up the javadocs and then will upload. Then feel free.
You ask me for my approach.
At first, I know what I can and what not. So I am a friend of my small steps to achieve benefits.
As I analyse myself, I see the âscript/commandâ in every part as the important part, until we change the approach totally. May I describe my planned evolution:
create skeleton â> done
create prototype for an existing binding with an architecture, which allow to change the interface to the 3rd party.
create hopefully 5% of the inferfaces to 3rd party for 80% Of the existing bindings
there will be a rest with painful individual interfaces to 3rd party.
â-
Now the big change:
Do we really need for the existing interfaces own thingclasses? I donât think so. It is only necessary for the human developers. We could reduce them to one generic thingclass. But I need more knowledge about the possible interfaces and things.
At this point we are able to integrate a new âbindingâ just by adding a xml-file during runtime. That is what you like! Only for the painful complex bindings we need anything else.
Within openhab you could change the design, that one âold bindingâ (our generator-binding) may offer several âvirtual bindingsâ described in the xml-file.
My described path make the generator not necessary at the 5th point. I know it. But at the moment, I think the architecture for the 5th step is not as stable as I like, because I do not have the knowledge about the structure of all bindings.
From an IT-security and testing perspective I donât like compiling at runtime. On the one hand it is too complex for integrating and testing and on the other hand during an attack I would try to add malicious code into the XML-File which would be compiled and run. For that purpose we would need to check all commands within the xml-file.
Sorry, as I mentioned, my knowledge is too low / I have not as much time for that.
Because I am not part of the architecture team, I donât want to define the master architecture of all followed bindings. I see me as a really small light. I donât know, who is in that position or could do this?
For me it would be helpful to create the master architectural design of our prototype binding containing the pulling class of the status updates and the discovery method as sceleton. Then it should be easy for me to update the xml-file and create it. I wouldnât waste time in defining the for other developers âwrong binding designâ.
