But, what is the vision with the template system? You pick a template somewhere, and then the channels are partially filled out for you? Can the template fill out everything, I would assume that user input would be needed, and how would that look? Using ConfigDescriptionParameters like rule templates do?
There is a generic syntax for them but what properties there are for the Thing and Channels are necessarily different between the bindings.
I only know how it appears from the user side of things. And from that side the user creates the Channels manually. You do select the type of the Channel in the process.
If you operate at the Thing DSL/YAML/JSON level, absolutely. Just like with rule templates it’s a simple string substitution. It doesn’t matter how the binding handles it internally.
Very similar actually but that shouldn’t matter.
The same system that works for rule templates.
That’s the problem. The Zigbee binding absolutely supports all standard Zigbee devices. But stuff from Hue, Ikea, Xaiomi, Aquara, etc add their own crap to the Zigbee protocol. So for the binding to support those, special processing needs to be added. So many use Zigbee2MQTT because they have a much larger set of contributors that are better able to keep up with the non-standard stuff all these vendors throw out there.
If I were to make this a tempalte, the baseURL, username, and password properties would be parameters. The rest would be part of the template.
Here’s an MQTT one:
version: 1
things:
mqtt:topic:broker:basement_waveplus:
bridge: mqtt:broker:broker
label: Basement Airthings Wave Plus
location: Den
config:
availabilityTopic: waveplus_bridge/basement/status
transformationPattern:
- MAP:config:map:waveplus
channels:
radon_st:
type: number
label: Short Term Radon Average
description: ""
config:
stateTopic: waveplus_bridge/basement/radon_st
radon_lt:
type: number
label: Long Term Radon Average
description: ""
config:
stateTopic: waveplus_bridge/basement/radon_lt
co2:
type: number
label: CO2
description: ""
config:
stateTopic: waveplus_bridge/basement/co2
postCommand: false
unit: ppm
temperature:
type: number
label: Temperature
description: ""
config:
stateTopic: waveplus_bridge/basement/temperature
unit: °C
pressure:
type: number
label: Barometric Pressure
description: ""
config:
stateTopic: waveplus_bridge/basement/pressure
unit: hPa
humidity:
type: number
label: Humidity
description: ""
config:
stateTopic: waveplus_bridge/basement/humidity
unit: '%'
voc:
type: number
label: Volatile Organic Chemicals
description: ""
config:
stateTopic: waveplus_bridge/basement/voc
unit: ppb
illuminance:
type: number
label: Illuminance
description: Amount of light
config:
stateTopic: waveplus_bridge/basement/illuminance
unit: lx
battery:
type: number
label: Battery
description: ""
config:
stateTopic: waveplus_bridge/basement/battery
unit: '%'
For this one, the “waveplus_bridge/basement” part of each Channel config would be defined in the template and the rest would remain, though I might change the map transformation to something inline instead of a separate config.
In both cases these are generic bindings used to access some third party’s API. If you are using AdGuard Home or WavePlus_Bridge and these were published as templates, all a user would have to do is instantiate the Thing and supply a couple of parameters and they too can integrate these third party tools with OH with much less configuration.
Me too.
What I’m really thinking is a lot fo third party stuff have a relatively simple REST API. Heck, even NOAA’s weather service has a pretty simple REST API. One could support these with Thing templates instead of needing to build a brand new binding.
This could be particularly powerful for modbus I think based on what I know about how that works.
Yes, it’ll need it’s own category. I can’t see these going anywhere in the existing categories. And we should try to support defining more than one Thing per posting. That works for rule templates now so I don’t think it’ll be too hard for Thing templates.
I like how rule templates work now with the list of installed templates collapsed by default. But you can click through to get the list.
Under Add Manually you can get a list of all the templates for that binding to choose from if you are creating a Thing from a template. But it only shows templates for that binding.
For installation, to be consistent with widgets, block libraries, and rule templates there would be a new section under the bindings category in the Add-on store. That’s where you would browse, add or remove the tempaltes.
That would certainly work and require no changes to OH itself. One would just need to create a dedicated category in the forum to post them to.
But we already have all these others supported. It doesn’t see too much of a stretch to support Thing templates the same way.
You’d create each Thing individually but that’s no worse that it is now. And you probably wouldn’t do this for Zigbee in the first place (see discussion above about bindings that already support discovery). And this use case isn’t really the main use case these templates would be used for.
The main use case, as I see it, is someone somewhere figured out how to access all the data from NOAA (for example) using just HTTP Things, no rules no binding (Note, I did this myself a couple years ago and posted a tutorial about it somewhere). They can post a rule template to the forum and now any user can create the Things that can do this to with a few clicks. In the case of NOAA.
However, if there is a change needed, if it works like rule templates, we could have a regenerate button. So if we install an update to the template, or work with a local template, applying changes would just be a matter of selecting all the Things created with that template and hitting “regenerate”.
The full work flow to create a Thing from a template would be (assuming the template is already installed):
Go to Things and click +
Select the binding
Under “Add manually” select the template you want to use
Fill out the two-four properties defined by the template to make that Thing uinique (including ID, label and description)
Hit save
Repeat for the remaining Thing.
This is already a whole lot less work than manually creating each one fully individually and less error prone than copy/paste/edit or using “duplicate thing” because error checking is built in to the properties (e.g. if you need to select a Bridge Thing like you would for a Generic MQTT Thing, that property gives you a list of only MQTT Bridge Things to choose from. But most of the time, only one or two Things would be created from each template.
Updates would be:
Go to Things and select all the Things created with the template(s) that were updated
click “regenerate”
See above for how I see it working.
Here is what I would expect my HTTP thing to look like as a template:
version: 1
thingTemplates:
adGuardHome:
configDescriptions:
host:
# property config for a hostname and port number
user:
# plain text property config
password:
# password property config config:
baseURL: {{host}}/control
refresh: 30
timeout: 3000
delay: 0
bufferSize: 2048
username: {{user}}
password: {{password}}
authMode: BASIC_PREEMPTIVE
stateMethod: GET
commandMethod: POST
contentType: application/json
headers:
- ""
ignoreSSLErrors: true
channels:
protection_status:
type: switch
label: Protection Status
description: ""
config:
stateTransformation:
- JSONPATH:$.protection_enabled
commandTransformation:
- "JINJA:{\"protection_enabled\": {{value}}}"
stateExtension: /status
commandExtension: /dns_config
onValue: "true"
offValue: "false"
mode: READWRITE
last-success:
type: request-date-time
label: Last Success
last-failure:
type: request-date-time
label: Last Failure
Completely filled out for you, and the custom parts are asked for in a parameter.
You’re basically envisioning the same system that exists for rule templates
Still, I think there is something that has to be figured out about the link to ThingTypes/ChannelTypes, because I believe that all the parameters are already defined as ConfigDescriptionParameters there. So, if one could figure this out, there might not be a need to actually define any of that in the template, just to somehow indicate what parameters should be entered by the user.
No, this falls under the same “project” I worked on previously, trying to make the marketplace versioned. Currently, there is no versioning on the marketplace, so the concepts of “updated” or “new version” don’t exist. But, if versioning is ever implemented on the marketplace, they would benefit like everything else.
The way it is for rule templates is that you Ctrl+click the “template badge” to automatically select all rules that use that template. Then you just press “Regenerate”. There is no “automatic warning” that the currently installed version of the rule template is different from the version you used to generate the rule, because - they aren’t versioned.
It would be possible, but quite pointless as I see it. There would be no way for the user to check what has changed and to decide if he/she wants to update or not.
Ok - but I’d still imagine that some of the information could be recycled. Perhaps some of the settings would need to be set in their entirety?
The reason I’m focusing on that, is that the “worst part” of creating a rule template is, in my opinion, to create the config descriptions. Any manual work that could be avoided in that regard, I’m sure would be appreciated.
Pretty much. It’s the same problem as I see it, why not use the same solution?
But you may only want to replace part of a parameter like I showed with the baseURL parameter in the example above.
If the poster of the template is doing it correctly, there should be a change log under “Versions” that list what’s changed.
Not many would be able to do much with more than that (e.g. code diff between versions).
This assumes everyone posting a tempalte does it correctly but if we start to police it we end up back with the same things that were rejected before regarding too much policing.
Yes, and I don’t see “clearly” how this could be done, but let’s say that you could find a way to either only list the name of the property, and if it matches something in the thing/channel type it would use those definitions, or you could specify the whole thing manually. Or even some kind of “inheritance” system where, if the name matched, it would populate it with what’s from the thing/channel type, and then let your setting override those. So, you could get away with just filling in what’s differet.
The thing/channel type definitions are already pre-filled with all kind of “goodies”, like option values, labels, descriptions, the correct context, the type, etc.
That’s not what I meant - this is how it would work with versioning support, where you would know which version you have installed, and which version is offered. This was about using a timestamp for when the post was last updated (or something like that) to trigger some kind of “update available” notification, and my point here is that it would be pretty futile as long as you have no idea what version you have installed. The “update available” would be activated if you corrected a typo in the post, not only when you actually release a new version, and since the user has no way of knowing what he/she has installed, most would probably believe that this meant that they didn’t have the latest one installed, when there’s not really any correlation between the notification and the version.
The focus of a Thing template should be first and foremost to make things easier for the user of the template. Anything that makes the job of the author of the template easier at the expense of the user should be avoided.
Which will not be correct and relevant or understandable taken out of context much of the time. Even if those “goodies” are there, they often will need to be overridden to make the vastly reduced form presented to the user make sense.
And the context may not always be correct for the specific parameter being filled in so that needs to be overridden too some times.
But you could know when you installed it. If you installed it before the update you can assume there’s a new version. It’s not perfect but it is servicable.
What’s the harm here though? The user ends up updating to the exact same version they have now. I’m not saying it’s ideal but if it’s a choice between never knowing an update is available and needing to manually check and do everything compared to needlessly updating a little more often than otherwise would be required because updates to the text of the post were made but not the code, needlessly updating is a far better end user experience over all and a great improvement of what exists now.
In practice, these posts are not updated very frequently.
I don’t see those as opposing things, if it’s easier for the author to make it “properly”, it’s easier for the user to use it as well.
I was just saying that I think it would be beneficial if a way to utilize the existing information was found. I’m not sure how this would be linked, but say for example that the name of the parameter was the link. For fields that should be specified in their entirety, like username, password, hostname etc., only the names would need to be specified in the template, and the info would be populated. When you need partial field information, there could be a “convention” that you prefixed the name with part or something like that, or just that you chose another name, and nothing would be inherited and the entry would be “completely manual”.
How would you know? This isn’t currently stored anywhere as far as I know, and there are major problems with the “installed add-ons” entry in the JSONDB as I’ve mentioned many times before, where it keeps getting wiped out for various reasons.
I know something about this, because it’s basically the same challenge that I’ve been trying to solve in my marketplace versioning branch. There currently is nowhere to store the information, and add-ons of different types are handled very different, so it’s very hard to find some way that can apply to all. Storing it in the JSONDB entry that is created for marketplace add-ons is basically pointless, because it is wiped out for a number of reasons, and a lot of refactorings would be required to rectify that so that this information could actually be something you could depend on.
I’m probably not aligned with most people here, I’m aware, but to me this goes straight to “integrity”. You’ll basically lie and say that there’s an update when there might not be, and you’d leave the confusion of figuring this out to the user. Even if doing a “double update” has no real harm, it’s a waste of the user’s time, to first update the add-on, and then regenerate all the things, when nothing has really changed. To me, “honesty” always comes first, and you should not say that “there is an update” if there isn’t.
And, you still have the same challenge as mentioned above. Where would you store the information about the timestamp of your currently installed add-on/template? The only “logical” place would be in the JSONDB marketplace table, but it would be futile, because it is regularly wiped. I bet that if you open up the JSONDB marketplace table on pretty much any installation, and compare the contents with what marketplace add-ons are actually installed, it will only be correct if you have nothing from the marketplace installed, or before you’ve restarted OH after installing them.
Not if the user has to figure out every topic manually. One of the main uses for this would be to provide, for example, the root part of a URL or an MQTT topic and the tempalte supplies the rest. If the end user is limited to filling out in total only those properties that are predefined by the binding tha’s not possible.
And the perfect wins out over the good and the end users suffer because it can’t be made perfect.
But, this isn’t what I suggested. What I suggested was to try to find a way to reuse these when the whole field is substituted, like for a username, hostname/IP address, password etc. I never suggested preventing a manual definition in the template, or just replacing part of the value.
I’m just expressing my opinion, and shedding light on some obstacles. To me, lying/misleading and “the perfect” is on opposite sides of the scale though, integrity is on the “the absolute basics that have to be fulfilled”, not on the “perfect” end.
How can you show a diff when you don’t know what version or date is installed on the system? This is what I’m trying to tell you guys, that information doesn’t exist.
Please read what I wrote above, this is one of the major challenges with making things versioned in the first place. It’s not a small challenge to get this to work reliably. Sure, it’s easy to just slam it in somewhere and call it a day, but it will fail, and it will cause just more user frustration and long-standing bugs with no easy fix.
