I also wanted to add the ability to specify the list of addons to be installed in the yaml file. It’s a bit like addons.cfg, but you can also list marketplace addons in there. The benefit is that multiple yaml files can independently list which addon(s) they each require, so each file is a complete package, and the combined list of addons then get installed.
But some people rejected the idea so it didn’t go ahead. I still have it stored in my branch somewhere I think.
It’s not the idea that is the problem, it’s the implementation. There are numerous reasons why that wouldn’t work. Maintaining/installing add-ons is already a challenge as it is, it would be very challenging to make mixing this in with general YAML files a good experience. As was discussed before, making a YAML alternative to addons.cfg would be doable, but there needs to be one authoritative source for this, which is why you’ll already get into trouble today if you try to both install from the UI and use addons.cfg.
As far as I remember, you argued that if the YAML file had to be separate, it was pointless. I don’t know if that’s true or not, I have never considered using file-based add-on configuration, because I appreciate being able to install marketplace add-ons using the UI.
This was a deliberate part of the whole YAML config file design. You should be able to modularize it as you want to, and you can freely use folder structures and file naming to aid you in how you want to organize it.
If you add YAML Composer into the mix, you can do a lot with the YAML configuration as it is now.
There are still a few things missing, like transformations and scenes.
It has been a while so I forgotten many of the details: have I actually made a PR which you were able to see the implementation? I couldn’t find that PR.
Right now, addons.cfg will trump UI-installed addons except for marketplace at boot time. Meaning any addons (except the marketplace) installed via the UI but not listed in addons.cfg will be uninstalled at boot time. It’s a matter of establishing a rule, for example: YAML > addons.cfg > UI. We can adjust the implementation and documentation in order to make this not confusing to the users.
The YAML implementation could install marketplace addons, and also JSON sources so it could do everything.
That is the opposite to what I remember, but as I said it has been a while.
It’s a shame it didn’t make it. I think it would’ve been a great addition to the yaml configuration’s ability to offer a truly packaged solution.
I don’t think so, I think it became clear during the discussion that it wouldn’t work. I also suspect that you’d met plenty of obstacles had you actually tried to implement it.
I don’t remember all the details now, it’s better to dig up where it was discussed before. Wasn’t that on GitHub somewhere?
It’s not only about “rules”, how do you actually track this? When users have copied YAML snippets here and there, or had AIs generate them, and they don’t have any overview over what’s where, how do they find that document to uninstall that particular add-on that causes problems? What happens if different YAML files conflict? Would it just be “additive”, or would you also have the ability to “deny” certain add-ons from installing? Or certain versions?
Then there’s the startup timing. Are the YAML files parsed in time, so that the system has the final answer to what bundles should and shouldn’t be installed when it needs that information? If versioning becomes a reality, you’d have further possibility for conflicts, with different YAML documents asking for different versions of the same add-on.
These are “challenges” I see immediately, but instead of trying to think through it all again, try to find where it was discussed.
There’s no natural separation between “marketplace add-ons” and other add-ons. An add-on is an add-on, it can have different sources. It can be the community marketplace, a JSON marketplace, a local JAR, a local KAR, or a Karaf feature (which downloads using Maven). “Official” add-ons comes as standard either as KARs or Karaf features, but the marketplace also has support for KAR files (although quite broken IMO). I think most of the JSON marketplace add-ons are actually KARs. So, the distinction is quite artifical.
I don’t think it would ever work in any way satisfactory if made part of the “general YAML format”. As a YAML alternative to addons.cfg, sure, but is YAML actually a preferable format/syntax?
I’m not so sure, I actually think YAML is one of the more “messy” ways you can present information. The one good thing it has going for it, is the multiline strings that let you embed whole scripts or documents inside the YAML while keeping it readable. Except for that, I’d prefer almost any other format, JSON, or even XML.
In any case, making it possible to have addons.yaml instead of addons.cfg, would be a pretty marginal “improvement”.
from memory I already had a working implementation but didn’t submit a PR due to the push backs.
Deletion was always the troubling part. Don’t forget to consider that part. I think and add-on installed that isn’t listed by any of the three sources would be sufficient, but early on the problem was addons.conf was considered the source of truth. Consequently if the add-on wasn’t listed there it was asusmed the intent was to remove it because there was no way to tell if an add-on was installed via the UI or if the add-on was originally installed via addons.conf but later removed.
I think, even if we ignore the other issues, the fact that some obscure YAML file filled with all kind of other stuff, could mean that a user couldn’t get rid of an add-on, in itself is very troubling. How do you suppose we “help” these users?
The addon could carry a metadata for its installation method (yaml - including the model / file name, ui, addons.cfg) and displayed in the UI so the user knows exactly where to look if it can’t be uninstalled from the UI. It could work like everything else (rules, things, items, etc), when it’s defined in file, it has a “lock” icon.
In any case what you described is already the case right now - whilst you can uninstall anything from the UI, anything defined in the addons.cfg will be reinstalled again upon boot - as they should be. Except the marketplace stuff which addons.cfg doesn’t touch.
The addons.cfg is a beautiful mechanism to enforce the list of addons installed on the system and I love how it works. I know I can depend on my system having the required addons as long as my addons.cfg is there.
I don’t follow at all. As far as I know, addons.cfg doesn’t work like that. It will also uninstall anything that’s not listed, if something is listed. So, you can’t use it in combination with UI installation, if you want to keep what you install.
As for the metadata, it goes back to the same thing about the broken JSONDB entry. You can’t actually “store” anything about the add-on, because it gets wiped, and it falls back to the “fallback mechanism” which is, for bindings, to check what exists in the “cache” folder, and then build the Addon instance using the marketplace, which means that the information would be lost. Likewise for e.g. official add-ons, the Addon instance isn’t stored, it’s build every time by parsing XML files in the bundles combined with some hardcoded “rules” for links to documentation etc.
So, as of today, there’s nowhere to store any additional information about add-ons, you’re restricted to the information that can be extracted from “the source of the add-on”.
This is true, but it doesn’t do it for the marketplace addons.
Say I have addons.cfg
binding=mqtt
If I then use the UI to install lgwebos (not a marketplace binding) and esphome (a marketplace binding). Upon restart, the lgwebos will be uninstalled because it’s not listed in addons.cfg, but the esphome binding will remain as is. I’ve just tested this to confirm.
In other words:
- addons.cfg will enforce the list of official addons only.
- It does not touch marketplace addons. It does not install them nor uninstall them.
Yes, and this is specific, hard-coded behavior. Are you saying that YAML should replicate this behavior? That would mean that:
No it won’t work like that. It would keep everything across all the loaded yaml.
Negative assumptions? My idea of making something, is to cover all the cases, not only the best case scenarios. So, it’s not about “negative assumptions”, it’s about finding ways to deal with the various challenges before implementation is started, so that when rolled out, it doesn’t cause a lot of problems.
Just “focusing on the positive” and ignoring every obstacle is very easy, it’s not worth much at all IMO. What “counts” is being able to also handle the obstacles. I’m frankly quite fed up with trying to “salvage” situations that arise because somebody “choose to only focus on the positive” (which means to ignore the problems). Yet, I find myself in that situation over and over again, which I guess contribute to my tendency to try to get people to take the obstacles into consideration before they make something.
I think that, should this be implemented, it will potentially cause chaos, and we should at the very least have an option that had to be turned on to enable “YAML add-on installations”. That way, when people can’t figure out why the system behaves as it does, they can just turn that option off.
Still I don’t know how you plan to handle conflicting configuration, where different YAML files ask for different versions of an add-on. Will you then just go for the most recent version? Log an error?