Marketplace versioning with embedded resource

As you know, I can’t make PRs - but we could do a “collaboration” where you are the author :wink:

That works for me :smile:

Actually, it would require some substantial changes, since as far as I know, the Marketplace has no concept of version, and it would require regular checks - which would cause a much larger “load” on the forum. So, apart from establishing some convention for how to “tag” what version an add-on is, a caching solution of some sort might be required. I don’t know if the forum has a vast capacity of course, I’m only assuming that it wouldn’t be ideal to do it directly.

That said, the check would only be required for installed add-ons, and if done infrequently enough it might not be that big a problem…?

It would also require changes both in Core and MainUI, and MainUI scares me because while I’m comfortable with Java, the story is very different with JavaScript, Vue etc.

I doubt we should go that route.
In the past we had a lot of trouble with discourse leading to a lot of downtimes.
The foundation then upgraded the server massively which also increased the costs a lot.
From my perspective, a solution outside discourse would be best.

1 Like

The marketplaec API does support a non-Discourse type marketplace. See Smarthome/J as an example. I’m not sure exactly how that works but I know Discourse is not involved at all.

So maybe it makes sense to explore creating something closer to a “real” marketplace rather than remaining limited to Discourse. Those who just want to quickly throw up something can still use Discourse. Others who have a more involved something to post might benefit from something more dedicated to doing just that job.

Obvuiously this is much more involved than adjusting what we have now, but this is OH 5 so let’s think big! :wink: a

1 Like

I know and this will be a good start to look at.
But this still does not have an update feature so needs to be extended…

Right, but it’s bound to be more flexible than using the forum in the long run. I just wanted to point out we are not necessarily stuck using the forum forever.

1 Like

I’ll just add that making Core fetch the add-ons from a different source doesn’t look very difficult. I think the majority of the work is to establish “the other thing” - I mean, “build” the repository, figure out where it should run, how to add/edit add-ons etc.

It must run on some server somewhere, and if you’re not going to use the forum, a whole UI might need to be created for people to add/modify stuff. That was why I was thinking of just “caching” the information from the forum, so that there was no need to “build a new UI” for doing that. There’s also the complication with user accounts/edit permissions etc., that is already handled by the forum.

The update feature isn’t necessarily so difficult, depending on how one wants it to work. I was thinking loosely about it earlier today, and I imagined maybe just adding another API endpoint that “checks for updates” and returns a list of add-ons that can be updated. They could then be updated by using the existing uninstall and install endpoints, or by making another one that does both in one operation. With this design, it would be up to the UI (or anything else for that matter) to decide how often to check, and the list of “upgradable” add-ons returned should make it relatively easy for the UI to give the user a notification, offer to upgrade selected add-ons etc.

But, all this requires that the marketplace repository, whatever form it might have, have version info. If you limit the “version format” to the format already used by OSGI (bundle version) and that is used by OH, there are already ready-made methods for parsing and comparing versions.

Don‘t get me wrong, my comment was not speaking against it. I like the idea, just wanted to highlight it will not be quick to implement.

My feature suggestions/wishes are:

  • Notify the user about updated addons in the MainUI:

    • could be like this (dont mind the ugly graphics - just demonstrating concepts):
      CleanShot 2025-01-06 at 18.37.03

      and this:
      CleanShot 2025-01-06 at 18.39.08
    • In the respective list of installed components (bindings, automations, UI etc), show components with updates at the top as well so the user easily can see availalbe updates.
  • Support for multiple versions so the user can choose an older version if the latest and greatest doesn’t work (ie a beta version). Could also be some sort of “tracks” like “stable” and “latest”.

  • Support for multiple OH version builds, ie OH5, OH 4.3.x, OH 4.2.x etc, without the developer having to maintain multiple versions of the same addon (today: multiple Discourse posts for each version range)

  • Some sort of download statistics so the dev (and everyone) can see which addons that have a user base, and which doesn’t.

  • Link to the issue tracker, documentation and codebase for each addon, make it easy for the end user to report issues in the right place, not just as message no 503 in a crazy long forum thread.

  • Support for all kinds of marketplace components, not just (jar) bundles

That was quite the list :wink:

I’m not UI expert, but I was thinking that the natural place to show this in the UI would be the same notification that’s used for discovered devices, orphaned links etc. But that might be too itrusive? Anyway, without a implementable plan for how to host/store the Marketplace data, the rest of the details are a bit “premature” IMO.

This is outside the scope of the marketplace. Not even “official” add-ons can meet this requirement. I could see ways for the developer to have just the one post if there is a way to tag the version range for each resource in a given post. But the developer will still need to build and maintain OH version specific jar files. To change that will require a very significant effort in OH core to achieve and it very well could be impossible.

The marketplace already does support more than just jar bundles. Do you mean all your wish list to apply to all four categories supported by the marketpalce? If the latter I would assume any changes would support widgets, rule templates, and block libraries too. If the former I’m confused.

He must mean that one can host multiple JARS for different versions in the same “Marketplace entry”. It’s not possible to use the same JAR for different versions of OH when there are breaking changes.

But that was not clear from his post. I understood it the same way than @rlkoshak.

Some other wishes like link to documentation, issue tracker etc. are already possible within the marketplace post. It is just not used.

I’m sorry if it wasn’t clear, but @Nadahar understood my intentions correctly.
Point is “1 addon entry with multiple version ranges support - via possibly multiple artifacts” Maybe such support could also help avoid the unnecessary marketplace addon uninstallation on OH upgrades - given that there is an entry with a version range covering the new OH version.

I was merely suggesting that similar functionality be available in a new version, not saying it doesn’t exist in the current Discourse based Marketplace.

Yes if possible. I’m not personally using some of the component types, and thus have little knowledge about any limitations. Maybe you are aware of some?

All true pain points I’ve felt myself, hoping that someone will alleviate :wink:

Each type has a slightly different template to parse but what works for bundles overall should work for the rest too.

I just had a look at the discourse forum and stumbled across this

This might be a way to get information on updated marketplace widgets/bindings etc…
Don‘t know if this can be implemented in MainUI …

I think the update check should be done by Core, the existing logic is already there, and it’s quite easy to do there. It will also mean that the update check can be done from the REST API, and is available to “any UI”.

I’m not sure exactly how you would use information about “recently edited” posts in any useful way. I mean, what would “recently” be and how would that be “relevant” for the individual OH installation. The criteria for “update available” should be if the add-on is more recent than the currently installed version, not when it was updated?

It also seems to me that the plugin they discuss is for “admin” use only. That means that every OH installation would need an “admin API key” for the forum, which probably isn’t a very good idea :wink:

But, I don’t think looking for recently updated posts is required. To implement upgrade functionality an add-on versioning system must be established. When the versioning system is in place, it’s easy to compare versions and find which plugins can be updated regardless of when the post was last edited.

Or am I missing something?

I would store the edit date of the post on install. Then you can periodically check to see if the first post has been edited since it was installed.

Since you have to edit the first post to post an update to the entry and one is unlikely to edit that first post except when there is an update, it could be used as a proxy for when there is an update. It won’t be perfect, especially at the beginning when the developer is building the docs, but it’s better than nothing.

That would indeed kill the idea I think.

1 Like