[SOLVED] Install marketplace addon via REST API returns 200 but nothing is installed

Using the REST API to install marketplace add-ons gives return code 200, which suggests all went well, but nothing is installed.
This is the relevant part of Ruby script.

rule "InstallMarketBindings" do
  description "Scan for market addons and install them at startup"
  on_start at_level: 80
  run do
[...]
    after(60.seconds) do
      # Loop through each addon
      bindings.split(",").each do |bindingEntry|
        newBinding = bindingEntry.split(":")[1].strip
        bindingName = bindingEntry.split(":")[0].strip
        logger.info "Install #{bindingName} (#{newBinding}) from marketplace"
        uri = URI.parse("http://openhab/rest/addons/#{newBinding}/install")
        header = {'Content-Type': 'text/plain', 'Accept': '*/*', 'X-OPENHAB-TOKEN': '***REDACTED***'}
        # Create the HTTP objects
        http = Net::HTTP.new(uri.host, uri.port)
        request = Net::HTTP::Post.new(uri.request_uri, header)
        request.body = "id=#{newBinding}"
        # Send the request
        response = http.request(request)
        logger.warn "Error #{response.code} installing marketplace addon #{bindingName} (#{newBinding})" unless response.code == "200"

And this is what gets logged:

[INFO ] [ort.loader.AbstractScriptFileWatcher] - (Re-)Loading script '/openhab/conf/automation/ruby/install_bindings.rb'
[INFO ] [scripting.rule.install_bindings.rb:4] - Install Market bindings defined in addons.cfg upon system start (20s delayed)
[INFO ] [scripting.rule.install_bindings.rb:4] - Install binding marketplace:107692 from marketplace
[INFO ] [scripting.rule.install_bindings.rb:4] - Install binding marketplace:144790 from marketplace
[INFO ] [scripting.rule.install_bindings.rb:4] - Install binding marketplace:149855 from marketplace
[INFO ] [scripting.rule.install_bindings.rb:4] - Install binding marketplace:149990 from marketplace

Any idea what Iā€™m doing wrong?

UPDATE:

  1. Tested different formats for the binding id: just the number, with ā€˜addon-ā€™ prefix, with ā€˜binding-prefixā€™, with ā€˜marketplace:ā€™ prefix - always the same result.
  2. Tried with an OH add-on, and it gets installed just fine. Does this mean that the REST API - despite returning 200 - cannot install marketplace add-ons?

Is there anything else apart of code 200 returned in the body of the result in case the addon is installed ?
I just did a trial in the API explorer by using abcdefgh12345 as addonid and it returned code 200 with 0 length for the body message.

1 Like

Note, the Marketplace category is only for posting add-ons, rule templates, ui widgets or blockly libraries. All discussions need to take place in other categories. The forum posts are how these are all actually published and made available to OH to install. Iā€™ve moved this post.

If you are using addons.cfg, anything you do through the REST API is going to become undone. Itā€™s still one or the other, not both, and addons.cfg always takes precedence.

Beyond that, I think you might need to either look at the code, look at the developer tools in the browser to see the URLs used when installing one, or set up postman or something to capture the web requests between MainUI and the server to see what the proper ID is.

Assuming it uses the same end points as regular add-ons, you can use the GET endpoint to experiment with until you get the ID right.

1 Like

Thanks for the suggestions.

Sorry about that. Iā€™ll pay more attention next time.

I am aware of that and I use addons.cfg for ā€˜OpenHAB nativeā€™ addons, but to my knowledge there is no way to specify marketplace adding in adding.cfg.

Thatā€™s what Iā€™m going to do and figure out how it is actually retrieved/installed.

Noticed that too after more testing. I am going to dig deeper and get back on this.

Spending just a little bit of time with the browser dev tools solved the issue as suggested by @rlkoshak.
Turns out the URI should be https://openhab/rest/addons/marketplace:{addonId}/install?serviceId=marketplace. The query parameter serviceId was needed.

The script now looks like this, for anyone interested:

rule "InstallMarketAddons" do
  description "Install marketplace addons at system start"
  on_start at_level: 80
  run do
    logger.info "Install Marketplace addons defined in addons.cfg upon system start (w/15s delayed)"
    # Extract the addons line from `conf/services/addons.cfg`
    #TODO: Add errorhandling to ini file I/O
    cfgLines = IO.readlines('{{ openhab_conf_dir }}/services/addons.cfg')
    addons = cfgLines[cfgLines.index{ |line| line =~ /market_binding/ }].split('=')[1].strip
    #TODO: Add check for non-existing 'market_binding' line
    after(15.seconds) do
      # Loop through each addon
      addons.split(",").each do |addonEntry|
        addonName, addonId = addonEntry.split(":")
        #TODO: Check for ill-formatted entries
        logger.info "Install addon #{addonName} (#{addonId}) from marketplace"
        uri = URI.parse("http://openhab4-ui/rest/addons/marketplace:#{addonId}/install?serviceId=marketplace")
        header = {'Content-Type': 'text/plain', 'Accept': '*/*', 'X-OPENHAB-TOKEN': '{{ openhab_api_token }}'}
        # Create the HTTP objects
        http = Net::HTTP.new(uri.host, uri.port)
        request = Net::HTTP::Post.new(uri.request_uri, header)
        request.body = "id=#{addonId}"
        # Send the request
        response = http.request(request)
        # Looks like the API always returns 200 with an empty body, so no use logging it
        #TODO: Add check if addon is actually installed
        logger.debug "Result #{response.code} (#{response.body}) - installing marketplace addon #{addonName} (#{addonId})"
      end
    end
  end
end

Note that I ā€˜abuseā€™ addons.cfg where I add a line like this:

market_binding = Unifi Protect Binding:107692,Magic Binding:144790,Philips Air Binding:149855

This entry is ignored by OH core (at least so farā€¦).

The fact that the API always returns 200 with an empty body could be considered a bug IMHO. Perhaps I should file an issue.

1 Like

I would expect the line to start with ā€œmarketplaceā€ instead of ā€œmarket_bindingā€. I agree about the 200 return code and that deserves an issue on core.

1 Like

I agree, and changed it.
Also added handling the case when thereā€™s no marketplace entry. The script for anyone interested:

rule "InstallMarketAddons" do
  description "Install marketplace addons at system start"
  on_start at_level: 80
  delay 15.seconds  # Let OH core do its magic first
  run do
    # Extract the marketplace addons line from `conf/services/addons.cfg`
    cfgLines = IO.readlines('{{ openhab_conf_dir }}/services/addons.cfg')
    if lineIndex = cfgLines.index{ |line| line =~ /marketplace/ } then
      logger.info "Install Marketplace addons defined in addons.cfg at system start"
      addons = cfgLines[lineIndex].split('=')[1].strip
      # Loop through each addon
      addons.split(",").each do |addon|
        addonName, addonId = addon.split(":")
        #TODO: Check for ill-formatted entries
        logger.info "Install addon #{addonName} (#{addonId}) from marketplace"
        # Create the HTTP object
        uri = URI.parse("{{ openhab_api_url }}/addons/marketplace:#{addonId}/install?serviceId=marketplace")
        header = {'Content-Type': 'text/plain', 'Accept': '*/*', 'X-OPENHAB-TOKEN': '{{ openhab_api_token }}'}
        http = Net::HTTP.new(uri.host, uri.port)
        request = Net::HTTP::Post.new(uri.request_uri, header)
        request.body = "id=#{addonId}"
        # Send the request
        response = http.request(request)
        # Note: Looks like the API always returns 200 with an empty body, so no need to log it
        logger.debug "Result #{response.code} (#{response.body}) - installing marketplace addon #{addonName} (#{addonId})"
        #TODO: Add check if addon is installed
      end # of loop through each entry
    end # if
  end
end

Reported issue #3987.

UPDATE:
Just to confirm: this works for all types of add-ons (bindings, UI, etc.)

I expanded the script to also install Json 3rd party add-ons.

CFG_FILE = '{{ openhab_conf_dir }}/services/addons.cfg'

iniLines = nil


def readIni(key, iniFile=CFG_FILE)
  logger.debug "Reading addons.cfg and return \"#{key}\" items"
  # Extract the requested entry line from `conf/services/addons.cfg`
  iniLines = IO.readlines(iniFile) if iniLines.nil?
  if lineIndex = iniLines.index{ |line| line =~ /^#{key}/ } then
    return iniLines[lineIndex].split('=')[1].strip
  end
  return nil
end


def httpAction(httpRequest, httpUrl, httpBody=nil)
  logger.debug "Sending HTTP #{httpRequest} to #{httpUrl} with body \"#{httpBody}\""
  # Create the HTTP object
  uri = URI.parse(httpUrl)
  header = {'Content-Type': 'application/json', 'Accept': '*/*', 'X-OPENHAB-TOKEN': '{{ openhab_api_token }}'}
  http = Net::HTTP.new(uri.host, uri.port)
  request = httpRequest.downcase == "put" ? Net::HTTP::Put.new(uri.request_uri, header) : Net::HTTP::Post.new(uri.request_uri, header)
  request.body = httpBody unless httpBody.nil?
  response = http.request(request)
  logger.warn "Result #{response.code} (#{response.body}) - installing service/add-on @ #{httpUrl} failed" unless response.code == "200"
  return true ? response.code == "200" : false
end


rule "InstallAddons" do
  description "Install marketplace add-ons at system start"
  on_start at_level: 40

  # Step 1: Install any Json 3rd party add-on services specified
  run do
    logger.info "Install Json 3rd party add-on services (defined in addons.cfg) at system start"
    services = readIni("addonservice")
    unless services.nil?
      services.split("|").each do |service|
        serviceName, jsonUrl = service.split(",")
        serviceName = serviceName.strip
        jsonUrl = jsonUrl.strip
        #TODO: Check for ill-formatted entries
        logger.info "Install JSON 3rd party add-on service \"#{serviceName}\"@\"#{jsonUrl}\""
        result = httpAction("Put", "{{ openhab_api_url }}/services/org.openhab.jsonaddonservice/config", "{ \"urls\": \"#{jsonUrl}\" }")
      end
    end
  end
  #TODO: Check if services are installed (async) before we continue
  delay 10.seconds  # For now: let OH do its magic

  # Step 2: Install the defined add-ons
  run do
    # Extract the marketplace add-ons entry from the ini file
    addons = readIni("marketplace")
    logger.info "Install Marketplace add-ons (defined in addons.cfg) at system start"
    unless addons.nil?
      addons.split("|").each do |addon|
        addonName, addonType, addonId = addon.split(",")
        addonName = addonName.strip
        addonType = addonType.strip.downcase
        addonId = addonId.strip
        #TODO: Check for ill-formatted entries
        logger.info "Install addon \"#{addonName}\" from #{addonType == "marketplace" ? "Community Marketplace" : "Other Add-ons"}"
        result = httpAction("POST", "{{ openhab_api_url }}/addons/#{addonType}:#{addonId}/install?serviceId=#{addonType}")
        # Note: The API always returns 200 with an empty body (see PR )
        #TODO: Add check if addon is actually installed (async)
      end
    end
  end

end

There are basically 3 types of add-ons: OpenHAB provided add-ons, marketplace add-ons, and Json 3rd party add-ons (shown under ā€˜Other Add-onsā€™ in the Add-on Store).

  • The OpenHAB Distribution add-ons can of course be installed in the usual way (either specified in addons.cfg or in the Add-on Store).
  • The Community Marketplace add-ons can be installed through the Add-on Store or REST API. For this I added functionality in the script to install it from addons.cfg as well.
  • The Other Add-ons are in the Add-on Store referencing Json 3rd Party Add-on services as shown in the System Settings. For this I also added functionality in the script to install them via addons.cfg.

There are two new entries for addons.cfg: one to define any Json 3rd party add-on services (as sources for other add-ons) and one to define the non-distribution add-ons to install at system start.
The first entry looks like this:

addonservice = SmartHome/J Add-ons, https://download.smarthomej.org/addons.json

You can add more services, separated by ā€˜|ā€™.

The actual add-ons to install are specified with:

marketplace = SemanticHomeMenu1, marketplace, 152205 | Unifi Protect Binding, marketplace, 107692 | Tuya SmartHome Binding, json, org-smarthome-binding-tuya

The marketplace and json elements specify if it is a Community Marketplace add-on or Json 3rd party add-on. The number specifies the ID as the add-on is known in the Marketplace. org-smarthome-binding-tuya is how the 3rd party add-on is referenced in the Json add-on store.
You can find these specifications by running the browser with DevTools (F12) and see what is send when installing from the Add-on Store.

Fast forwarding to openHAB 4.1.1, is there an out-of-the-box way to install a Smarthome/J addon using only .cfg files?

I have this line in runtime.cfg:

org.openhab.jsonaddonservice:urls=https://download.smarthomej.org/addons.json

This causes the addons from the Smarthome/J collection to appear in the MainUI. I have this line in addons.cfg:

binding = astro,<...other core bindings...>,zwave,org-smarthome-binding-tuya

but none of org-smarthome-binding-tuya, tuya, json:org-smarthome-binding-tuya etc. are the correct incantation and all result in the log entry:

2024-xx-xx xx:xx:xx.xxx [WARN ] [core.karaf.internal.FeatureInstaller] - The binding add-on 'various attempts as above' does not exist - ignoring it.

I must continue to have a fully text-based openHAB config, as many do, so Iā€™m hoping there is a way now or soon.

Thanks!

As far as I can tell, marketplace add-ons are only supported through the JSONDB. Add-ons installed from either the OH Marketplace or Smarthome/J do not appear in addons.cfg. You can definitely install it through the karaf console or the REST API but I donā€™t see a way to install add-ons from files in the $OH_CONF folders.

Thanks for the reply. Does a binding installed through the karaf console or the REST API get uninstalled at restart if addons.cfg is used? Previous information says clearly that addons.cfg is an all-or-nothing proposition.

Does anyone have any thoughts on whether there is a desire to keep text-file-only users in red-haired-stepchild status in terms of features and documentation, or is it only a resource issue? Hopefully a pull request would be accepted that would re-elevate text-file-only users to first-class citizens again, but there would be no reason to create one if it can be known ahead of time that it would not be accepted.

I donā€™t think so. Itā€™s all or nothing for the ā€œofficialā€ add-ons for sure because what happens is OH will copy the contents of $OH_CONF/services/addons.cfg to $OH_USERDATA/config/org/openhab/addons.conf. (or something like that, Iā€™m going from memory).

When you install an ā€œofficialā€ add-on through the karaf console it modifies the addons.config file in userdata. Then if you also have stuff in the addons.cfg file in conf, the next time that OH syncs those two files anything not in addons.cfg gets lost.

But as far as I can tell, the marketplace add-ons do not appear anywhere in the $OH_USERDATA/config folder. I can only find reference to these add-ons in $OH_USERDATA/jsondb. Therefore the sync process described above for addons.cfg wouldnā€™t apply.

There was some progress made to develop a unified YAML based text file config (as ugly as it is, YAML is currently the best compromise between human readability and the needed expressiveness) to harmonize the way configs look in the MainUI code tab and what one would put in text file configs to make moving between the two approaches much easier (e.g. one could copy the YAML they see in MainUI into a text file and they are good to go) and to make it easier on the developers to continue to maintain and extend the text configs (Xtext is challenging to work with and there are few maintainers left who are able and willing to touch it).

Unfortunately there was too much discussion and argument and the effort was abandoned by the developer, much to my dismay.

I would definitely love to see that work picked up again. Iā€™m not sure it would be significantly more challenging than creating a new file just for the marketplace add-ons. And it would definitely make it much easier to keep the feature set supported between managed configs and unmanaged configs in sync.

But even without going that far, I see no reason why a pull request for this wouldnā€™t be accepted, but expect lots of opinions on format and overall approach. All I can recommend is the only opinions that really matter are the maintainerā€™s who have to approve the PR.

In general, there are not many developers who are outright hostile to text based configs. There just are not many who are willing to actually work on them. And while people mean well, the lengthy discussions and arguments on github do little except make sure that no one will want to volunteer to work on them.

1 Like

Thank you very much for your considered reply and the generosity of your time in giving it.