Upgrading openHAB: best practise to monitor previously installed Marketplace add-ons

I understand, that initially all Marketplace add-ons are not migrated, but are uninstalled for the next version. I went into this trap, as I upgraded from 4.2.1 to 4.2.2 and lost the “log4j2 extra”. In htis case, the pax-logging runs fine within both minor versions, but ok. :wink:
I could write me a list of things “to do”, with every upgrade process, but I’m tending to forget those kind of things. Is there anything to help my ADHD-brain to remember this? like writing a rule checking on marketplace-addons and sending notifications? or anything other ootb functionality for that?

if a rule would do: how would I check on a specific installed Marketplace Add-On? Is there some check already available within JS Scripting? I can check for bindings, but how do i check for this kind of bundle?

First, I think the way marketplace add-ons are handled during upgrades right now is broken. So please file an issue. It really should not just silently remove the add-ons.

In fact, I don’t think it’s supposed to. It’s supposed to clear the cache but then reinstall those add-ons which are still compatible (based on the title of the posting on the marketplace). However, I’ve seen cases where the file that lists the installed add-ons ($OH_USERDATA/jsondb/org.openhab.marketplace.org) that gets emptied out. I think that is a bug that I’ve not had time to investigate yet.

Adding something to upgrade.log might be useful. I think upgrade.log gets created for all users, not just Docker users. If not, all that contains is the output generated by apt upgrade so watching the output when you upgrade OH will show. That’s important to check becuase all the breaking changes will be listed there and other important information as well so you chould be checking that. It might be worth a warning there to double check your marketpalce and third party add-ons.

Do these add-ons have Things? You could have a rule that checks to see if these things go online X minutes after startup and if not send an alert telling you something is wrong based on that.

Another approach can be to create a rule that queries the REST API. If you hit the addons/{add-on-id} end point with “marketplace” as the service ID and the number part of the add-on ID (corresponds with the forum post number) you’ll get a JSON, one of the properties of which is whether it’s installed.

For example:

http://myohaddress:8080/rest/addons/154240?serviceId=marketplace

pulls

{
  "uid": "marketplace:154240",
  "id": "thedoctor",
  "label": "The Doctor Binding helps find issues with your system [4.0.0;4.9.9]",
  "version": "",
  "maturity": "beta",
  "compatible": true,
  "contentType": "application/vnd.openhab.bundle",
  "link": "https://community.openhab.org/t/154240",
  "author": "Skinah",
  "verifiedAuthor": false,
  "installed": true,
  "type": "binding",
  "detailedDescription": "<p>This binding is one I have been thinking of building for a while, it is designed to help diagnose issues on someones system that does not know Linux/Java and has no idea what free VS available ram means, let alone the Java heap. Install the binding and get a system check up. The system info binding requires you to know what to look at and how to interrupt the results, this binding will hopefully over time get better at finding issues in an automated way. If you have some ideas on what it can check for, then feel free to post.</p>\n<p>If you enjoy the binding, please consider sponsoring or a once off tip as a thank you via the links. This allows me to purchase software and hardware to contributing more bindings. Also some coffee to keep me coding faster never hurts <img src=\"https://community.openhab.org/images/emoji/twitter/slight_smile.png?v=12\" title=\":slight_smile:\" class=\"emoji\" alt=\":slight_smile:\" loading=\"lazy\" width=\"20\" height=\"20\"></p>\n<h3><a name=\"sponsor-skinah-on-github-sponsors-httpsgithubcomsponsorsskinah-1\" class=\"anchor\" href=\"#sponsor-skinah-on-github-sponsors-httpsgithubcomsponsorsskinah-1\"></a><a href=\"https://github.com/sponsors/Skinah/\" rel=\"noopener nofollow ugc\">Sponsor @Skinah on GitHub Sponsors </a></h3>\n<p>Paypal can also be used via<br>\n<code>matt A-T pcmus D-O-T C-O-M</code></p>\n<h2><a name=\"features-2\" class=\"anchor\" href=\"#features-2\"></a>Features</h2>\n<p>Binding already helps to warn in your logs when:</p>\n<ul>\n<li>CPU overheats</li>\n<li>Heap is wrongly sized</li>\n<li>Heap is growing and not shrinking back when garbage collections are done. OOME Out of memory errors and memory leaks should get detected and picked up early.</li>\n<li>Ram is full or getting close to 100% full to give you a warning something needs to be looked at.</li>\n<li>Detects when Raspberry Pi power supply or cable is not good enough.</li>\n<li>Allows you to graph the heap after it is first cleaned by the garbage collector.<br>\n<div class=\"lightbox-wrapper\"><a class=\"lightbox\" href=\"//community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/original/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f.png\" data-download-href=\"/uploads/short-url/8LsZumxBsDH2CC0ToYAhtNRP2Jp.png?dl=1\" title=\"heap\" rel=\"noopener nofollow ugc\"><img src=\"//community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/optimized/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f_2_690x347.png\" alt=\"heap\" data-base62-sha1=\"8LsZumxBsDH2CC0ToYAhtNRP2Jp\" width=\"690\" height=\"347\" srcset=\"//community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/optimized/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f_2_690x347.png, //community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/optimized/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f_2_1035x520.png 1.5x, //community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/optimized/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f_2_1380x694.png 2x\" data-dominant-color=\"131314\"><div class=\"meta\"><svg class=\"fa d-icon d-icon-far-image svg-icon\" aria-hidden=\"true\"><use href=\"#far-image\"></use></svg><span class=\"filename\">heap</span><span class=\"informations\">2354×1187 26.4 KB</span><svg class=\"fa d-icon d-icon-discourse-expand svg-icon\" aria-hidden=\"true\"><use href=\"#discourse-expand\"></use></svg></div></a></div></li>\n</ul>\n<p>Not yet implemented but planned to look into possible addition:</p>\n<ul>\n<li><s>Raspberry Pi power supply is not good enough</s> Added</li>\n<li>Swap file is getting used a lot or runs out of space</li>\n<li>Zram checks</li>\n<li>Watch for continually growing number of Processes and Threads</li>\n<li>Check addon jar files are all the same version as openHAB</li>\n</ul>\n<p>Example log output in DEBUG</p>\n<pre><code class=\"lang-auto\">2024-03-16 06:04:40.650 [INFO ] [.thedoctor.internal.TheDoctorHandler] - Will include health checks for your:Raspberry Pi 3 Model B Plus Rev 1.3\n\n2024-03-16 06:04:40.654 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Pi is not reporting any current throttle conditions.\n\n2024-03-16 06:04:40.708 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Heap is only 24% full, and ranges from 0% to 24%\n\n2024-03-16 06:04:40.710 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: RAM is 46% full\n\n2024-03-16 06:04:40.733 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: CPU temperature is 52.078c\n2024-03-16 06:05:56.668 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Full Pi throttle code is 80008\n\n2024-03-16 06:05:56.670 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Pi, Soft temperature limit active\n\n2024-03-16 06:05:56.671 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Your Pi's power supply or cable was not good enough to supply power without an under-voltage event occuring.\n\n2024-03-16 06:05:56.672 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Heap is only 18% full, and ranges from 18% to 28%\n\n2024-03-16 06:05:56.676 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: RAM is 67% full\n\n2024-03-16 06:05:56.679 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : CPU temperature is 61.762c and may cause instability. Do you have a heatsink and fan?\n\n2024-03-16 06:06:11.683 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Full Pi throttle code is 80000\n\n2024-03-16 06:06:11.684 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Your Pi's power supply or cable was not good enough to supply power without an under-voltage event occuring.\n</code></pre>\n<h2><a name=\"changelog-3\" class=\"anchor\" href=\"#changelog-3\"></a>Changelog</h2>\n<h3><a name=\"version-03-4\" class=\"anchor\" href=\"#version-03-4\"></a>Version 0.3</h3>\n<ul>\n<li>Fixed bug in reading the raspberry Pi throttle codes.</li>\n<li>Changes to logging to give less output.</li>\n</ul>\n<h3><a name=\"version-02-5\" class=\"anchor\" href=\"#version-02-5\"></a>Version 0.2</h3>\n<ul>\n<li>Added Raspberry Pi power supply and throttle code checks</li>\n<li>Adjusted heap detection a little based on real world testing of new pi setup with defaults.</li>\n</ul>\n<h3><a name=\"version-01-6\" class=\"anchor\" href=\"#version-01-6\"></a>Version 0.1</h3>\n<ul>\n<li>initial release</li>\n</ul>\n<h2><a name=\"resources-7\" class=\"anchor\" href=\"#resources-7\"></a>Resources</h2>\n<p><a href=\"http://pcmus.com/openhab/TheDoctorBinding/org.openhab.binding.thedoctor-4.2.0-SNAPSHOT.jar\" class=\"onebox\" target=\"_blank\" rel=\"noopener nofollow ugc\">http://pcmus.com/openhab/TheDoctorBinding/org.openhab.binding.thedoctor-4.2.0-SNAPSHOT.jar</a></p>\n<h2><a name=\"source-code-8\" class=\"anchor\" href=\"#source-code-8\"></a>Source Code</h2>\n<aside class=\"onebox githubfolder\" data-onebox-src=\"https://github.com/Skinah/openhab-addons/tree/doctor\">\n  <header class=\"source\">\n      <img src=\"https://github.githubassets.com/favicons/favicon.svg\" class=\"site-icon\" width=\"32\" height=\"32\">\n\n      <a href=\"https://github.com/Skinah/openhab-addons/tree/doctor\" target=\"_blank\" rel=\"noopener nofollow ugc\">github.com</a>\n  </header>\n\n  <article class=\"onebox-body\">\n    <h3><a href=\"https://github.com/Skinah/openhab-addons/tree/doctor\" target=\"_blank\" rel=\"noopener nofollow ugc\">GitHub - Skinah/openhab-addons at doctor</a></h3>\n\n  <p><a href=\"https://github.com/Skinah/openhab-addons/tree/doctor\" target=\"_blank\" rel=\"noopener nofollow ugc\">doctor</a></p>\n\n  <p><span class=\"label1\">Add-ons for openHAB. Contribute to Skinah/openhab-addons development by creating an account on GitHub.</span></p>\n\n  </article>\n\n  <div class=\"onebox-metadata\">\n    \n    \n  </div>\n\n  <div style=\"clear: both\"></div>\n</aside>\n",
  "configDescriptionURI": "",
  "keywords": "",
  "countries": [],
  "connection": "",
  "imageLink": "//community-openhab-org.s3.dualstack.eu-central-1.amazonaws.com/optimized/3X/3/d/3d6f05ef5208a97a425c88d587adb0ea0b585c5f_2_1024x516.png",
  "properties": {
    "jar_download_url": "http://pcmus.com/openhab/TheDoctorBinding/org.openhab.binding.thedoctor-4.2.0-SNAPSHOT.jar",
    "like_count": 22,
    "updated_at": "May 10, 2024, 3:34:14 AM",
    "auto_content": "2024-03-16 06:04:40.650 [INFO ] [.thedoctor.internal.TheDoctorHandler] - Will include health checks for your:Raspberry Pi 3 Model B Plus Rev 1.3\n\n2024-03-16 06:04:40.654 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Pi is not reporting any current throttle conditions.\n\n2024-03-16 06:04:40.708 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Heap is only 24% full, and ranges from 0% to 24%\n\n2024-03-16 06:04:40.710 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: RAM is 46% full\n\n2024-03-16 06:04:40.733 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: CPU temperature is 52.078c\n2024-03-16 06:05:56.668 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Full Pi throttle code is 80008\n\n2024-03-16 06:05:56.670 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Pi, Soft temperature limit active\n\n2024-03-16 06:05:56.671 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Your Pi's power supply or cable was not good enough to supply power without an under-voltage event occuring.\n\n2024-03-16 06:05:56.672 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: Heap is only 18% full, and ranges from 18% to 28%\n\n2024-03-16 06:05:56.676 [DEBUG] [.thedoctor.internal.TheDoctorHandler] - GOOD: RAM is 67% full\n\n2024-03-16 06:05:56.679 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : CPU temperature is 61.762c and may cause instability. Do you have a heatsink and fan?\n\n2024-03-16 06:06:11.683 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Full Pi throttle code is 80000\n\n2024-03-16 06:06:11.684 [WARN ] [.thedoctor.internal.TheDoctorHandler] - BAD : Your Pi's power supply or cable was not good enough to supply power without an under-voltage event occuring.\n",
    "created_at": "Feb 29, 2024, 11:16:26 AM",
    "views": 2070,
    "posts_count": 22,
    "tags": [
      "binding",
      "published",
      "beta"
    ]
  },
  "loggerPackages": []
}

You can get the installed status using JSONPATH $.installed.

Note the above is for The Doctor add-on.

If you’ve more than one add-on to check you can hit the addons endpoint and filter through the results to find which ones are installed and compare that to what you expect.

If you pull http://myohaddress:8080/rest/addons?serviceId=marketplace the following code shold give you the names of the add-ons that are installed compared to a list of add-ons you expect to be installed. (Note this is not tested, there will almost certainly be bugs. I’m assuming an auth token to access the API.)

var expected = [ "Ephemeris Binding", "The Doctor Binding helps find issues with your system"];
var url = 'http://myohaddress:8080/rest/addons?serviceId=marketplace';
var headers = { "accept": "application/json",
                "Authorization": "your API token",
                "WWW-Authenticate": "Basic"};
var allAddons = Json.parse(actions.HTTP.sendHttpGetRequest(url, headers, 2000)); // probably should do some error checking here
var installed = allAddons.filter(addon -> addon.installed == true).map(addon -> addon.label); // array of the labels of the installed marketplace addons
var missing = expected.filter(addon -> !installed.includes(addon));
var unexpected = installed.filter(addon -> !expected.includes(addon));
var message = "";
if(missing.length > 0) {
 message = "The following expected marketplace add-ons are not installed: " + missing.join(', ');
}
if(unexpected.length > 0) {
  if(message !== "") message += '\n';
  message +="The following unexpected add-ons from the marketplace are installed: " + unexpected.join(', ');
}
if(message != "") {
  // generate alert
}

You could also probably use executeCommandLine and the karaf console to get a list of installed bundles but despite the pain of messing with the HTTP authorization I think it’s easier to parse through the JSON.

1 Like

Thanks a lot!

from what I read here i assumed, it was done deliberately:

this works just fine! Thanks!

(just a few simple changes:)

var expected = [ "Ephemeris Binding", "The Doctor Binding helps find issues with your system"];
var url = 'http://myohaddress:8080/rest/addons?serviceId=marketplace';
var headers = { "accept": "application/json",
                "Authorization": "Bearer oh.APIToken.UjC...",
                "WWW-Authenticate": "Basic"};
var allAddons = JSON.parse(actions.HTTP.sendHttpGetRequest(url, headers, 2000)); // probably should do some error checking here
var installed = allAddons.filter(addon => addon.installed == true).map(addon => addon.label); // array of the labels of the installed marketplace addons
var missing = expected.filter(addon => !installed.includes(addon));
var unexpected = installed.filter(addon => !expected.includes(addon));
var message = "";
if(missing.length > 0) {
 message = "The following expected marketplace add-ons are not installed: " + missing.join(', ');
}
if(unexpected.length > 0) {
  if(message !== "") message += '\n';
  message +="The following unexpected add-ons from the marketplace are installed: " + unexpected.join(', ');
}
if(message != "") {
  // generate alert
}
console.info("INFO check: ", message);
1 Like

If you look further down on that thread you’ll see the following though:

Jan says compatible add-ons should be reinstalled and even back then I identified this behavior as a potential bug.

At a high level, the marketplace add-on jar files definitely do need to be removed during an upgrade same as official add-on jar files. However, if the versioning of the add-on’s marketplace title indicates it’s still compatible the add-on jar should be redownloaded and installed.

The bug I’m seeing is that the JSONDB files that is supposed to list all the marketplace add-ons that are/should be installed gets emptied out somehow so after clearing the cache it doesn’t see any marketplace add-ons to reinstall.

1 Like

Would it make sense for the actual upgrade process to warn users about incompatible marketplace addons prior to any upgrade? It might avoid a number of issues for users by having them hold off a week or two and checking back for when updates have been made (or going ahead with the upgrade but being aware that they may lose some functionality temporarily).

1 Like