Metadata unreadable through MetadataRegistry following restart

Today, following a rare memory issue (several OutOfMemoryErrors in the log), I did the typical stop/clean-cache/start/restart cycle to recover. I bet it has been over a year since I had a memory issue, so quite rare. That’s not the topic I’m posting about.

I’m posting because the JavaScript rule that reads metadata (code below) was coming back undefined. The metadata was right there in REST API, no apparent problem. Not knowing what else to try, I copied (ctrl-C copied) the metadata, deleted it, then recreated it all through REST API. Low and behold, my rules read the metadata just as before the memory error.

Thoughts about a better way to recover than deleting/recreating the metadata?

FWIW: I’m one of those who have to reinstall the NGRE binding every time I restart openHAB. Sure hope that is fixed when 3.0 rolls out!

Thanks.

  • Platform information:
    • Hardware: amd64/16G RAM/500G Storage
    • OS: Debian GNU/Linux 10 (buster), Linux 4.19.0-6-amd64
    • Java Runtime Environment: Zulu11.43+55-CA (build 11.0.9.1+1-LTS)
    • openHAB version: openHAB 2.5.9-1
'use strict';
var OPENHAB_CONF = Java.type("java.lang.System").getenv("OPENHAB_CONF");
load(OPENHAB_CONF + '/automation/lib/javascript/core/osgi.js');
load(OPENHAB_CONF + '/automation/lib/javascript/core/rules.js');
load(OPENHAB_CONF + '/automation/lib/javascript/core/actions.js');
load(OPENHAB_CONF + '/automation/lib/javascript/core/utils.js');

var MetadataRegistry = get_service("org.eclipse.smarthome.core.items.MetadataRegistry");
var Metadata = Java.type("org.eclipse.smarthome.core.items.Metadata");
var MetadataKey = Java.type("org.eclipse.smarthome.core.items.MetadataKey");

JSRule({
    name: "TOD_Lighting",
    description: "Rule to update lighting based on TOD",
    triggers: [
        ItemStateChangeTrigger("vJSTimeOfDay")
    ],
    execute: function(module, input){
        // 
        var curTOD = getItem("vJSTimeOfDay").state;
        //  - cycle through gDimmers group and commands metadata for each TOD state
        ir.getItem("gDimmers").members.forEach(function(dimmer) {
            var testReadMetadata = MetadataRegistry.get(new MetadataKey("TOD",dimmer.name));
            var metaToSend = testReadMetadata.configuration[curTOD];
        //  - skip if no metadata for the TOD
        if (metaToSend==undefined) {
                logInfo("JS Lighting: no metadata at " + curTOD + " for " + dimmer.name)
            }
            else {
                logInfo("JS Lighting: sending metadata " + metaToSend + " to dimmer " + dimmer.name);
                events.sendCommand(dimmer, metaToSend);    
            }
        });
    }
});

One of those… as in there are others? I’m not aware of this issue being reported before. Do you use addons.cfg? Do other add-ons also disappear after a restart?

Maybe typical for you, but having to clear the cache on a system that has not changed sounds more like faulty hardware than software. A restart should be enough to clear up OOM errors.

Is it not working consistently, or does it work sometimes? Was the ItemRegistry accessible at this time? After clearing the cache, do you restart OH at least twice? When this issue occurs, does restarting OH help?

I suggest writing a script to populate/modify the metadata for your Items.

You’re using the other helper libraries :+1:… are you not aware of metadata.js? If so, why have you chosen to not use it?

Thanks for the quick reply!

One of those… as in there are others? I’m not aware of this issue being reported before. Do you use addons.cfg? Do other add-ons also disappear after a restart?

Searching through the community I found at least one other mention of having to reinstall NGRE on restart. I presume there are others. The only uncommented line in my addons.cfg is for the openhabcloud service. NGRE is the only add-on I have trouble with.

Maybe typical for you, but having to clear the cache on a system that has not changed sounds more like faulty hardware than software. A restart should be enough to clear up OOM errors.

Fair enough. I can be a belt-suspenders personality, and it seems like fairly common advice on these pages. But I’ll scale it back.

Is it not working consistently, or does it work sometimes? Was the ItemRegistry accessible at this time? After clearing the cache, do you restart OH at least twice? When this issue occurs, does restarting OH help?

The rule worked reliably before this one incident. No errors with the ItemRegistry after the restart. Other rules that operate on the item registry without using metadata were OK. I only restarted once after the start. Rather than stop/clean/start/restart, I can be a restart/restart person. I can’t see how to recreate this issue, but if it happens again I’ll see if that is sufficient.

I suggest writing a script to populate/modify the metadata for your Items.

Sensible suggestion. Put it on my list of things to do. I’m an amateur – this rule is only the second piece of JavaScript I’ve written (better to say cribbed from these pages). It’s a bit of an investment, but step by step, right? Maybe I can get my daughter to write one for me :wink:

Really appreciate you and the other pros here who are tirelessly helpful to us dabblers.

That would do it! Once you use addons.cfg for a certain type of add-on, you have to use it for all any other add-on of that type. openhabcloud is in the misc category, so all other add-ons in the misc category need to be installed using addons.cfg, or they will not be reinstalled after an OH restart. The new rule engine is also in the misc category, so it cannot be installed through Paper UI, or it will be uninstalled after a restart, because the addons.cfg says just openhabcloud is to be installed. I install everything through the addons.cfg and here is that line from my addons,cfg…

# A comma-separated list of miscellaneous services to install (e.g. "myopenhab")
misc = ruleengine, openhabcloud

There are warnings about this in the documentation, too.

The topic you linked to was a very different issue, unless you are also seeing those exceptions. If you are, then there is a problem in the jsondb file that holds your JSON/UI created rules. If you don’t have any of those, then it is definitely not something you are experiencing.

After clearing the cache, many people have reported OH not starting up properly until restarting at least one additional time. Some report needing multiple restarts. One of the issues is that Items are not loaded properly, which is why I asked. The ItemRegistry is separate from the MetadataRegistry, but they are loosely linked. You issue seemed much more hardware based, IMO. If you do not see any issues in the first restart after clearing the cache, then you are one of the lucky ones.

Here is a snippet of what I use. It is in Jython, but other than the imports, the same can be used in JS using the metadata.js.

from core.metadata import set_metadata#, remove_metadata

set_metadata("gOutside_Door_Trigger", "area_triggers_and_actions", {
    "notification_action": {
        "limited": True,
        "active": {
            "delay": 3600,
            "types": {
                "audio": True,
                "mobile": True
            },
            "Priority": 0,
            "recurring": True
        }
    }
}, overwrite=True)

set_metadata("US_GarageAttached_Dimmer", "area_triggers_and_actions", {
    "light_action": {
        "inactive": {
            "delay": 180
        }
    }
}, overwrite=True)

set_metadata("DS_MasterBathroom_Speaker_Player", "area_triggers_and_actions", {
    "speaker_action": {
        "inactive": {
            "delay": 30
        }
    }
}, overwrite=True)

set_metadata("DS_FamilyRoom_TV_LED_Color", "area_triggers_and_actions", {
    "light_action": {
        "lux_item_name": "ESP12E_01_Luminance",
        "active": {
            "modes": {
                "Morning": {"lux_trigger": 55, "low_lux": {"brightness": 10, "hue": 100, "saturation":100}},
                "Day": {"lux_trigger": 250, "low_lux": {"brightness": 0, "hue": 0, "saturation":0}},
                "Evening": {"lux_trigger": 250, "low_lux": {"brightness": 10, "hue": 255, "saturation":100}},
                "Night": {"lux_trigger": 250, "low_lux": {"brightness": 10, "hue": 240, "saturation":100}},
                "Late": {"lux_trigger": 55, "low_lux": {"brightness": 10, "hue": 0, "saturation":100}},
                "Party": {"lux_trigger": 0, "low_lux": {"brightness": 10, "hue": 0, "saturation":100}}
            }
        }
    }
}, overwrite=True)

set_metadata("DS_FamilyRoom_TV_Bulb", "area_triggers_and_actions", {
    "light_action": {
        "lux_item_name": "ESP12E_01_Luminance",
        "active": {
            "modes": {
                "Morning": {"lux_trigger": 55, "low_lux": {"brightness":10}},
                "Day": {"lux_trigger": 250, "low_lux": {"brightness":98}},
                "Evening": {"lux_trigger": 250, "low_lux": {"brightness":98}},
                "Night": {"lux_trigger": 250, "low_lux": {"brightness":20}},
                "Late": {"lux_trigger": 55, "low_lux": {"brightness":1}},
                "Party": {"lux_trigger": 0, "low_lux": {"brightness":55}}
            }
        }
    }
}, overwrite=True)

:grinning_face_with_smiling_eyes: For whatever small part I’ve played, you are very welcome! I enjoy helping!

The new rule engine is also in the misc category, so it cannot be installed through Paper UI, or it will be uninstalled after a restart, because the addons.cfg says just openhabcloud is to be installed.

That’s it! I have a shred of memory about that warning, and took it mainly as a housekeeping matter. I would not have come up with this as the solution. I uninstalled the rule engine in PaperUI, added it to addons, and it loaded like a one would expect. Two restarts, as the doctor prescribes.

Thanks, too, for the script snippet. I’ll cobble that together for modify/restore purposes along with the helper library. If openhab crashes on me again I’ll see if the metadata issue comes back and if two restarts fixes it. Gratefully that is very rare.

1 Like

Just to clarify… two restarts after clearing the cache. The cache is cleared when using the OH update script.

Another thing you can do is when something like this happens for anything stored in JSONDB. openHAB makes a backup every time there is a change to the JSONDB files automatically. And sometimes if OH crashes (e.g. that out of memory error out mentioned) while writing out to the JSONDB file that file will not be completely written. So it’s possible your metadata JSONDB file was corrupted during the crash.

So you can compare the “live” version of the file in /var/lib/openhab2/jsond with the most recent backup in /var/lib/openhab2/jsondb/backups and if the differences indicate corruption, restore the backup.

Also a great tip. Seeing the file structure, that should be straightforward. If it happens again, maybe studying the backups can shed light on why the metadata was available in REST API but not readable through MetadataRegistry.

Let me add in thanks for all your tutorials and design pattern work. I’ve been reading through your 3.0 tutorials to get ready for that upgrade over the holiday break.

1 Like