openHAB 4.0 SNAPSHOT discussion

[rlkoshak (Rich Koshak)]

@J-N-K

Please shut openHAB down, delete the jsondb-files related to the community marketplace and the json 3rd party addon service (org.openhab.marketplace.json and org.openhab.jsonaddonservice.json) and start again.

Interestingly I didn’t have an org.openhab.jsonaddonservice.json file but that’s probably because I’ve not installed anything except from the OH marketplace even though I’ve added Smarthome/J.

After starting OH back up the errors occurred again. Looking back at JOSNDB and I think there’s another file I need to delete, marketplace.ruletemplates.json. After doing that still the errors persist. I’ve some UI Widgets installed from the marketplace too. Perhaps I need to delete the ui_components files? Same behavior after deleting the uicomponents files too.

In case it’s relevant, I have show unpublished enabled as well.

The REST API call still returns 404 not found.

Does the path $.countries point to any clues? I’ve done a search through my remaining JSONDB files and it’s only appearing in backups of the org.openhab.marketplace.json file.

[J-N-K (Jan N. Klug)]

Yes, it’s a clear indicator that the marketplace storage is the issue, the exception comes from the place where the storage is read (AbstractRemoteAddonService:103). I don’t understand why that results in a 404, but that’s another issue.

Maybe if you delete the file it is restored from backups?

[rlkoshak (Rich Koshak)]

@J-N-K

Maybe if you delete the file it is restored from backups?

That’s a thought.

First I tried to restart OH with the file gone just to see if the file gets recreated. If it was restored I would expect the file to return. But the file does not return and the error persists.

Next I deleted (note, I’m really just moving them so if you need to see thier contents I still have them) the backups of the org.openhab.marketpalce.json and marketplace_ruletemplates.json files from the backup folders. The word “countries” no longer appears in any file in the jsondb folder.

That works! Or at least that gets me to @hmerk’s state. I now see that Unable to retrieve marketplace add-ons: type must not be empty warning and the REST API returns an empty array. But that’s progress!

That backup behavior is weird and unexpected. It seems reasonable to use the latest backup when the file disappears I guess, but then the next time OH dumps the JSONDB to file (e.g. at a restart) I would expect it to recreate the “real” file instead of continuing to use the backup file. I’ve no doubt it would create a new file if a change is made, but if it’s going to use the backup I would expect it to restore the backup, not just continue to run off of the backed up file.

Is this expectation unreasonable or do you think I should file an issue?

[hmerk (Hans-Jörg Merk)]

@rlkoshak

I now see that Unable to retrieve marketplace add-ons: type must not be empty warning and the REST API returns an empty array. But that’s progress!

This one is solved by removing „show unpublished Addons“
I could not check all marketplace posts, but definitely found some addons without any tags, which seems to be causing the issue.

[rlkoshak (Rich Koshak)]

@hmerk

This one is solved by removing „show unpublished Addons“

Yep. Unfortunately I was hoping to test my new Debounce rule template before I “published” it but that’s no biggie.

Now if we can get a fix for that ZigBee problem I’ll be back up to 100% on the 4.0 snapshots.

[crnjan]

Added a (potential) fix for Unable to retrieve marketplace add-ons: type must not be empty.

Change ignores add-ons with issues, giving:

2023-01-26 16:19:15.311 [WARN ] [ity.CommunityMarketplaceAddonService] - Ignoring marketplace add-on 'About the Add-on Marketplace category' due: type must not be empty

So I guess About the Add-on Marketplace category is the offending entry But IMHO it makes sense to skip those with error anyway and still display others …

[hmerk (Hans-Jörg Merk)]

@rlkoshak

zigbee:device:zg_coordinator:000d6f000a79badf:000D6F000A79BADF_1_voltage

This looks strange from my perspective. Do you really have the coordinator id twice in the name? Also having two underscores in the name could be an issue. Did you use this combination in openHAB 3 ?

[hmerk (Hans-Jörg Merk)]

@crnjan

So I guess About the Add-on Marketplace category is the offending entry

Not the only one, I found a couple of others and just added some tags, but might not have identified all.

[crnjan]

I only get that one logged and all others seem fine, at least they don’t fail to parse …

[hmerk (Hans-Jörg Merk)]

Ah, now I understand. It is failing, cause this post is not tagged as bundle, widget or rule template. It is not the published, beta, stable etc tags.

[rlkoshak (Rich Koshak)]

@crnjan

skip those with error anyway and still display others …

Since this is not something that the end user can do anything about, I think the log should be DEBUG at most. We don’t want to clutter end users’s logs with errors or warnings they can’t fix and are not their problem.

@hmerk

This looks strange from my perspective. Do you really have the coordinator id twice in the name? Also having two underscores in the name could be an issue. Did you use this combination in openHAB 3 ?

Yes, these are the same ID’s these Things have been using since OH 2.x in fact. These were auto-discovered so the IDs are what ever the binding used when they were created.

We think we found the root cause (a one line change in the Zigbee binding is required). More details on the issue (link above).

[J-N-K (Jan N. Klug)]

The issue is that the ZigBee binding passes the full UID as string where it should only pass the id-part. The UID itself is fine. This was not working before, but did not result in an error. Since it is a clear bug to pass the UID instead of the id, this should be fixed in the binding.

[J-N-K (Jan N. Klug)]

I agree. It is at least unexpected that the configuration is not “restored” and the backup is used again. I don’t think it has a high priority to fix it, but please open an issue for that.

[rlkoshak (Rich Koshak)]

I’m back again with some new weirdness.

The following code:

var item = items.getItem('TestSwitch');
item.postUpdate( ((item.state == "ON") ? "OFF" : "ON") );
setTimeout( () => {
  console.log('changedSince true: ' + items.getItem('TestSwitch').history.changedSince(time.toZDT('PT-1M'), 'rrd4j'));
  console.log('changedSince false: ' + items.getItem('TestSwitch').history.changedSince(time.toZDT('PT-30S'), 'rrd4j')); 
}, time.toZDT('PT50S').getMillisFromNow());

generates a Java Heap Exception.

rm│2023-02-01 14:47:09.646 [WARN ] [ore.internal.scheduler.SchedulerImpl] - Scheduled job 'org.openhab.automation.script.ui.scratchpad.timeout.1' failed and stopped       │
20│org.graalvm.polyglot.PolyglotException: Java heap space                                                                                                                 │la
rm│        at org.rrd4j.core.Archive.fetchData(Archive.java:303) ~[?:?]                                                                                                    │
20│        at org.rrd4j.core.RrdDb.fetchData(RrdDb.java:986) ~[?:?]                                                                                                        │la
rm│        at org.rrd4j.core.FetchRequest.fetchData(FetchRequest.java:163) ~[?:?]                                                                                          │
20│        at org.openhab.persistence.rrd4j.internal.RRD4jPersistenceService.query(RRD4jPersistenceService.java:330) ~[?:?]                                                │
20│        at org.openhab.core.persistence.extensions.PersistenceExtensions.getAllStatesBetween(PersistenceExtensions.java:1263) ~[?:?]                                    │
20│        at org.openhab.core.persistence.extensions.PersistenceExtensions.internalChanged(PersistenceExtensions.java:307) ~[?:?]                                         │
20│        at org.openhab.core.persistence.extensions.PersistenceExtensions.changedSince(PersistenceExtensions.java:286) ~[?:?]                                            │
00│        at org.openhab.core.persistence.extensions.PersistenceExtensions.changedSince(PersistenceExtensions.java:259) ~[?:?]                                            │36
20│        at java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(DirectMethodHandle$Holder) ~[?:?]                                                                    │
20│        at java.lang.invoke.LambdaForm$MH/0x00000001015c8c00.invoke(LambdaForm$MH) ~[?:?]                                                                               │
20│        at java.lang.invoke.LambdaForm$MH/0x0000000100d59400.invokeExact_MT(LambdaForm$MH) ~[?:?]                                                                       │
20│        at <js>.changedSince(/openhab/conf/automation/js/node_modules/openhab/items/item-history.js:68) ~[?:?]                                                          │
20│        at <js>.:=>(<eval>:7) ~[?:?]                                                                                                                                    │

Changing persistence from rrd4j to mapdb works as expected (or at least doesn’t throw an error, but changedSince returns true in both cases which shouldn’t be the case but that’s a different issue.

[crnjan]

@rlkoshak - can you try latest snapshot where this is already included?

[rlkoshak (Rich Koshak)]

I tested on snapshot #3296 and I still get the exception.

Given that that fix was merged a mere 18 hours ago, maybe it won’t be available until #3297? I’m on Docker so the snapshots are potentially delayed a bit since a new image can’t be created until after the distro build completes.

Any idea where to look to see the status of the snapshot builds?

Edit: Indeed it looks like the snapshot is delayed. I just saw mention of #3298 in a new issue Things not in alphabetical order on UI · Issue #1673 · openhab/openhab-webui · GitHub so I may have to wait a couple of days to test this.

Edit2: OK, managed to test on 3299 and can confirm that the exception went away.

thanks!

[rlkoshak (Rich Koshak)]

This one is a minor nit pick but has the potential to become really annoying for upgraders.

After an upgrade from 3.4 to 4.0, the javascript tranformation add-on remains listed in /var/lib/openhab/config/org/openhab/addons.config and I can see no way to remove it though the UI directly.

I tried to explore the REST API but I couldn’t figure out a way to get any information or remove it from there.

I looked in the karaf console but I can’t find a way to list that it’s configured to install there. Obviously it doesn’t show up under bundles because it cannot be installed.

The only way I can figure out to remove it from being attempted to install (and the resultant error) is to manually edit addons.config.

Is there a way we can automatically remove this add-ons during the upgrade?

[rlkoshak (Rich Koshak)]

I wish I had more details to report on this one but I don’t. Please let me know what might prove useful and I’ll gather it.

What I’m finding is an inconsistent restart behavior with OH.

ScriptEngine for language not found

2023-02-09 09:07:38.776 [ERROR] [ipt.internal.ScriptEngineManagerImpl] - ScriptEngine for language 'application/javascript' could not be found for identifier: 241daa5c-789c-4e0d-a430-75c38fef80b2

About one in three restarts will generate a ton of these errors (I suspect one for each rule).

system runlevel triggered rules

Sometimes, these rules just never get triggered. I have them set to runlevel 40 but I’ve tried every run level available and see the same result. Occasionally (I can’t give a good measure for how often yet) they simply won’t trigger. Running them manually later works just fine and non-runlevel triggered rules run fine too.

I don’t have any runlevel rules where that is the only trigger for the rule, if that matters. There’s always at least one other trigger.

rules page fails to load
At least twice after a restart the rules page will fail to load. Scripts and Schedule loaded just fine. This one is really odd and may just be gremlins. I’ll report more when/if it happens again. It’s only happened to me twice so far. I’m mainly posting here in case others have see this.

Edit: I think I may have found the root cause of this one. I was attempting to avoid some “Multi threaded access” errors on a rule that calls another rule and sometimes it gets pounded. I’ve applied a gatekeeper to spread the commands out and as I’ve increased the timing I’m starting to see:

  │2023-02-09 09:22:29.408 [WARN ] [ab.core.internal.events.EventHandler] - Dispatching/filtering event for subscriber 'org.openhab.core.events.EventSubscriber' failed: Ta│
  │sk java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2f538f6d[Not completed, task = java.util.concurrent.Executors$RunnableAdapter@5f063a90[Wrapped t│
  │ask = org.openhab.core.automation.internal.TriggerHandlerCallbackImpl$TriggerData@6d9588f8]] rejected from java.util.concurrent.ScheduledThreadPoolExecutor@796b02e2[Ter│
  │minated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]                                                                                      │
  │java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2f538f6d[Not completed, task = java.util.conc│
  │urrent.Executors$RunnableAdapter@5f063a90[Wrapped task = org.openhab.core.automation.internal.TriggerHandlerCallbackImpl$TriggerData@6d9588f8]] rejected from java.util.│
  │concurrent.ScheduledThreadPoolExecutor@796b02e2[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]                                   │
  │        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065) ~[?:?]                                                   │
  │        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833) ~[?:?]                                                                           │
  │        at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:340) ~[?:?]                                                 │
00│        at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:562) ~[?:?]                                                       │
20│        at java.util.concurrent.ScheduledThreadPoolExecutor.submit(ScheduledThreadPoolExecutor.java:715) ~[?:?]                                                         │
20│        at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:748) ~[?:?]                                                                    │
20│        at org.openhab.core.automation.internal.TriggerHandlerCallbackImpl.triggered(TriggerHandlerCallbackImpl.java:57) ~[?:?]                                         │
20│        at org.openhab.core.automation.internal.module.handler.GroupStateTriggerHandler.receive(GroupStateTriggerHandler.java:144) ~[?:?]                               │
20│        at org.openhab.core.internal.events.EventHandler.lambda$0(EventHandler.java:151) ~[?:?]                                                                         │
20│        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) [?:?]                                                                                │
20│        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]                                                                                               │
20│        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]                                                                        │
20│        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]                                                                        │
20│        at java.lang.Thread.run(Thread.java:833) [?:?]

Now this would make sense if gatekeeper created a separate timer for each event awaiting it’s time to run but it doesn’t. It has one looping timer that works off a queue.

More investigation is required but this is a problem I’m sure I’m causing through abusing OH in ways it can’t handle.

**No config description"
Every reboot I see this warning:

2023-02-09 08:56:09.678 [WARN ] [core.thing.internal.ThingManagerImpl] - No config description for 'channel-type:http:channel-config-switch' found when normalizing configuration for 'http:url:adguard:protection_status'. This is probably a bug.

This is related to the HTTP binding (see Simple AdGuard Home Integration using the HTTP Binding).

I’m not sure what to do about it but the Things still appears to work.

[laursen (Jacob Laursen)]

@rlkoshak

Every reboot I see this warning:

2023-02-09 08:56:09.678 [WARN ] [core.thing.internal.ThingManagerImpl] - No config description for 'channel-type:http:channel-config-switch' found when normalizing configuration for 'http:url:adguard:protection_status'. This is probably a bug.

This is related to the HTTP binding (see Simple AdGuard Home Integration using the HTTP Binding).

I don’t think this is directly related to the HTTP binding, but could be related to bindings having channel configuration. I’m seeing the same for a new binding I’m developing, but not consistently. I think it may get fixed by:

Improve thing initialization and enable thing-type updates

[MartOs]

Hi,

I’m finding a few bugs and wondering if anyone knows anything about them before I submit issues on gitub.

1. time.Duration

      const timerDuration = items.getItem('BackHall_Ceiling_Light_MotionTimer').state // A Number Item

      console.info(time.Duration.ofSeconds(1))
      console.info(time.Duration.ofMinutes(1))

      console.info(time.Duration.ofSeconds(timerDuration))

give me PT1S, PT1M, and PT1S respectively.

But

console.info(time.Duration.ofMinutes(timerDuration))

throws an error:

ArithmeticException: Multiplication overflows: 1 * 60: ArithmeticException: Multiplication overflows: 1 * 60

and ofHours says something similar:

ArithmeticException: Multiplication overflows: 1 * 3600: ArithmeticException: Multiplication overflows: 1 * 3600

only ‘ofSeconds’ seems to work.

2. things.getThings() .setLabel
When I change a thing’s label with JS Scripting, it works, but doesn’t stick after a reboot.

things.getThing('test:thing:45636344247:62464367').setLabel('A New Label')

3. items.replaceItem
With JS Scripting, if I set the giBaseType to ‘String’, it doesn’t apply this to the new item. Other base types and aggregation functions work fine.

items.replaceItem({
        type: 'Group',
        name: 'Item_Name',
        label: 'Item Label',
        groups: ['group1', 'group2'],
        tags: ['Setpoint'],
        category: 'lightbulb',
        giBaseType: 'Switch'
        //  groupFunction: aggregationFunction
    });

4. Applying metadata
When I apply metadata (any kind) using JS scripting, it works, but the UI displays ‘Not Set’

let itemToEdit = items.getItem('Test_Item);
itemToEdit.replaceMetadata('stateDescription', '', { options: 'ON=MOTION,OFF=NO MOTION'} );

5. material icons
I can apply a material icon and it works in most places.

but it shows as blank in a default list widget

******** I Posted this in the wrong place and am moving it here. I already got a reply which I’ll post here: *******

Please post to openHAB 4.0 SNAPSHOT discussion where the 4.0 SNAPSHOT problems are discussed.

@MartOs

1. time.Duration

Note that timerDuration in this case is a String. The .state is always a String. You might need to parse it to a Number first.

But in any case, this is 100% implemented in the joda-js library so any issue would need to be filed and corrected there. It’s not an OH problem.

@MartOs

2. things.getThings() .setLabel
When I change a thing’s label with JS Scripting, it works, but doesn’t stick after a reboot.

How is the Thing defined? In a text file or through the UI?

What use case would have you changing the label of a Thing?

@MartOs

When I apply metadata (any kind) using JS scripting, it works, but the UI displays ‘Not Set’

Note, this code is not correct (missing a closing ’ on the first line).

Which version of openhab-js are you using? The metadata was completely reworked in the latest release.

@MartOs

5. material icons

Show the full config from the “Code” tab.

@MartOs

Am I doing something wrong? Seems unusual to find 5 bugs in a row

You are testing the 4.0 SNAPSHOT. Tons of breaking changes are being introduced with almost every build. Welcome to the bleeding edge.