No device answering anymore in HomeKit after OH3 reboots

fnu · August 12, 2021, 10:42pm

Hi there,

after rebooting OH3 @ RPi3, HomeKit basically stops working with my OH HomeKit bridge. Only solution out I found so far is to delete the openHAB bridge in HomeKit and re-add it.

But that’s pretty annoying, as I have to run again and again thru HomeKit initial setup, mapping the items to the rooms, actually around 40 devices …

Is that normal behavior? Or is there a way to prevent that behavior?

A reboot of that openHABian Raspberry is honestly nothing special will happen here and there for various reasons e.g. updates …

Cheers

fnu · August 12, 2021, 11:52pm

Hmm, ok, found an old thread saying the secret is mapdb persistence … installed, made it default, hopefully correctly … lets see.

yfre · August 13, 2021, 7:29am

it is not normal behaviour. i do regular updates of my raspberry and reboot every second months or so. after reboot everything is back.
i have not done anything with mapdb… to my understanding, mapdb is used to store the state of the device but not the configuration. homekit configuration is stored in a file (homekit.json)

fnu · August 13, 2021, 9:54am

@yfre Thanks for replying, but what can then be the topic here?

homekit is as far as I can see directly available after reboot, other services does find it immediately and point to. Also delete and re-add openhab Bridge to HomeKit does work immediately …

fnu · August 13, 2021, 10:01am

Search for homekit.json, gave me 2 files on the system, but both of them don’t contain any mapping, just 5 entities, privateKey, user, salt, accessory_count, mac and revision …

yfre · August 13, 2021, 11:11am

it is correct, homekit.json consits only these attributes. but they are important to pairing.

pairing is like this

iphone send its mac address and key.
openhab stores mac and key as user in homekit.json
openhab sends its mac address and key to iphone
pairing done

if iphone does not find accessories anymore (no response) then it is usually one of following issue

network issue
iphone sends another mac (see private wifi Use private Wi-Fi addresses in iOS 14, iPadOS 14, and watchOS 7 - Apple Support)
homekit.json got broken and has no user entry anymore

not sure when your homekit.json were created, maybe they a new and after reboot you had no homekit.json or homekit.json without user entry

fnu · August 13, 2021, 11:56am

I de-installed openHAB homekit yesterday, installed mapdb, and installed homekit again in OH. homekit.json has the timestamp according to that now.

For my home WiFi I have disabled private MAC address on my iPad and also iPhone. My iPad is defined as HomeKit Bridge/CentralCommand and I setup HomeKit stuff on my iPad, as it is more comfortable than on iPhone.

But my iPad is also running always iOS Beta versions, actually iOS15. That comes with a “Privat-Relay” addon for iCloud … hmm, I wonder if that could have influence here.

From my first basic understanding it shouldn’t, but well …

fnu · August 13, 2021, 12:38pm

Obviously “Privat-Relay” can be disabled per individual WiFi connection in iOS15.

So, done for my home WiFi, albeit still not sure if this is the reason.

fnu · August 14, 2021, 11:05am

Ok, to make it short nothing did help so far, restarting OH3 makes all devices in HomeKit immediately unavailable.

mapdb didn’t do the trick, but I leave it installed, as it looks like a slim helpful addon, after reading documentation. It does store last status’/values, being available after next restart.

New iOS “Privat-Relay” forth and back did not change a thing at all, tested 4 times w/ and w/o.

In PaperUI is a setting for homekit regarding mDNS. Shall that be enabled or not?

And if yes, what is the line for “services/homekit.cfg”?

yfre · August 15, 2021, 10:58am

can you check homekit.json? do you have “user” there? anything in the log files?

mDNS is the service that publish the ip address of the homekit bridge so that home app can find it. openHAB has built-in mDNS used for other binding. this switch defines whether homekit should use mDNS from openHAB or start a new mDNS service just homekit.

in general, usage mDNS from openHAB is a good idea as it save memory, etc. but there were reports that it is not always working for homekit. therefore the switch.

fnu · August 15, 2021, 1:26pm

@yfre I deleted homekit.json several times for testing purposes, to check what’s happening with it.

A “user” attribute gets reliable created as soon as I add openhab to HomeKit at my iPad and also reliable deleted if I remove openhab bridge from HomeKit, when all devices are shown as unavailable resp. “not answering” … I checked that life watching “homekit.json”.

And that wonders me the most, I don’t see any messages in log after reboot RPi or just openhab regarding homekit. Whereas to be honest I haven’t seen any message related to HomeKit.

So, ok, need to find out, how to enable “using openhab mDNS for homekit” in “homekit.cfg”.

fnu · August 16, 2021, 9:23am

So, enabled tracing according to documentation, restarted OH and:

openhab> log:tail io.github.hapjava
11:17:34.716 [WARN ] [a.server.impl.connections.HttpSession] - Exception encountered while verifying pairing
java.lang.Exception: Unknown user: 7CD5A814-42C8-4766-B69D-7AB5A4AB6C1B
        at io.github.hapjava.server.impl.pairing.PairVerificationManager.stage2(PairVerificationManager.java:113) ~[bundleFile:?]
        at io.github.hapjava.server.impl.pairing.PairVerificationManager.handle(PairVerificationManager.java:51) ~[bundleFile:?]
        at io.github.hapjava.server.impl.connections.HttpSession.handlePairVerify(HttpSession.java:127) [bundleFile:?]
        at io.github.hapjava.server.impl.connections.HttpSession.handleRequest(HttpSession.java:56) [bundleFile:?]
        at io.github.hapjava.server.impl.connections.ConnectionImpl.doHandleRequest(ConnectionImpl.java:56) [bundleFile:?]
        at io.github.hapjava.server.impl.connections.ConnectionImpl.handleRequest(ConnectionImpl.java:49) [bundleFile:?]
        at io.github.hapjava.server.impl.http.impl.AccessoryHandler.channelRead0(AccessoryHandler.java:52) [bundleFile:?]
        at io.github.hapjava.server.impl.http.impl.AccessoryHandler.channelRead0(AccessoryHandler.java:17) [bundleFile:?]
        at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) [bundleFile:4.1.63.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [bundleFile:4.1.63.Final]
        at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:61) [bundleFile:4.1.63.Final]
        at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:370) [bundleFile:4.1.63.Final]
        at io.netty.util.concurrent.DefaultEventExecutor.run(DefaultEventExecutor.java:66) [bundleFile:4.1.63.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [bundleFile:4.1.63.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [bundleFile:4.1.63.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [bundleFile:4.1.63.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]

Then on removing openHAB bridge from HomeKit, no pairing exception …

11:24:23.064 [TRACE] [ver.impl.http.HomekitClientConnection] - 500 /pair-verify
11:24:23.081 [TRACE] [.server.impl.http.impl.LoggingHandler] - WRITE PooledUnsafeDirectByteBuf(ridx: 0, widx: 101, cap: 232) [/192.168.178.44:12345]:
485454502F312E312035303020496E7465726E616C20536572766572204572726F720D0A436F6E74656E742D4C656E6774683A2031390D0A436F6E6E656374696F6E3A206B6565702D616C6976650D0A0D0A6A6176612E6C616E672E457863657074696F6E

11:24:23.106 [TRACE] [erver.impl.http.impl.AccessoryHandler] - Terminated HomeKit connection from /192.168.178.44:12345
11:24:27.203 [TRACE] [erver.impl.http.impl.AccessoryHandler] - New HomeKit connection from /192.168.178.44:12345
11:24:27.206 [TRACE] [.server.impl.http.impl.LoggingHandler] - READ PooledUnsafeDirectByteBuf(ridx: 0, widx: 165, cap: 2048) [/192.168.178.44:12345]:
504F5354202F706169722D76657269667920485454502F312E310D0A486F73743A206F70656E4841425C3033322832292E5F6861702E5F7463702E6C6F63616C0D0A436F6E74656E742D4C656E6774683A2033370D0A436F6E74656E742D547970653A206170706C69636174696F6E2F70616972696E672B746C76380D0A0D0A06010103208BDBAB1BC00C09D5A9823A586A428E536F4FF3574999755EECCE694A3571FB18

11:24:27.227 [TRACE] [.impl.pairing.PairVerificationManager] - Starting pair verification for openHAB
11:24:27.248 [TRACE] [ver.impl.http.HomekitClientConnection] - 200 /pair-verify
11:24:27.254 [TRACE] [.server.impl.http.impl.LoggingHandler] - WRITE PooledUnsafeDirectByteBuf(ridx: 0, widx: 244, cap: 256) [/192.168.178.44:12345]:
485454502F312E3120323030204F4B0D0A436F6E74656E742D747970653A206170706C69636174696F6E2F70616972696E672B746C76380D0A436F6E74656E742D4C656E6774683A203134300D0A436F6E6E656374696F6E3A206B6565702D616C6976650D0A0D0A0601020565E8CE21F9EE425D1598339FC36F8C53D176E57833024492054302790C5B55CA31A8B4572AD1EB3894B232D4E200FD88B21D78EF0A6BD2E47FA0A69D70F8AD216838C7912F0BBF079C4C726FAC771541DC22E11707BFFD908780989A891895024C68839065EF0320113703B26950238F0EF0DF166CDF1BF983F120976CF78D8A4AE9897A72399757

11:24:27.309 [TRACE] [erver.impl.http.impl.AccessoryHandler] - Terminated HomeKit connection from /192.168.178.44:12345

yfre · August 16, 2021, 5:10pm

ok. either the user (= device) was removed from homekit.json for some reasons or the device has changed his MAC address (maybe due private wifi settings)

yfre · August 16, 2021, 5:11pm

if you have older homekit.json you can check whether you had this user there

fnu · August 16, 2021, 5:25pm

Once again, no rolling MAC or IP/IPv6, all is static for all devices in my home environment!

Only the MAC address generated by openhab into homekit.json I don’t have under control.

There are older files under “./backup”, but don’t contain any user entry anymore.

What user is that? What/who does generate it? Can it be controlled?

yfre · August 16, 2021, 9:11pm

“user” is device, e.g. iphone/ipad. it stores the MAC address which it has received from the device.
if it says "Unknown user: " that means that the mac address received from device was not found in the homekit.json
following activities remove “user” from homekit.json

device was unpair from openhab
clearPairings was executed from openhab karaf console

if none of these activities was done, then it is a bug in the binding or openhab

fnu · August 16, 2021, 10:46pm

Needed to search in prior what you mean here, so, none of those actions done so far.

But I could try to clearPairings, actually is anyway no device paired to OH … I’m little tired to walk thru HomeKit again and again …

yfre · August 17, 2021, 7:34am

clearPairings would basically delete “user”, i.e. device, from homekit.json so that openHAB will be ready for pairing again.

i would suggest, pair it again and make a copy of homekit.json afterwards so that you can restore it if it get destroyed after openhab / pi reboot

fnu · August 18, 2021, 8:43am

So, albeit my openhabian installation was just a few weeks old, I did a new installation from scratch yesterday, just to sort out I screwed anything within that. I took the chance to find out how fast I can recover in case a SD card is going to die, as I try to maintain my setup in plain files. It did work out good, recovery time is 1-1,5h, most time is going into waiting for openhabian getting ready after initial starts and restarts …

After all I have all my devices up and running and also HomeKit, saying in log to provide 42 devices for homekit.

@yfre Now added my openhab bridge again to HomeKit on iPad, scanning QR code. Checking directly what happens to “homekit.json”, I can see an user entry added. So, I copied that “homekit.json” for backup purpose.

Then the usual run thru HomeKit, sorting the devices to my HomeKit home. All good and I had the impression HomeKit reacts faster on my iPhone now.

This morning I checked again “homekit.json” and found out, there’s no user entry anymore! But HomeKit is usable perfectly on my iPad (HomeKit controller) and iPhone.

So, your idea an user is missing in homekit.json after reboot of OH might be the point. But that is obviously eradicated by openHAB itself … from unknown reason and time. In my case within the last 12-16h.

yfre · August 18, 2021, 9:24am

thank you for the investigation.
it is still unclear why the user get removed.
there are only these 2 cases in source code of homekit binding for “remove user”:

unpairing triggered by device
clearPairings (= unpairing triggered from karaf console)

but this is not your case.
there were some reports about user gets deleted when homekit configuration saved from UI but i could not reproduce it.