[homekit] "No response" error on all openHAB items

erikvdv1 · August 23, 2021, 11:14am

I have also had the problem that every few days all HomeKit devices show ‘no response’.

In my OH logs I found the following exception:

22:48:50.966 [WARN ] [javax.jmdns.impl.tasks.Responder     ] - Responder(*snip*)run() exception
java.io.IOException: Operation not permitted
        at java.net.PlainDatagramSocketImpl.send(Native Method) ~[?:?]
        at java.net.DatagramSocket.send(DatagramSocket.java:695) ~[?:?]
        at javax.jmdns.impl.JmDNSImpl.send(JmDNSImpl.java:1665) ~[bundleFile:3.5.7]
        at javax.jmdns.impl.tasks.Responder.run(Responder.java:154) [bundleFile:3.5.7]
        at java.util.TimerThread.mainLoop(Timer.java:556) [?:?]
        at java.util.TimerThread.run(Timer.java:506) [?:?]

After looking into the code of jmDNS I changed the following line and I have no issues since then.

A pre-built JAR file with the fix can be downloaded here: Dropbox - jmdns-3.5.7.jar - Simplify your life

Replace it with the original jmdns-3.5.7.jar. If you’re using docker, mount the JAR under /openhab/runtime/system/org/jmdns/jmdns/3.5.7/jmdns-3.5.7.jar

I suspect the exception is triggered by a mDNS message sent by one of my other devices on the network. After the exception occurs the OH jmdns client is stopped.

If other people can confirm this solves their ‘no reponse’ problem as well, I will create a pull request to jmdns.

bodomenke · September 14, 2021, 11:14am

I am experiencing the same problem (openHab devices with „no response“ in Apple Home app). In my case

toggling the mDNS-switch in HomeKit-Integration of openHab or
even a restart of openHab

does not heal the issue.

Is anybody able to confirm that replacing jmDNS with the updated version of @erikvdv1 does fix the problem?

Thanks

Nathaniel85 · September 15, 2021, 10:40am

@bodomenke I seem to have the same issues as you do, and neither restart or toggling the mDNS setting help. I think thats related to mDNS not working which does not seem to be my issue.

I confirmed with the DNS Discovery app that _hap._tcp. Exists and openHAB is listed. Information here including IP and port is accurate to binding. One thing I noticed is the sf=1 entry means openHAB is in pairing mode for HomeKit….
But, it was already paired. This happened after a openHAB restart last night.

I also confirmed that replacing the jmdns.jar file did not fix my issue.

I am running openHAB 3.1.0.

Any ideas anyone? I prefer not to re-pair again I just re-added and configured 60+ items and had it working until this morning.

yfre · September 15, 2021, 10:45am

good investigation.
the logic for pairing mode is simple. if there is an “user” entry in the homekit.json file then openhab is already paired, if not then advertise openHAB as ready for pairing with mDNS.

please check how your homekit.json file looks like. openhab also makes backup of this file, maybe you will find one with the user entry

bodomenke · September 15, 2021, 4:06pm

@yfre I will definitely check that as soon as I am back at home tomorrow or Friday.

Can you in the meanwhile bring some light into the following questions:

a) Why do you use the users MAC address as user ID?
b) Do you know, what causes, that the homekit.json file gets changed/corrupted/deleted?

Thanks

Bodo

yfre · September 15, 2021, 10:42pm

Hi Bodo

regarding the MAC address, this is something the underlaying library Java-HAP does. it calls createUser with the mac address + username of the paired device.
check here

github.com

hap-java/HAP-Java/blob/f4a9872d303a1108e14e97fd54f9c3927f236400/src/main/java/io/github/hapjava/server/impl/pairing/FinalPairHandler.java#L68

    
      
                    "Pair-Setup-Controller-Sign-Salt".getBytes(StandardCharsets.UTF_8),
                    "Pair-Setup-Controller-Sign-Info".getBytes(StandardCharsets.UTF_8)));
            byte[] okm = new byte[32];
            hkdf.generateBytes(okm, 0, 32);
          
          
  byte[] completeData = ByteUtils.joinBytes(okm, username, ltpk);
          
          
  if (!new EdsaVerifier(ltpk).verify(completeData, proof)) {
              throw new Exception("Invalid signature");
            }
            authInfo.createUser(authInfo.getMac() + new String(username, StandardCharsets.UTF_8), ltpk);
            return createResponse();
          }
          
          
private HttpResponse createResponse() throws Exception {
            HKDFBytesGenerator hkdf = new HKDFBytesGenerator(new SHA512Digest());
            hkdf.init(
                new HKDFParameters(
                    k,
                    "Pair-Setup-Accessory-Sign-Salt".getBytes(StandardCharsets.UTF_8),
                    "Pair-Setup-Accessory-Sign-Info".getBytes(StandardCharsets.UTF_8)));

i have not questioned this until now. but maybe we should indeed remove the mac address part.

we had in the last 2 years multiple reports about users entries got removed from homekit.json, with OH2.5 and OH3.x) the remaining part of homekit.json was still correct, only the user entry was missing. i could not reproduce this issue on my side. We did intensive investigation with the community here but we have never found the root cause or at least some indications for it. the problem disappear from alone after sometime in most of the cases.

from the homekit binding code perspective the user will be removed only in 2 cases

clearPairings is executed in karaf console
paired device request unpairing.

there are no other calls of “removeUser” in the code
so, it is still mystery for me

bodomenke · September 16, 2021, 5:51pm

Thanks Eugen.

I have checked my homekit.json:

It includes no user, but the following fields:
accessory_count, privateKey, salt, mac, and revision, but no user.

The MAC address is not known to me. It is not my iPhone I used for pairing or any other device in my network AFAIK. I did/do not use Private WiFi Address.

Any way I can restore the HomeKit integration in a state that is is working again?
I have 22 Zwave items for which I added meta data and I would avoid going thru the whole process especially if the problem can happen again.

Kind regards

Bodo

bodomenke · September 16, 2021, 6:56pm

I have just

removed HomeKit integration and re-added it,
repaired openHab via my iPhone
checked homekit.json for “user”: success
made an backup of homekit.json
re-added all items to Home.app again (that was easy and fast, since all metadata still available)
added additional items to HomeKit
Opened json.db for checking total numbers of accessories: now in total 37

IMPORTANT: While I was checking in step 7 the homekit.json for the number of “accessory_count”, guess what!? homekit.json does NOT contain the user any longer!

So, somewhere in between step 4 and 7 the homekit.json got touched and user entry removed.So it seems, that updates to the file (e.g. with accessory_count) corrupts the file…?

Interesting, even if homekit.json does not contain “user” Apple Home.app still sees openhab bridge working and all accessories responding. I guess this will last until I restart openHab.

What do you think?

EDIT: My saved homekit.json includes three users. The corrupted none. My saved file says revision 32, the corrupted one says 47.

EDIT2: I shut openHab down, replaced the corrupted homekit.json with the saved copy and started openhab again: success. Openhab bridge and all accessories working. So the copy of homekit.json saved me.

yfre · September 16, 2021, 7:55pm

thank you for the investigation. yes, homekit.json backup was a good idea.

what is strange - it is always removing the user and nothing else.
if the file would get corrupted completely or different parts would be removed, then i could think about caching or some interruption during file writing. but it is alway the user. and in the code i have not found any logic to remove the user from file, except of 2 cases mentioned above: clearPairing and unpairing from device…

strange.

bodomenke · September 16, 2021, 9:38pm

Hhmm. Let’s think trough it.

I did only add HomeHit meta data to existing items. It seems to me that during this process the homekit.json receives updates, which corrupts the file by removing the user.

I am not a developer. But would it be possible to debug step by step through the process of adding Homekit meta data to items to see, if there is anything which triggers the corrupting update to the homekit.json?

bodomenke · September 16, 2021, 9:49pm

@Nathaniel85 Pls check your homekit.json as I did (see thread upwards) and confirm if you are OK there. Maybe you have the same issue as I have.

Prillin · September 17, 2021, 1:47am

I have posted my finding before here:

For me editing the Metadata has no effect on the Homekit.json file, I have made at-least 100 changes to the metadata of my items since I first discovered the work around I use. its when I click save in HomeKit Add-on settings part of OpenHab that I lose integrity of the.json file.

Personally I make a backup of the .json file and replace it if my Items get stop responding. So far i’ve only needed to do this the first time I figured it out, and have since avoided making changes to the settings for HomeKit and thus far have not had this issue for more then 70 days now.

I am interested to know if anyone else can reproduce my findings. I don’t think it matters but I am running my OH Server on MacOS 10.14.6.

bodomenke · September 17, 2021, 4:45am

I can confirm, that I just added HomeKit meta data to an additional item, accessory_count got incremented with the add. item, but user was NOT removed. Strange…

Prillin · September 17, 2021, 5:00am

What happens if you go into the settings for the HomeKit add-on, change something like " * Use Fahrenheit Temperature" option, then save the changes and check the .json file. I suspect if you repeat that a few times, you will be able to produce a .json file that has lost data. I know I was messing with my Nest thermostat when I made the discovery. I know I was changing, Use Fahrenheit option often trying to get the temperatures to covert properly so Homekit displayed the correct Temperature in F. I was also changing the “Thermostat Target Heating/Cooling Mapping” and “Thermostat Current Heating/Cooling Mapping” frequently as well.

I suspect the damage is done when saving the settings to disk, but then only discovered at a later time when the .json file is reloaded. If I remember correctly Saving the settings did not Always produce the issue for me, but then again I did not systematically test each value individually. It’s possible only 1 of 2 of the of the settings might cause the damage, and I just happened to not change them some times witch would appear to me an intermittent issue.

I am not 100% certain but I also suspect that the file only gets rewritten to disk if one of the values has changed, so opening settings and changing nothing or deleting and reentering the same values then clicking save won’t cause the .json file to be rewritten, and only if something actually changes will the save take place.

bodomenke · September 17, 2021, 6:07am

I just changed setting “Start Delay” in HomeKit add-on multiple times and saved.
No change of homekit.json…

I believe we might have a coincidence between changes in HomeKit add-on, as experienced by @Prillin, or item updates, as experienced by me, with corruption of homekit.json.

Edit: Just added new Zwave devices and added HomeKit metadata to their items. No change of homekit.json. So, it seems it was really a coincidence before that I observed corruption of homekit.json after adding homekit meta data to items…

stefan13 · September 20, 2021, 12:37pm

Hello Eugen,

I have the same problem again.
User is removed from the JSON file.
However I found an old backup file with 2 User entries.
I copied the first user section into the recent JSON db file and restart the bundle.
Now SF value is 0 instead of 1 and Homekit does work again.
BTW - Does it make sense to have 2 users?

yfre · September 20, 2021, 12:59pm

hi Stefan,

openhab creates user for each paired device (it stores the public key of the device used for encryption).

however, to my understanding not every device need to be paired as they get connected via icloud. but i dont have details on this.

i have 3 users in my homekit json but a more than 3 ios devices at home.

bodomenke · September 26, 2021, 8:42am

I cannot track down the issue any further.
For me it is definitely not caused by

adding Homekit meta data to an item
changing the settings of the Homekit addon

Have now multiple backups of homekit.json.

Let‘s see what happens…

stefan13 · September 26, 2021, 9:50am

Hi Eugen,

Homekit stopped working yesterday when I activate my new iPhone.
Could be by coincidence as I cannot find openhab in Discovery app (_hap._tcp) anymore in .
I checked the homekit.json file and the user still exit in the file.
Toggling the mDNS option in openhab doesn’t solve the issue - also clearing the bundle doesn’t help.

==> /log/openhab.log <==
2021-09-26 12:00:38.485 [WARN ] [.server.impl.connections.HttpSession] - Exception encountered while verifying pairing
java.lang.Exception: Unknown user:

I guess this error could be related

yfre · September 27, 2021, 6:55am

Hi Stefan,

do you see openhab in Discovery app after you have changed the mDNS option? actually you should it see. maybe something with network setting, e.g. wrong wifi.

once you see it, it either will work or you would need to re-pair.