Issues when pairing HomePilot with Hue Emulator

Hundertvolt1 · November 3, 2023, 9:54am

Platform information:
- Hardware: RPI4 / 8GB
- OS: openHABian
- openHAB version: 4.0.3 stable

I am using a Rademacher Homepilot together with an openHAB server.
At this moment, there is “only” a HTTP binding available where I found a lot of possibilities recently.

So far, so good, but the whole thing only works close to real-time acceptable if you poll the Homepilot to death with 2 secs interval. But well, just close to acceptable.

As HomePilot is compatible to Hue and can “even officially” handle multiple bridges, my idea is to use the Hue Emulation from OH and introduce proxy items as “Switchable” devices. These can be added to scenes and triggers in the HomePilot config, and in OH they can trigger events on update. Et voila, we have event trigger forwarding for Homepilot for time-critical applications and can waaaaay relax the polling interval

And now I ran into issues described in many similar ways, but yet none of the solutions worked for me. My current state:

Set up the Hue Emulation just as is.
Changed the HTTP port of OpenHAB to 80 in etc/default/openhab (just to be sure I don’t mess up a port forwarding, I can do this later when it actually works ). Rebooted. Works, the openHAB UI now is reachable at port 80.
Enable pairing mode.
Under http://<openHAB_IP>/api/status, the reachability test looks great:

URL	Responds?	Ours?
http://192.168.85.2:80/description.xml	yes	yes
http://192.168.85.2/description.xml	yes	yes
http://[2a06:be00:346:0:b541:a13e:ba8b:c78d]:80/description.xml	yes	yes
http://[2a06:be00:346:0:b541:a13e:ba8b:c78d]/description.xml	yes	yes
http://[fe80:0:0:0:e58c:84c4:68b5:90a5]:80/description.xml	yes	yes
http://[fe80:0:0:0:e58c:84c4:68b5:90a5]/description.xml	yes	yes

With pairing mode still on / active, I was a bit confused by http://<openHAB_IP>/api/testuser/lights and http://<openHAB_IP>/api/testuser/sensors both giving the response:
[{"error":{"address":"testuser/sensors","description":"Not Authorized","type":1}}]
Now, anyhow, I tried to pair with HomePilot. I made sure pairing mode is on in OH, started searching for new devices in HomePilot, and actually, a new HUE Bridge was immediately detected. But whenever I tried to add this device, HomePilot informed me that the pairing process failed. I tried again and again and again, also with add unknown user ID enabled, with API v1 emulation enabled, but nope.
Then, I enabled debug logging in the emulator - enable discovery:

2023-11-03 10:40:41.014 [INFO ] [io.hueemulation.internal.ConfigStore] - Using discovery ip 192.168.85.2
2023-11-03 10:40:41.015 [INFO ] [io.hueemulation.internal.ConfigStore] - Hue Emulation pairing enabled for 60s
2023-11-03 10:40:41.108 [INFO ] [ueemulation.internal.upnp.UpnpServer] - Hue Emulation UPNP server started on 192.168.85.2:80

No logs to be found while HomePilot scans for new devices (but finds it). Then I click on “Pair” in the HomePilot and this appears in the OH log:

2023-11-03 10:41:13.293 [DEBUG] [ulation.internal.HueEmulationService] - REST request GET /
2023-11-03 10:41:13.294 [DEBUG] [ulation.internal.HueEmulationService] - REST response: [{"error":{"address":"/","description":"Not Authorized","type":4}}]

Note that this is a bit different from the “Not Authorized” with the manual tests, it just requests the address “/” and the error type is “4” instead of “1”.

Now I’m a bit lost… help is very much appreciated!

EDIT:
I can connect to / pair with the emulated bridge with the All4Hue app. All on port 80. It displays the virtual devices as well. Yet, interestingly, autodetection did not work, I had to enter the IP manually.

So it seems that the emulator is not actually “broken” and runs so far.

Vice versa this means that maybe the Homepilot does the pairing process somewhat differently. It tries to access http://<openHAB_IP>/api/ directly.
It’s a bit strange, because the All4Hue app does not do such a thing at all, it starts with a GET CONFIG.

And, I just tested - the original Hue bridge would answer with:

[{"error":{"type":4,"address":"/","description":"method, GET, not available for resource, /"}}]

to this request, no matter if the button was pressed or not. Whatever they want to achieve with this, but it seems to expect the reply above instead of

 [{"error":{"address":"/","description":"Not Authorized","type":4}}]

The only difference I can see is the order of the elements and the description which should actually not be such a big deal for JSON…

Help even more appreciated

MikeJMajor · November 4, 2023, 7:59am

How did you determine that? Are you sure it’s “/api/” and not “/api”. If it’s the latter then there’s info in the documentation on how to solve that.

Hundertvolt1 · November 4, 2023, 8:18pm

Thanks for the quick reply, very valid question indeed, I did not check this exactly and tried many new things…

TL;DR: All the measures are in place, but still the same error

What I did in detail:

Set the native openHAB port back to its default 8080, reboot.
Install nginx as described here in the reverse proxy tutorial, check the default page is reachable under port 80.
Set up the actual reverse proxy → success, OH is reachable under p8080 and p80 now. No further steps (like auth, SSL or the like) done.
Set up the config as described:

server {
  listen 80;
  location / {
    proxy_pass                              http://localhost:8080/;
    proxy_set_header Host                   $http_host;
    proxy_set_header X-Real-IP              $remote_addr;
    proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto      $scheme;
  }
}

server {
  listen 80;
  location /api {
    proxy_pass http://localhost:8080/api/;
  }
}

I tried again with several variations for detection (default, unknown user keys, API v1 emulation), but always the same drama as before:

2023-11-04 21:06:08.277 [DEBUG] [ulation.internal.HueEmulationService] - REST request GET /
2023-11-04 21:06:08.278 [DEBUG] [ulation.internal.HueEmulationService] - REST response: [{"error":{"address":"/","description":"Not Authorized","type":4}}]

and my HomePilot informing me that pairing failed, just as before
Therefore I hope for more ideas from you

By the way - I tried to verify stuff allover and over again with the All4Hue app. Meanwhile, I have 16 (!) different users when checking /api/status/. And I have no idea how to delete them…I already tried uninstalling / reinstalling the service, deleting hueEmulationUsers.json in the jsondb, but no way, the whole user list stays there… where can I do this?

Just for completeness, here the description.xml file, if that helps any further…

<root xmlns="urn:schemas-upnp-org:device-1-0">
<specVersion>
<major>1</major>
<minor>0</minor>
</specVersion>
<URLBase>http://192.168.85.2:80/</URLBase>
<device>
<deviceType>urn:schemas-upnp-org:device:Basic:1</deviceType>
<friendlyName>openHAB Hue Emulation (192.168.85.2)</friendlyName>
<manufacturer>Royal Philips Electronics</manufacturer>
<manufacturerURL>http://www.openhab.org</manufacturerURL>
<modelDescription>Philips hue Personal Wireless Lighting</modelDescription>
<modelName>Philips hue bridge 2015</modelName>
<modelNumber>BSB002</modelNumber>
<modelURL>http://www.meethue.com</modelURL>
<serialNumber>CAE2DB2C010E</serialNumber>
<UDN>uuid:cae2db2c-010e-4b75-8eac-c73a31d84b65</UDN>
<gatewayName>openHAB</gatewayName>
<presentationURL>index.html</presentationURL>
<iconList>
<icon>
<mimetype>image/png</mimetype>
<height>48</height>
<width>48</width>
<depth>24</depth>
<url>hue_logo_0.png</url>
</icon>
<icon>
<mimetype>image/png</mimetype>
<height>120</height>
<width>120</width>
<depth>24</depth>
<url>hue_logo_3.png</url>
</icon>
</iconList>
</device>
</root>

Hundertvolt1 · November 8, 2023, 7:57pm

Well, that was wild. I’m quite stubborn as long as I have the feeling it could actually work.

After hours of testing with a diyHue installation on another SD card (which pairs with Homepilot instantaneously!) and field-by-field checking of network frames in Wireshark I found the solution.

First of all: The actual bug seems to be done by the HomePilot himself.
Whenever it requests information via the Hue API, it adds a HTTP header

Accept: text/plain

which is, if you’re actually expecting a JSON response, some quite impressive bullshit

Fun fact: All the other Hue Emulators just ignore this header. The openHAB emulator is more dilligent here - it reads this field, knows that it’s returning Json and answers NULL. So the “mistake” of openHAB is to adhere closely to the spec, woohoo and if you just leave away this header it works.

The absolutely good news is: If you have nginx in place anyhow, it allows filtering headers. And exactly this is the solution.

So if you have a Rademacher HomePilot (or SmartHomeBox, as they call it by now), you have to configure the Hue Emulator just by default and let it announce Port 80. Set nginx as proxy to Port 8080 as described in the tutorial, and then add the following lines:

      location /api {
        proxy_pass http://localhost:8080/api/;
        proxy_set_header Accept "";
    }
     location /api/ {
       proxy_pass http://localhost:8080/api/;
       proxy_set_header Accept "";
    }

This will cater for the case of the missing / in the URL as well as throw away the “Accept” header. And then it magically works - pairing as well as seeing the new devices in HomePilot.

And, don’t forget, this gives you something like a superfast callback possibility from HomePilot to OH!

Hundertvolt1 · November 9, 2023, 4:28pm

Well, everything up and running so far.

One more question, may be important:
Did anyone ever notice that the file hueEmulationUsers.json (in the json database) contains two timestamps: creation of the user, and last usage of this account.
Last usage is, well, interesting, if you look at devices like HomePilot which is polling values every 2…5 secs. I checked this down to the file system.

yes, the timestamp gets updated with every API request.
yes, it’s being written into the jsondb every time.
yes, it’s also copied into the /backup folder of the jsondb every time.
yes, it’s just a few bytes everytime

… it will, therefore, for sure kill your SD card at some point. For a more or less useless value…

In these investigations I also noticed that it’s more or less the same with org.openhab.core.config.discovery.DiscoveryResult.json.

For now, I moved the two json files and the backup folder into a ZRAMed folder and symlinked these locations in the jsondb. But maybe this would be worth a closer look…?