Android app with OH4 (remove the Accept-Encoding header)

Apps & Services Android App
1

After upgrading to OH4 I can’t get the android app to work in remote mode anymore, using basic auth using nginx as the reverse proxy. If anyone know of any changes or has it working on nginx would be grateful if you posted your config. App works fine in local mode.

What used to happen in 3.x was the following, with my remote set to https://mywebite/openhab with basic auth user “bill”

172.56.167.141 - bill [10/Oct/2023:15:07:55 -0400] "GET /openhab/api/v1/settings/notifications HTTP/2.0" 302 0 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.56.167.141 - bill [10/Oct/2023:15:07:55 -0400] "GET /openhab/rest/ HTTP/2.0" 200 2275 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.56.167.141 - bill [10/Oct/2023:15:07:55 -0400] "GET /openhab/rest/sitemaps HTTP/2.0" 200 212 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.56.167.141 - bill [10/Oct/2023:15:07:55 -0400] "POST /openhab/rest/sitemaps/events/subscribe HTTP/2.0" 200 358 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.56.167.141 - bill [10/Oct/2023:15:07:55 -0400] "GET /openhab/rest/sitemaps/main/main?includeHidden=true HTTP/2.0" 200 5255 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"

Now it seems that the “openhab” gets truncated eventually and the android app just times out:

172.58.236.52 - bill [11/Oct/2023:18:19:04 -0400] "GET /openhab/rest/ HTTP/2.0" 200 449 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.58.236.52 - bill [11/Oct/2023:18:19:04 -0400] "GET /openhab/rest/sitemaps HTTP/2.0" 200 130 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.58.236.52 - bill [11/Oct/2023:18:19:04 -0400] "GET /rest/sitemaps/main/main?includeHidden=true HTTP/2.0" 302 138 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.58.236.52 - bill [11/Oct/2023:18:19:04 -0400] "POST /openhab/rest/sitemaps/events/subscribe HTTP/2.0" 200 358 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"
172.58.236.52 - bill [11/Oct/2023:18:19:04 -0400] "GET /login?url=https://bill.net/rest/sitemaps/main/main?includeHidden=true&vouch-failcount=&X-Vouch-Token=&error= HTTP/2.0" 302 260 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.
172.58.236.52 - bill [11/Oct/2023:18:19:05 -0400] "GET /rest/sitemaps/main/main?includeHidden=true HTTP/2.0" 302 138 "-" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.133 Mobile Safari/535.19"

I’m sure it’s a config error on my part but it was working until I upgraded which is why I was wondering if anyone else had an issue. The request GET /rest/sitemaps is incorrect and results in a redirect to my login page because it is not under openhab.

Here is my current nginx config for openhab that was working until I upgraded.

        location /openhab/ {
         auth_basic                              "Username and Password Required";
         auth_basic_user_file                    /etc/nginx/passwords/openhab; #htpasswd -c openhab username
         proxy_set_header Authorization "";      #remove this header as far as OH3 is concerned

         proxy_pass http://openhab/;

         proxy_set_header Host                 $http_host;
         proxy_set_header X-Real-IP            $remote_addr;
         proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for; # add the real ip of the client
         proxy_set_header X-Forwarded-Proto    $scheme; # tell openhab we came in via tls
         proxy_set_header Upgrade              $http_upgrade;

         sub_filter_types application/javascript text/xml text/css text/javascript application/json text/plain;
         sub_filter_once off;
         sub_filter '/rest/' '/openhab/rest/';
         sub_filter '"/icon/' '"/openhab/icon/';
         sub_filter '"/basicui/' '"/openhab/basicui/';
         sub_filter '"/chart' '"/openhab/chart';
      }

Thanks.

2

Wouldn’t it be easier to use a dedicated subdomain like https://openhab.mywebsite instead of a subpage ?
What I am wondering is if the filters are (still) correct. Especially those that use double quotes.

3

The problem with that is renewing the TLS certs. I didn’t ever bother to find a way to autorenew a letsencrypt wildcard cert to have multiple domains. Last time I checked their process was not something that could be automated with my DNS provider.

I have different stuff running below my web site. The sub_filters were working fine until I upgraded from 3.4 to 4.0 the other day. I don’t think it’s the android app per se, but whatever change was made affects the android app due to the way I connect (it breaks the older and newer versions of the app). I have another URL that reverse proxies the basicui itself using okta, and that still works fine.

I will play around and see if can find out what changed. Just wondering if anyone else had come across it.

4

out of curiosity may I ask what your DNS provider is ? You may send a PM in case you don’t want to share it here.
I also use letsencrypt and use a script that was shared on the internet.

Did you check if the double quotes are still at their location in the traffic/files ?

5

I use namecheap.

Will check on the quotes. They do work fine on basicui.

6

It’s interesting the app gets the list of sitemaps via

GET /openhab/rest/sitemaps HTTP/2.0" 200

and looking at the content of the response it is properly rewritten to.

[
  {
    "name": "main",
    "label": "Main",
    "link": "https://xxx.net/openhab/rest/sitemaps/main",
    "homepage": {
      "link": "https://xxx.net/openhab/rest/sitemaps/main/main",
      "leaf": false,
      "timeout": false,
      "widgets": []
    }
  }
]

Yet, the very next thing from the app is…
GET /rest/sitemaps/main/main?includeHidden=true HTTP/2.0
and that of course fails…

In my logs using 3.4 it shows correct:
"GET /openhab/rest/sitemaps/main/main?includeHidden=true HTTP/2.0" 200

I will have to boot up a version of 3.4 and see what the difference is. Version shouldn’t matter based on what I’ve seen so far…

7

I am wondering if sub_filter matches your complete use case.
As far as I understand sub_filter only rewrites the response. If a rewrite is required for a request it is not covered.

8

I have a wildcard let’s encrypt certificate for *.mydomain.com and running OH behind reverse proxy on openhab.mydomain.com
It’s really easy to setup, I dont see a difference between wildcard and non wildcard certificate. the process is the same, set and forget. ( How To Create Let's Encrypt Wildcard Certificates with Certbot | DigitalOcean )

9

I assumed that was what the app was doing - getting a list of sitemap urls (from the response). The sitemap urls are correct.

I can of course rewrite /rest to /openhab/rest (or just reverse proxy /rest), but I should not have to do that. Prior to 4.0 it just worked. Leaving something as generic as /rest at top level with basic auth invites hacking…

10

Well this was an interesting problem to fix.

It seems like OH4 changed something w/r to the webserver where it now sends back compressed responses if the client supports it. Previously it would be uncompressed. A caveat to using sub_filter is that it does not work with compressed upstream responses. Superficially testing an OH3 and OH4 server with curl would produce the same exact json results. Only when you show the headers would you see the latest OH has

vary: Accept-Encoding, User-Agent

This was the key piece of information needed to fix the problem.

Therefore, for anyone reverse proxying using nginx using paths below root, add the following to the nginx.conf


         proxy_set_header Accept-Encoding "";    #sub_filter does not work with compressed upstream responses

So the complete config is:


        location /openhab/ {
         auth_basic                              "Username and Password Required";
         auth_basic_user_file                    /etc/nginx/passwords/openhab; #htpasswd -c openhab username
         proxy_set_header Authorization "";      #remove this header as far as OH is concerned
         proxy_set_header Accept-Encoding "";    #sub_filter does not work with compressed upstream responses



         proxy_pass http://openhab/;

         proxy_set_header Host                 $http_host;
         proxy_set_header X-Real-IP            $remote_addr;
         proxy_set_header X-Forwarded-For      $proxy_add_x_forwarded_for; # add the real ip of the client
         proxy_set_header X-Forwarded-Proto    $scheme; # tell openhab we came in via tls
         proxy_set_header Upgrade              $http_upgrade;

         sub_filter_types application/javascript text/xml text/css text/javascript application/json text/plain;
         #sub_filter_types *;

         sub_filter_once off;
         sub_filter '/rest/' '/openhab/rest/';
         sub_filter '"/icon/' '"/openhab/icon/';
         sub_filter '"/basicui/' '"/openhab/basicui/';
         sub_filter '"/chart' '"/openhab/chart';
         sub_filter '"/api' '"/openhab/api';

      }