[SOLVED] Using openHABian with HTTPS and already certified domain (Synology NAS)

Boby · February 15, 2018, 10:44pm

Hi,
I have a synology diskstation at home and use their DNS service for my IP address.
When I try to install HTTPS reverse proxy in openHABian, I always get the following message:

“Sadly there was a problem setting up the selected option. Please report this │
│ problem in the openHAB community forum or as a openHABian GitHub issue.”

Here’s what I found in the console:

Installing DNS utilities...
Paketlisten werden gelesen...
Abhängigkeitsbaum wird aufgebaut....
Statusinformationen werden eingelesen....
dnsutils ist schon die neueste Version (1:9.10.3.dfsg.P4-12.3+deb9u4).
Das folgende Paket wurde automatisch installiert und wird nicht mehr benötigt:
  python-ndg-httpsclient
Verwenden Sie »sudo apt autoremove«, um es zu entfernen.
0 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Obtaining public IP address... <...>
Obtaining domain IP address... <...>
Public and domain IP address match
Installing NGINX...
Paketlisten werden gelesen...
Abhängigkeitsbaum wird aufgebaut....
Statusinformationen werden eingelesen....
nginx ist schon die neueste Version (1.10.3-1+deb9u1).
Das folgende Paket wurde automatisch installiert und wird nicht mehr benötigt:
  python-ndg-httpsclient
Verwenden Sie »sudo apt autoremove«, um es zu entfernen.
0 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Installing password utilities...
Paketlisten werden gelesen...
Abhängigkeitsbaum wird aufgebaut....
Statusinformationen werden eingelesen....
apache2-utils ist schon die neueste Version (2.4.25-3+deb9u3).
Das folgende Paket wurde automatisch installiert und wird nicht mehr benötigt:
  python-ndg-httpsclient
Verwenden Sie »sudo apt autoremove«, um es zu entfernen.
0 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Creating password file...
Adding password for user openhab
gpg: key 8B48AD6246925553: public key "Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 1
gpg:               imported: 1
OK
gpg: keyserver receive failed: Keine Daten
OK
Ign:1 http://repos.azulsystems.com/debian stable InRelease
OK:2 http://repos.azulsystems.com/debian stable Release
OK:3 http://mirrordirector.raspbian.org/raspbian stretch InRelease
OK:4 http://ftp.debian.org/debian jessie-backports InRelease
OK:5 http://archive.raspberrypi.org/debian stretch InRelease
OK:7 https://repos.influxdata.com/debian jessie InRelease
OK:8 https://deb.nodesource.com/node_7.x stretch InRelease
Ign:9 https://dl.bintray.com/fg2it/deb jessie InRelease
OK:10 https://dl.bintray.com/fg2it/deb jessie Release
Ign:6 https://openhab.jfrog.io/openhab/openhab-linuxpkg unstable InRelease
OK:12 https://openhab.jfrog.io/openhab/openhab-linuxpkg unstable Release
Paketlisten werden gelesen... Fertig
Abhängigkeitsbaum wird aufgebaut.
Statusinformationen werden eingelesen.... Fertig
Alle Pakete sind aktuell.
Installing certbot...
Paketlisten werden gelesen...
Abhängigkeitsbaum wird aufgebaut....
Statusinformationen werden eingelesen....
certbot ist schon die neueste Version (0.10.2-1).
Das folgende Paket wurde automatisch installiert und wird nicht mehr benötigt:
  python-ndg-httpsclient
Verwenden Sie »sudo apt autoremove«, um es zu entfernen.
0 aktualisiert, 0 neu installiert, 0 zu entfernen und 1 nicht aktualisiert.
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Creating Let's Encrypt certificate...
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Enter email address (used for urgent renewal and security notices) (Enter 'c' to
cancel):<...>

-------------------------------------------------------------------------------
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server at
https://acme-v01.api.letsencrypt.org/directory
-------------------------------------------------------------------------------
(A)gree/(C)ancel: A
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for <...>
Using the webroot path /var/www/gizmo.diskstation.me for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. <...> (http-01): urn:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://<...>/.well-known/acme-challenge/gtgXVodPPGk0_TNah-f5Zjal6jGKWl2OsdBXL9r9UnM: "<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Not Found</title>
</head>
<bo"

IMPORTANT NOTES:
 - If you lose your account credentials, you can recover through
   e-mails sent to robert.kroess@kabelnet.at.
 - The following errors were reported by the server:

   Domain: <...>
   Type:   unauthorized
   Detail: Invalid response from
   http://<...>/.well-known/acme-challenge/gtgXVodPPGk0_TNah-f5Zjal6jGKWl2OsdBXL9r9UnM:
   "<html>
   <head>
   <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
   <title>Error 404 Not Found</title>
   </head>
   <bo"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.
 - Your account credentials have been saved in your Certbot
   configuration directory at /etc/letsencrypt. You should make a
   secure backup of this folder now. This configuration directory will
   also contain certificates and private keys obtained by Certbot so
   making regular backups of this folder is ideal.
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

If I enter nothing (blank) for the domain name in the dialogue, OH2 is not reachable from outside via HTTPS.
What can I do to use the certificate which already exists for the domain (and which is maybe somewhere stored on my Synology)?

Thank you,
Boby

cgeo · February 15, 2018, 11:08pm

The fact that you have a certificate in synology does not mean that it will be used in your openhabian. You will need to either move it there or just get a new one from openhabian.

I am guessing you are trying the latter ? I guess from the error that let’s encrypt cannot reach your domain. What happens if you add a txt file in

http://yourdomain/.well-known/acme-challenge

and try to access it from the internet ?

ie http://yourdomain/.well-known/acme-challenge/test.txt

You should be able to see the contents of the txt file

Boby · February 16, 2018, 8:13pm

Hi @cgeo,
I tried now to leave the domain blank and then I got a “successful” during the setup.
Though, my “sites-enabled” file for openhab looks a bit strange for my opinion:

#################################
# openHABian NGINX Confiuration #
#################################

## Redirection
server {
   listen                          80;
   server_name                     localhost;
   return 301                      https://$server_name$request_uri;
}

## Reverse Proxy to openHAB
server {
#    listen                          80;
   listen                          443 ssl;
    server_name                     localhost;
   add_header                      Strict-Transport-Security "max-age=31536000; includeSubDomains";

## Secure Certificate Locations
   ssl_certificate                 /etc/ssl/certs/openhab.crt;
   ssl_certificate_key             /etc/ssl/certs/openhab.key;

    location / {
        proxy_pass                              http://localhost:8080/;
#        proxy_buffering                         off;  # openHAB supports non-buffering specifically for SSEs now
        proxy_set_header Host                   $http_host;
        proxy_set_header X-Real-IP              $remote_addr;
        proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto      $scheme;

## Password Protection
       auth_basic                              "Username and Password Required";
       auth_basic_user_file                    /etc/nginx/.htpasswd;
    }

## Let's Encrypt webroot location
#   location /.well-known/acme-challenge/ {
#       root                                    /var/www/localhost;
#   }
}

# vim: filetype=conf

“Strange” because:

proxy_pass: http://localhost:8080/;
Does that mean, that requests coming via https (443) will be forwarded to port 8080? Is that okay?
The location you mentioned is somehow commented out:

## Let's Encrypt webroot location
#   location /.well-known/acme-challenge/ {
#       root                                    /var/www/localhost;
#   }

Unfortunately, I have not really experience when it comes to SSL and reverse proxies.
I only want to access a) my NAS and b) openhab from outside via SSL under the same domain name (but different ports)

Where is my mistake?

Thanks,
Boby

cgeo · February 16, 2018, 8:55pm

I am not an expert on nginx but I see already some issues

Your server_name need to be your fqdn ie openhab.mydomain.com
When you use an ssl certificate it is issued for an fqdn so your reverse proxy need to match what the CN of the certificate is. Of course you also need to access your openhab using the same fqdn and not hostname / ip address etc

Next you have your certificates. Right now you point to an openhab.crt which I assume is the self signed certificate that is created automatically by openhab. This needs to be replaced by the location of the let’s encrypt certificates. .crt is the certificate and the .key is the private key (that should be kept absolutely secret!)

In the location you put the actual url of your openhab server ie its ip address. Now you just redirect to the http part which is not correct. You will need to use https ie https://ipaddressofopenhab:8443/

Here is my config in case it help

server {
        listen 80;
        server_name mydomain;
        return 301 https://$server_name$request_uri;
        }
server {
        listen 443 ssl;
        server_name mydomain;

        ssl_certificate /etc/letsencrypt/live/mydomain/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/mydomain/privkey.pem;
        add_header Strict-Transport-Security "max-age=31536000";
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        location / {
                proxy_pass https://ipaddressofopenhab:8443/;
                proxy_set_header Host $http_host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
                auth_basic "Username and Password Required";
                auth_basic_user_file /etc/nginx/.htpasswd;
                }
        location /.well-known/acme-challenge/ {
                root    /var/www/mydomain;
        }
}

Check also the documentation

Boby · February 16, 2018, 9:14pm

Thank you, still fighting…
Now I got this:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
An unexpected error occurred:
There were too many requests of a given type :: Error creating new authz :: too many failed authorizations recently: see https://letsencrypt.org/docs/rate-limits/
Please see the logfiles in /var/log/letsencrypt for more details.

I really start to hate that all…

cgeo · February 16, 2018, 9:26pm

This has nothing to do with your reverse proxy. This is your attempt to get the lets encrypt certificate. Which guide did you follow and what do you run to get this error ?

All these need patience It’s normal that the first time it will take time

you are hitting this, nothing to worry about

There is a Failed Validation limit of 5 failures per account, per hostname, per hour. This limit is higher on our staging environment, so you can use that environment to debug connectivity problems.

Boby · February 19, 2018, 8:36pm

Solved now…I uninstalled nginx completely and set everything manually up following this guide:
https://docs.openhab.org/v2.2/installation/security.html

Still don’t know why the openhabian setup failed, but since my problem is solved now, it’s not my problem anymore

gilhmauy · December 24, 2019, 9:40am

Hi @Boby, this covers exactly what i’m looking for - using an existing lets encrypt certificate obtained by my diskstation.

I’ve read the instructions about securing communication from the link you’ve posted - sounds good! But i have some more questions then:

Have you skipped the step Using Certbot to create/obtain the certificate in order to use the exported certificate files (cert.pem, chain.pem, privkey.pem) from your diskstation?
How do you handle the renewal of the certificate on your openHAB instance, do you renew the certificate manually each time it expires or do you’ve created some kind of cronjob to do that?

Merry christmas to everyone!