Attempting to setup the reverse proxy for external access to my openhabian instance. I attempted to accomplish this at first using openhabian-config but ran into issues with my ISP - namely that they cannot and will not open port 80 so even with the forwarding enabled on my router, the letsencrypt self-test was failing during the automated installation.
Once I figured this out, I followed the instructions for manual instllation found here Securing Communication and Access | openHAB
It appears things are mostly working now (using port 8443 since 443 is used for other things in my house) and I get a prompt for credentials when I attempt to access my installation externally. However, once provided, I’m presented with an plain white page stating “500 Internal Server Error nginx/1.18.0” and I get a similar message from the openhab app.
I only know just enough to be dangerous about this kind of thing and I’m not sure how to proceed with troubleshooting. Any suggestions?
Hi Keith,
A few questions for you.
Is your nginx running on same server as your openhab or are you running a standalone version of nginx on a separate device?
What does the niginx logs show when you repo the 500 error?
sudo tail -f /var/log/nginx/error.log
if I understood this part of your post correctly
you changed the server listen block to this for your external?
server {
listen 8443 ssl;
server_name mydomain_or_myip;
But when you changed it did you in advertently also change the proxy redirect as well to this?
proxy_pass http://what-ever_your_openhab_ip_or_host_name_ is:8443/;
or if you even changed it to https and your open hab ip and port to 8443.
if so change it back to port 8080 so your internal traffic from the reverse proxy to the open web server is running on http port 8080 and see if you still get the 500.
also if you have them both on same server you can not bind 2 different applications openhab and nginx to use the same port 8443 (openhab’s default SSL port is 8443) .
This is why I am asking if they coexist on same device
The big benefit to a reverse proxy is to take incoming external traffic from one port and ip (normally http traffic default port 80 or https traffic default port 443 and redirect it to a different port and ip usually a internal private ip and different non standard port, Oh and obviously the added functionality of having basic authentication to add as a enhancement to controlling access as well.
One other point it is possible to run your internal traffic from the nginx proxy to openhab as secure but you will need to add the self-signed cert that openhab generates during its first startup and setup You would need export it from your openhab and then import it into your cert store for nginx or even replace it with a new different cert and your own custom hostname that resolves in DNS for openhab but that is a whole different topic and set of steps to walk through.
Hope this helps and good luck.
Also make sure you take a hard look at all the recommendations for hardening anything you expose to the web the bad guys are always looking and port scanning!.
Okay, I did a bit of digging and realized my file permissions were all screwed up; couldn’t find something to show the correct permissions for the sites-enabled page so I cleaned it all away with an atp purge. Started over with a copy of my old nginx proxy definitions and now, in the openhab android app, I’m getting a) HTTP 401 Authentication failed and b) no traffic in the error log.
To better-explain my setup: I have port forwarding set up on my router from 8443 externally to my openhabian instance port 443. The proxy is set to listen on port 80 and redirect to 443, as described in the page mentioned above. Server 443 has proxy_pass set to localhost:8080.
I have set the remote server URL to my DDNS hostname (using https://[my domain]:8443) and use the username and password set up during the creation of my .htpasswd file.
Attempting to log in through a mobile browser gives me a loop of being asked for credentials where, sometimes, the interface presented looks very OpenHAB-esque but when I provide credentials, the loop starts over again. At least with the mobile browser, the error log updates with a password mismatch so I think that rules out any issues with my port forward and reverse proxy settings (but would be happy to be proven wrong).
Hi Keith,
I am still a bit unclear as to your set up The page you reference above discuses incoming traffic from external arriving via your external Ip to port 80 on your nginx
as this listen block shows
server {
listen 80;
server_name mydomain_or_myip;
return 301 https://$server_name$request_uri;
}
it then does a permenent redirect 301 to 443 as this listen block shows
server {
listen 443 ssl;
server_name mydomain_or_myip;
# Cross-Origin Resource Sharing.
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow_Credentials' 'true' always;
add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range' always;
add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS,PUT,DELETE,PATCH' always;
# openHAB 3 api authentication
add_header Set-Cookie X-OPENHAB-AUTH-HEADER=1;
ssl_certificate /etc/letsencrypt/live/mydomain/fullchain.pem; # or /etc/ssl/openhab.crt
ssl_certificate_key /etc/letsencrypt/live/mydomain/privkey.pem; # or /etc/ssl/openhab.key
add_header Strict-Transport-Security "max-age=31536000"; # Remove if using self-signed and are having trouble.
and a redirect to 443 and adds all the required headers
then the proxy forwards it to the openhab server on http://localhost:8080 (again this is assuming they are collocated on same device together.
as the listen block shows.
location / {
proxy_pass http://localhost:8080/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Authorization "";
satisfy any;
allow 192.168.0.0/24;
allow 127.0.0.1;
deny all;
auth_basic "Username and Password Required";
auth_basic_user_file /etc/nginx/.htpasswd;
}
so again are you colocating your openhab install and the nginx on the same device?
Also the default unless you have changed that for a openhab install is http port 8080 and for SSL https port 8443 and it creates a self signed certificate to support SSL communications
so the final target of proxy_pass should be the URL that you connect internally on you private network behind your router to your open hab install same as what you would put in a browser
Update: This line from the link you provides clearly states the assumptions.
The recommended configuration below assumes that you run the reverse proxy on the same machine as your openHAB runtime. If this doesn't fit for you, you just have to replace proxy_pass http://localhost:8080/ by your openHAB runtime hostname (such as http://youropenhabhostname:8080/).
Ok so I carefully reread your last response again and I think I have a clearer idea and better understanding of what you are saying.
so you are using your router to port forward external traffic arriving on your public IP on port 8443 to a internal private IP on port 443 skipping the permanent 301 redirect of port 80 to 443 when the traffic arrives at your nginx proxy.
you then use your nginx to forward the traffic to your openhab install on a private internal IP on port 8080. Is that correct?
So assuming a few things
lets simplify for a moment and take your router and your all the other stuff out of the mix.
lets say your nginx network interface is configured for 192.168.100.5
and your open hab is configured on 192.168.100.7
so if you opened a browser and put https://192.168.100.5 do you get your openhab main page?
How about if you put http://192.168.100.5 in the browser?
Do you get a error from your browser for a certificate issue?
like mismatched address?
If you do and you select ignore does it present your home page in openhab?
Also did you enable basic Authentication in openhab? I forgot to ask what version you are on but if it is 3.2 flavor then in settings API security click advanced and toggle the enable basic authentication .
for testing you can do everything internal and target your nginx listener and confirm it all works .
If you have a internal DNS server set up you can add and A record for your domain that points to your nginx and clear that mismatch error and use the FQDN . If you do not have a DNS server you can edit you host file on the PC you are using the browser to test with and confirm all of the settings and internal routing and reverse proxy is configured correctly.
Once you can hit the nginx listener from either of the configured ports internally you can add the router from the outside into the mix. You may also want to tick out the basic authentication part and just make sure all the routings and binding are working then add that functionality back into the mix. Then once you have that working try going through your router firewall and port forwarding and see if it works that way. There are some gotchas for SSL and header modifications that come into play so taking this one step at a time should help you see where it is failing.
Yes, they are both on the same hardware.
Correct
Since both nginx and OH are on the same machine, I’ll just use the same IP for this discussion.
If I put in https://192.168.1.11 in the browser, I get the apache landing page on my other server running something totally unrelated and the address in the bar changes to my dynamic DNS domain.
If I force http://192.168.1.11 in the browser, I get the same as above redirected to https.
If I instead use http://192.168.1.11:8080 I get my OH interface
If I use my dynamic DNS [hostname]:8443 I get my OH interface with no password or crdential challenge but there is a security risk warning for a self-signed security cert.
If I disable the port forwarding for port 443 in my router and navigate to 192.168.1.11, I then get my OH page.
Thank you for assisting me with this, Justin. This is at the bleeding edge of what I know about networking; I appreciate the help.
Humm if I understand you already have an Apache web server running on 192.168.1.11 interface and it already has a binding to port 443?
Then you say
If I force http://192.168.1.11 in the browser, I get the same as above redirected to https.
That is expected as your http to https redirect is taking action
When you say other server I assume you are referring to something like a Apache tomcat JVM and it is running on the same hardware that you have nginx and openhab running on is that correct?
So it has already set a listener and bound to that IP and port 443.
“If I instead use http://192.168.1.11:8080 I get my OH interface” normal behavior as always right.
“If I use my dynamic DNS [hostname]:8443 I get my OH interface with no password or credential challenge but there is a security risk warning for a self-signed security cert.”
And when you do this you are hitting the openhab binding that is bound to port 8443 and the cert warning you are seeing is the self signed one created when you first setup openhab so that is also expected.
How are you expecting for that incoming packet to know it is supposed to go to the nginx and not to the apache server? There are ways you could accomplish this but something like nginx has to be the traffic cop and handle the traffic direction to determine whether that packet is supposed to be a direct pass through to Apache on port 443 or to be rewritten to openhab on port 8080. It is possible but that is far from the very basic config that is provided in the sample config you currently referencing.
The challenge is you are already binding port 443 to the Apache instance so nginx is not going to be able to bind and listen on port 443 as well this goes back to what i said previously you can only bind one application(as in either nginx or Apache or openhab) to a specific port at a time on the same interface. if you are going to be locked into to only having 1 network interface. but if your other app on Apache can deal with port redirection it is possible.
instead you could put all inbound traffic to a different port on nginx say 4430 and to use a second server block listening that has a unique Host header for your Apache forward it to your 443 Apache and then a completely separate server block listening on port 4430 that is your openhab and is configured with unique host header for openhab and forwarding it to port 8080. Then Apache binding will only hear requests that arrive on port 4430 that have that have his Apache defined hostheader value. Openhab will only see traffic when its her hostheader name arriving on the port 4430 nginx interface.
So now currently any traffic that is arriving on interface 192.168.1.11 on port 443 is sent to your Apache server no matter what .
Think about it like you have 2 web sites that are running on the same web server. one website is Apache and you are reverse proxying all its matched filtered traffic to it and then you have a different filter for the Openhab site and you a reverse proxying (url rewriting ) its traffic to a different port (back end application).
Hopefully I am making sense?
Try not to let the Certificate resolving confuse you. it only comes into play later. However if you do use this approach you will need to have your lets encrypt cert configured to support wild cards then you can have something like https://myapache.[mydomain.com] and https://myopenhab.[mydomain.com] for your host header names. (the server_name) in each server block and obviously not the ‘https://’ part
You could test all of this without SSL on your internal subnet and use any of your ports for testing if you wanted to.
Of course there are many other ways to do this(like location push) but the complexity will likely be way more involved.
Update: you could create a duplicate config file with a different name in you site-enabled and all of your Apache info but the complexity will be the redirect from http to https the port 80 to 4430 may not behave as expected but you could try it if you wanted to have separate config files for each. I have not done it exactly that way myself in the past but it may work ok. (I am old school and usually just put it all in the same config file)
Followup Update:
So I did test 2 config files and using the wild card approach
and you could do it that way
assuming you wanted to use port 4430 as your traffic cop on your router port forward change it from forwarding to 192.168.1.11 port 443 to 192.168.1.11 port 4430
and lets say your domain name is keithscooldomain.selfip.net and you updated your lets encrypt cert to support a wildcard and you logged into your dyndns.org account (or whom ever is giving you external IP resolver) and enabled wild card support for it (I think dyndns.org still allows wild cards used to be a checkbox you would click mark where you updated your domain name that said enable wild cards for this domain (i.e such as www. )been a while since I used them and I heard they have made some changes).
you then would make a copy of your web config and give it a separate name but save it in same location as the one you have already then to make changes to 80 listen block on both files like below.
in the config to send traffic to your openhab you would have some like this.
listen 80;
server_name myopenhab.keithscooldomain.selfip.net;
return 301 https://$server_name:4430$request_uri;
}
now in the server listen block that used to be 443 you would change it to something like this.
server {
listen 4430 ssl;
server_name myopenhab.keithscooldomain.selfip.net;
ssl on;
error_page 497 =301 https://server_name:$server_port$request_uri;
All the rest would stay the same.
For your Apache config you would do the same like so
server {
listen 80;
server_name myapache.keithscooldomain.selfip.net;
return 301 https://$server_name:4430$request_uri;
}
server {
listen 4430 ssl;
server_name myapache.keithscooldomain.selfip.net;
ssl on;
error_page 497 =301 https://server_name:$server_port$request_uri;
but you would also change the proxy_ pass value
proxy_pass https://localhost:443/;
Then if you do not need the basic authentication for your apache you could REM out
# proxy_set_header Authorization "";
or if you do and need it but need a different set of creds you could create a different .htpasswd file and give it a different name and update the nginx config you made for apache only so it that points to the new one. You can also
# add_header Set-Cookie X-OPENHAB-AUTH-HEADER=1;
REM out Openhab 3 api auth that does set the cookies for Openhab 3 as well just in case they mess with what ever your Apache is doing…
check your change with nginx -t
restart nginx and test away
that should work ok
Hope it is clear enough
Should be pretty easy really.
Let me know how it goes.
Best of luck
Respectfully,
Justan Oldman
Okay, I think I’m beginning to understand this but I still don’t know how to troubleshoot my problem. To explain a little more - I have two servers running two different applications. One is the RPi running OpenHab, the other is running a dropbox-like service listening on port 443 (this is where the Apache message is coming from).
After reading through your descriptions, and considering what my ISP is restricting me to, it looks like I’ll have to specify the port incoming traffic to my local network goes to for OpenHab. I’d rather not fuss with changing the port for my dropbox-like thing, since support isn’t the best, and I’d like to stick with modifying traffic to the “server” running OH. Additionally, from my perspective, it appears the traffic is hitting the OH server, but the authentication is the problem now. I can set up port forwarding from 8443 (external) to 192.168.1.11:443 (internal) and I’m still getting the, what I’ll call, authentication loop.
It seems to me the problem lies with the authentication to NGIX (or perhaps earlier?) because of the authentication loop. If I watch the error log, using sudo tail -f /var/log/nginx/error.log there doesn’t seem to be any traffic during my log in attempts, until I use invalid credentials.
Wow… okay… It appears I was somehow missing the following lines in the port 443 listener of my sites-enabled file:
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Authorization "";
Added these in, restarted nginx, et voila! Remote connection now works.
I have no idea how I missed this (pretty certain I was using a copy and paste of the full example at the bottom of the example page) but it was enough to break this thing…
Thank you SO MUCH for the time and examples and research you put into this, @justaoldman - I’m really grateful for the time you volunteered to help troubleshoot such a silly oversight. I wish I had figured this out days ago but I’m grateful for the lessons learned.
Awesome glad you got it figured out and working!
enjoy!