Nginx reverse proxy & grafana - broken image (authorization problem)

Hi,
I try to change all my charts to grafana and are only partly successful.
To integrate a webview items works, but not on my smartphone -> know issue.

But I completely fail at integrating graphs as images! They’re alwas broken.
Here’s my configuration

Sitemap:

Image url="http://<external_dns_address>/render/dashboard-solo/db/temperaturen?orgId=1&panelId=3&width=1000&height=500&tz=UTC%2B01%3A00" refresh=60000

That’s like I copied it from the grafana interface (generated image)

When I open up my browser, I find the following address of the image:

http://<external_dns_address>/proxy?sitemap=home.sitemap&widgetId=030100&t=1517261870857

And the image is broken. When I try to display the address of the image (see above) in a separate browser window, then I get:
401 Authorization Required
nginx/1.10.3

In grafana, I have restricted the user signup and opened the anonymous access.
Still, I get this authorization problem.

Who can give me a hint?

Thanks,
Boby

If I understand you correctly, you are trying to add grafana graphs to your sitemap and then view them remotely.

You need to put the internal ip or dns of your grafana server in the sitemap image url.

BasicUI will go to that internal location to get the image and then serve it up. It basically is proxying.

Nginx then proxies the entire basicui site.

I have internal grafana graphs on my sitemap in this manner and they are visible via browser or ios mobile app either internal or external to my network.

@Moxified, thank you. That’s what I don’t get:
My sitemap looks that way:

Image url="http://192.168.1.50:3000/render/dashboard-solo/db/temperaturen?orgId=1&panelId=3&width=1000&height=500&tz=UTC%2B01%3A00" refresh=60000

And this is what I find on the website, when I use the external address to look at it:

http://<my_external_dns_name>/proxy?sitemap=home.sitemap&widgetId=030100&t=1517268470931

Why on earth is the correct link to the image being changed that way?
When I directly surf to the local address which I do use in the Image item, then the chart is being displayed wonderfully…

Oh ok, then you have it correct. I’m not sure why it doesn’t work then. It should change the url on the outside to have the external dns like that.

Obviously openhab doesn’t seem to have the right to access the rendered picture of grafana although I’ve set up everything as stated in the tutorial. Using the other URL in a webview works like a charm, but not on my mobile devices.

The question is: Where else can I check some settings to grant openhabian access to the rendered pictures in grafana, @ThomDietrich ?
Not sure if it’s a NGINX problem - or a “regular” rights problem in linux between all the installed pieces of software on my Raspberry…But I already fail in looking some logs because I have no idea where it could happen and where to check what…

I give up. Seems like I’m the only one having this problem.

I’m sorry Boby, I wish I could help but I don’t know what else to say. Mine works fine setup the way you have it.

Just to understand, are you trying to view your sitemap externally and the graph doesn’t show or are you trying to access the direct address to the widget that you posted?

What browser are you using?

If I log into my basicui remotely through nginx, it asks for my username and password, then displays my sitemap. This works fine. If I copy the link address for a graph and paste it in a new tab, the graph will open. If I paste this address into a different browser, it pops up a window asking for credentials. If I don’t put them, I get the 401 error.

1 Like

Thank you @Moxified, I know you can’t help me - seems like we’re the only two ones using nginx as reverse proxy :wink:
I normally do use Firefox as my primary browser, but the problem is consistent over various browsers.
The strange thing is: When I do use a WEBVIEW element, everything works (except on my iPhone, but the app is not able to process the grafana content as reported already). On my PC it works fine (Webview). When I do place the IMAGE element on the same sitemap, I only get a broken picture.

And there seems to be a problem, e.g in nginx or grafana configuration because if it’s a general problem with openhab, then also the webview woudln’t work…

To me this is odd. I’m not sure how nginx would cause this issue as it should blindly forward everything after you authenticate.

At this point, I would test bypassing nginx. This obviously requires you to expose your OH install directly to the internet but for a super short period while you actively monitor it shouldn’t be too risky. Just create a port forward to 8080 on your firewall, try accessing over 8080 from remote and see if it has the same behavior and then remove the forward.

You can certainly add source ip limitations to the firewall rules to help lock it down during this period of vulnerability.

There are many on here that use nginx. I would be surprised if we were two of 200 let alone of two that have this configuration. Chin up! These stupid problems are irritating but get resolved.

Good idea, tried to uninstall nginx and to access the way you described, but this messed up everything (not even able to access openhab from outside at all). So I ended up with nginx and the same old problem…

Now I’m completely puzzled…
I tried to display the generated image on the most simple html page thinkable:

<!DOCTYPE html>
<html>
<head>
        <meta http-equiv="Content-type" CONTENT="text/html; charset=utf-8">
        <link rel="stylesheet" type="text/css" href="/static/charts.css" />
</head>
<body>
        <img scr="http://192.168.1.50:3000/render/dashboard-solo/db/temperaturen?orgId=1&panelId=2&width=1000&height=500&tz=UTC%2B01%3A00/" height="500px" width="1000px" alt="Bild">
</body>    
</html>

And it’s NOT being displayed! Why this?
When I copy the url address into my browser, the picture is shown - but not within a html page on the same server???

No errors regarding this problem to be found in /var/log/nginx/error.log :frowning:

It kinda sounds like you just plain have some permissions issues going on with grafana.

As for the nginx test… I wouldn’t uninstall it, just point your external redirect (port forward, nat, whatever you are using) to use the default 8080 port of the OH dashboard.

What machine did you host that test web page on? You say same sever. Is your grafana a separate machine or on the same host as OH and Nginx?

Forwarding everything to 8080 on my local machine didn’t change the behaviour, so I switched back.
The uninstall was the last resort to exclude nginx problems, but as you said I also believe now, it’s a grafana permission issue.

It’s all on one “machine” - raspberry pi3 with openhabian, Influx, grafana and nginx activated.

And really: where the nginx log doesn’t show anything interesting, the grafana log holds some interesting messages for me:

t=2018-01-31T21:48:31+0100 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/render/dashboard-solo/db/temperaturen status=302 remote_addr=127.0.0.1 time_ms=5 size=29 referer=
t=2018-01-31T21:48:31+0100 lvl=eror msg="Anonymous access organization error: 'Main Org.': Organization not found"

Now I need to find out how to fix this problem in grafana…

Excellent find. On the right path now. Sounds like perhaps you have anonymous enabled for the wrong org or something. How many orgs do you have? I have only the one and it is listed under the anonymous config.

Well, I added the org defined in the interface (name: openHABian) in the grafana.ini now:

#################################### Anonymous Auth ##########################
[auth.anonymous]
# enable anonymous access
enabled = true

# specify organization name that should be used for unauthenticated users
;org_name = Main Org.
org_name = openHABian

# specify role for unauthenticated users
;org_role = Viewer

The error in the logfile is gone now; though I don’t get the rendered picture displayed, but only the ALT-text I’ve defined for the picture :frowning:

I guess I’ll remove and reinstall grafana again, maybe this will solve my issue…

Killed influxdb and grafana now from my systems since there’s no way to make it realibly working on all my devices. Influx also caused a lot of cpu load on my poor RPi3, so I better cling to the things which work flawless (=on-board charts, even if they are not as nice as grafana).

Thanks to everyone who tried to help me - but this problem stays unresolved.

I am having exactly the same problem. Grafana graph images render fine when the sitemap is accessed from my local network, but when accessed remotely I get a broken image.
As an experiment I opened up port 8080 and connected remotely. In this case the image rendered OK too - so from this the issue does seem to be related to Nginx.

The system is an Intel NUC running Ubuntu 18.04.

Here are some snippets of my configs:

sitemap:

 Image refresh=60000  url="http://localhost:3000/render/d-solo/mtUhoXUZz/heating?orgId=1&from=now-6h&to=now&panelId=2&width=800&height=400"

grafana.ini

[server]
root_url = %(protocol)s://%(domain)s:/grafana/

[auth.anonymous]
enabled = true

nginx config

server {
   listen                          80;
   server_name                    <redacted>;
   return 301                      https://$server_name$request_uri;
}

## Reverse Proxy to openHAB
server {
   listen                          443 ssl;
   server_name                     <redacted>;

    # Cross-Origin Resource Sharing.
   add_header 'Access-Control-Allow-Origin' 'http://localhost:8080/rest';
   add_header 'Access-Control-Allow_Credentials' 'true';
   add_header 'Access-Control-Allow-Headers' 'Authorization,Accept,Origin,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
   add_header 'Access-Control-Allow-Methods' 'GET,POST,OPTIONS,PUT,DELETE,PATCH';

## Secure Certificate Locations
   ssl_certificate                 /etc/letsencrypt/live<redacted>/fullchain.pem;
   ssl_certificate_key             /etc/letsencrypt/live/<redacted>/privkey.pem;
   add_header                      Strict-Transport-Security "max-age=31536000; includeSubDomains";

#   ssl_client_certificate              /etc/nginx/Client-CA.pem;       # trusted CA used to issue client certs
#   ssl_verify_client                   on;
#
# Password Protection
    auth_basic                              "Username and Password Required";
    auth_basic_user_file                    /etc/nginx/.htpasswd;

    satisfy     any;
    allow       192.168.1.0/24;
    allow       127.0.0.1;
    deny        all;

    location / {
        proxy_pass                              http://localhost:8080/;
#        proxy_buffering                         off;  # openHAB supports non-buffering specifically for SSEs now
        proxy_set_header Host                   $http_host;
        proxy_set_header X-Real-IP              $remote_addr;
        proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto      $scheme;

    }

    location /grafana/  {
        proxy_pass                              http://localhost:3000/;
        proxy_set_header Authorization "";      # stop nginx forwarding the basic auth header for nginx .httpasswd to grafana
    }

    location /frontail/ {
        proxy_pass                              http://localhost:9001;
        proxy_set_header Host                   $http_host;
        proxy_set_header X-Real-IP              $remote_addr;
        proxy_set_header X-Forwarded-For        $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto      $scheme;
    }
## Let's Encrypt webroot location
   location /.well-known/acme-challenge/ {
       root                                    /var/www/<redacted>;
   }
}

If I right-click on the broken image and select ‘View image’ I see this:

image

image

The JSON ‘Invalid username or password’ is from Grafana as I see this in my Grafana log:

t=2020-02-12T11:54:01+0000 lvl=eror msg="Invalid username or password" logger=context error="Invalid Username or Password"
t=2020-02-12T11:54:01+0000 lvl=info msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/render/d-solo/mtUhoXUZz/heating status=401 remote_addr=<redacted> time_ms=69 size=42 referer=https://<redacted>.org/basicui/app

So it appears to be a Grafana authentication issue caused when the sitemap is served via Nginx. Any thoughts on steps to troubleshoot?