Grafana and myopenhab.com - FINAL solution

create1st · February 8, 2019, 7:53pm

I hope I will not rediscover a wheel here but obviously all the very nice tutorials about integrating InfluxDB and Grafana do miss one final step - integration with myopenhab.com
They do mention how to configure habpanel to use Grafana link. This means that openhabian:3000 has to be exposed. This is a solution for a home network, not for an Internet. Especially when you do not want to forward port 3000 behind your router and use the service like no-ip.com to update your dynamic IP. When you have an overprotective internet provider, you are not even allowed to do that as all incoming connections are going to be filtered anyway.
So, the desired solution is to have a png served by your OpenHAB and not Grafana, so it can be nicely presented on habpanel and seamlessly exposed to myopenhab.
While I was checking the community resources I’ve found that all proposal for solutions of similar problems are not a perfect fit.

The ideal solution Have an Image item and using Http binding communicate with Grafana. Problem: It requires Base64 image, not a binary data. Ideal solution convert incoming binary to Base64 using JavaScript transformation. This does not work, at least I was not able to solve it. The problem is that JavaScript does get a string, not a byte array and conversion of that string to byte array does not give initial set of bytes which are on the wire. Maybe there is a way to do that, but it is hidden somewhere in the dark side of Internet. Someone is willing to implement Java, Groovy transformation?
@bloodboy Usage of Image Item. hack. This one is not so bad but… opening yet another port just to have FTPUpload binding working as a gate for an Image item doesn’t sound too convincing.
@rlkoshak Comprehensive Wunderground using HTTP Binding Example This is a great tutorial, but the proposal how to serve dynamic images is, at least to my taste, a way to abusive with the respect to violating the concept of serving static html resources. Downloading an image with wget is not so bad idea but storing them somewhere on the drive and then refreshing the screen with simple page refresh without nice possibility to even detect if an image has changed is not appealing at all. Yes, there is a hack for that in the tutorial as well.
The wget + Image solution Now is what I have invented. It is simple and it is based on Image item. All nice Angular page refreshment is happening in the background and of course we do have checking if we do have the same image or we do have something which is new and should be rendered by the browser.
a) Items Create Image item with nice name and define a channel

Image grafana_temperature_measure_salon_image "Salon" (grafana) {channel="ftpupload:imagereceiver:grafana_measure_salon:image"}

b) Rules Use cron (here every 15min and fetch tih image from Grafana using image share

Encode binary content of the file with Base64 and send it to the Image item. Downside of that is that you don’t have an Image content set for first 15min after restart but who cares? You can work this out with some rule which is intercepting OpenHAB startup or just accept this limitation.

import org.eclipse.xtext.xbase.lib.Functions
import java.nio.file.FileSystems;
import java.nio.file.Files;

val logName = "grafana.rules"
val Functions$Function1<String, String> getSnapshotImage = [url |
	val fileType = "png"; // url.substring(url.lastIndexOf('.') + 1);
	val tmpFile = "/tmp/grafana_snapshot." + fileType
    executeCommandLine("wget -O " + tmpFile + " \"" + url + "\"", 10000)
	val path = FileSystems.getDefault().getPath(tmpFile);
    val fileData = Files.readAllBytes(path);
	"data:image/" + fileType + ";base64," + java.util.Base64.encoder.encodeToString(fileData)
]

rule "Update Grafana snapshots"
when
    Time cron "0 0/15 * * * ? *" 
then
	logInfo(logName, "Refreshing Grafana")
	val imageSnapshotSalon = getSnapshotImage.apply("http://onepi:3000/render/d-solo/TsnclLmRz/salon?orgId=1&panelId=2&width=1000&height=500&tz=UTC%2B01%3A00")
    grafana_temperature_measure_salon_image.postUpdate(imageSnapshotSalon)
	logInfo(logName, "Refreshing Grafana completed")
end

c) Habpanel Create an image in habpanel and set Image item as a source of image

d) Enjoy the Grafana image on myopenhab.com with your mobile device while you are 1000km away from home.

Cheers,
Sebastian

rlkoshak · February 8, 2019, 8:14pm

Unlikely but you might be able to make it work with a Proxy Item and a Rule. Use the HTTP to populate a String Item then use the String Item to trigger a Rule that converts the String to bytes and then to Base64. Then update the Image Item with the Base64 String.

This approach was never really meant to apply to this sort of use case anyway and I’m not totally convinced it would work with Grafana anyway. I left it a little loose because Wunderground was known to change up their icon sets periodically (e.g. they’s use one set of icons for daytime and another set at night time but the actual weather conditions are the same).

This import is not necessary.

You should use the cleaner lambda definition syntax:

val getSnapshotImage = [ String url |

Does this solution depend on PhantomJS and Grafana’s static image URL? If so it should be noted that new versions of Grafana no longer come with PhantomJS and if you try to restore this library (lots of tutorials on the web) you might encounter significant performance problems with Grafana consuming a whole lot of resources rendering the charts. From what I’ve seen reported, an RPi can handle maybe one or two simultaneous renderings. Any more and it will cause your OH instance to be killed as the kernel desperately searches for more resources to give to PhantomJS.

Thanks for posting! It’s a great tutorial and I’m positive many will find it very useful. Particularly the loading and conversion of the image to Base64.

create1st · February 8, 2019, 8:38pm

Thanks for the code review I’m not familiar with this syntax and therefore the code is more plain Java style. No, the code does not depends on PhantomJS at least to my knowledge. I have a fresh installation of everything and I was not installing any addons. I’m using Grafana v5.1.4

rlkoshak · February 8, 2019, 8:46pm

The cleaner lambda syntax is the old way of doing it that became broken at some point (probably around OH 2.0 release) and then was fixed and usable again around OH 2.3 release. So you will see lots of examples that use the overly verbose and ugly Functions$Function1<Type, Type… syntax because for a little over a year the cleaner syntax didn’t work.

OK, so I think you are using the version just before they removed PhantomJS. On Grafana 5.2.4 I think and definitely by the latest version 5.4.3 the solo links will no longer work unless you add PhantomJS back to your installation. And if you do that then you will see the performance issues already mentioned.

create1st · February 8, 2019, 8:53pm

Ahh. I forgot to mention. I’m on RPI https://github.com/fg2it/grafana-on-raspberry

darkcheater · March 6, 2019, 8:55am

Thank you for this solution.
I am however not using the PNG image, I use a webview in my sitemap to have an interactive chart where I can do mouseovers or zooming. It of course only works locally because I don’t want to forward port 3000 to the internet.
Is there a similar solution to display the webview over myopenhab.org in the sitemap? For example a copy of the html file that is updated every 5 minutes?

create1st · March 6, 2019, 8:10pm

This is exactly a workaround to have the graph exposed over the internet. I have pretty much similar solution for camera view. The point is, myopenhab.org does not provide interaction routing functionality. It does provide the view and this is your limitation. If you do not want to expose the port over services like serveo.com or similar then image is your only option. To be frank, I was pretty much curious about the possibility of making the screen shots instead of using PhantomJS plugin from Grafana (check the comment from @rlkoshak) but I was completely unsuccessful with available command line tools.

noppes123 · September 29, 2019, 7:31am

Hi @create1st,

First, thanks for this nice tutorial.

Unfortunately, I am not successful in exposing it to openHAB cloud (running my own instance). I can access HABpanel via my openHAB cloud instance but the image is not shown (only the default image icon) - also not when accessing HABPanel locally. Tried different browsers as well. My setup (for testing I stayed as close to your original example as possible):

testgrafana.items

Image imgTempIndoor24H "Indoor Temperature" {channel="ftpupload:imagereceiver:grf_temp_indoor:image"}

testgrafana.rules

import org.eclipse.xtext.xbase.lib.Functions
import java.nio.file.FileSystems;
import java.nio.file.Files;

val logName = "grafana.rules"

val Functions$Function1<String, String> getSnapshotImage = [url |
	val fileType = "png"; // url.substring(url.lastIndexOf('.') + 1);
	val tmpFile = "/tmp/grafana_snapshot." + fileType
    executeCommandLine("wget -O " + tmpFile + " \"" + url + "\"", 10000)
	val path = FileSystems.getDefault().getPath(tmpFile);
    val fileData = Files.readAllBytes(path);
	"data:image/" + fileType + ";base64," + java.util.Base64.encoder.encodeToString(fileData)
]

rule "Update Grafana snapshots"
when
    Time cron "0 0/10 * * * ? *" 
then
	logInfo(logName, "Refreshing Grafana Indoor Temp Graph")
	val imageSnapshot = getSnapshotImage.apply("http://192.168.2.2:3000/d/DRmxtepWz/openhab2?orgId=1&fullscreen&panelId=2&from=now-24h&to=now")
    imgTempIndoor24H.postUpdate(imageSnapshot)
	logInfo(logName, "Refreshing Grafana Indoor Temp Graph completed")
end

The image item gets created:

openhab> smarthome:items list | grep img
imgTempIndoor24H (Type=ImageItem, State=raw type (image/png): 27087 bytes, Label=Indoor temp last 24H, Category=null)

And in HABPanel, the image is added like this (never mind the typo…):

The image is shown as the default image placeholder icon:

The image link in the browser for this image is: data:image/png;base64,PCFET0NUWVBFIGh0bWw+CjxodG1sIG......

What am I missing?

EDIT: From the official openHAB HABPanel documentation:
“The image widget can display an image, directly or via an openHAB String item, and can refresh it at regular intervals.”. No mention of the Image Item type…?

EDIT2: Probably not related, but I did notice an error in the browser console:

  ...
Service worker registered
openhab.service.js:35 Loaded 327 openHAB items
manifest.json:1 GET https://openhab.domain.ext/habpanel/manifest.json 401
manifest.json:1 Manifest: Line: 1, column: 1, Syntax error.

noppes123 · October 6, 2019, 7:05am

This turned out to be a ‘user error’: the URL to the Grafana image was wrong resulting in an HTML file to be stored as the PNG. The rest is history…

Next step: generalise this so I can support any number of graphs with a few simple rules.

Arpagor · February 28, 2020, 4:50pm

I encounter a problem during the wget I recover an html type file !!! and not a picture
I have a doubt about the URL to pass: I took the url of the direct link which is such as:

http://192.168.0.27:3500/render/d-solo/DN5lzAkgk/chauffage-general?panelId=2&orgId=1&width=1000&height=500&tz=UTC%2B01%3A00

this url looks correct in chrome:

in manual mode I get this:

wget -O /tmp/grafana_snapshot.png "http://192.168.0.27:3500/render/d-solo/DN5lzAkgk/chauffage-general?from=1582796828399&to=1582883228399&orgId=1&panelId=2&width=1000&height=500&tz=UTC%2B01%3A00"
--2020-02-28 17:36:46--  http://192.168.0.27:3500/render/d-solo/DN5lzAkgk/chauffage-general?from=1582796828399&to=1582883228399&orgId=1&panelId=2&width=1000&height=500&tz=UTC%2B01%3A00
Connexion à 192.168.0.27:3500… connecté.
requête HTTP transmise, en attente de la réponse… 302 Found
Emplacement : /login [suivant]
--2020-02-28 17:36:46--  http://192.168.0.27:3500/login
Réutilisation de la connexion existante à 192.168.0.27:3500.
requête HTTP transmise, en attente de la réponse… 200 OK
Taille : non indiqué [text/html]
Sauvegarde en : « /tmp/grafana_snapshot.png »

/tmp/grafana_snapsh     [ <=>                ]  10,27K  --.-KB/s    ds 0s      

2020-02-28 17:36:46 (49,6 MB/s) - « /tmp/grafana_snapshot.png » sauvegardé [10512]

and when I open the image file /tmp/grafana_sanpshot.png

<!DOCTYPE html>
<html lang="en">

<head>
  <meta charset="utf-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
  <meta name="viewport" content="width=device-width">
  <meta name="theme-color" content="#000">

  <title>Grafana</title>

  <base href="/" />

  <link rel="stylesheet" href="public/build/grafana.dark.css?v5.1.4">

  <link rel="icon" type="image/png" href="public/img/fav32.png">
  <link rel="mask-icon" href="public/img/grafana_mask_icon.svg" color="#F05A28">
  <link rel="apple-touch-icon" href="public/img/fav32.png">

</head>

<body ng-cloak class="theme-dark">
  <grafana-app class="grafana-app">

    <sidemenu class="sidemenu"></sidemenu>

    <div class="page-alert-list">
      <div ng-repeat='alert in dashAlerts.list' class="alert-{{alert.severity}} alert">
        <div class="alert-icon">
          <i class="{{alert.icon}}"></i>
        </div>
        <div class="alert-body">
          <div class="alert-title">{{alert.title}}</div>
          <div class="alert-text" ng-bind='alert.text'></div>
        </div>
        <button type="button" class="alert-close" ng-click="dashAlerts.clear(alert)">
          <i class="fa fa fa-remove"></i>
        </button>
      </div>
    </div>

    <div class="main-view">
      <div class="scroll-canvas" page-scrollbar>
        <div ng-view></div>

        <footer class="footer">
          <div class="text-center">
            <ul>
              <li>
                <a href="http://docs.grafana.org" target="_blank">
                  <i class="fa fa-file-code-o"></i>
                  Docs
                </a>
              </li>
              <li>
                <a href="https://grafana.com/services/support" target="_blank">
                  <i class="fa fa-support"></i>
                  Support Plans
                </a>
              </li>
              <li>
                <a href="https://community.grafana.com/" target="_blank">
                  <i class="fa fa-comments-o"></i>
                  Community
                </a>
              </li>
              <li>
                <a href="https://grafana.com" target="_blank">Grafana</a>
                <span>v5.1.4 (commit: a5fe24f)</span>
              </li>
              
              <li>
                <a href="https://grafana.com/get" target="_blank" bs-tooltip="'6.6.2'">
                  New version available!
                </a>
              </li>
              
            </ul>
          </div>
        </footer>
      </div>
    </div>
  </grafana-app>

  <script>
    window.grafanaBootData = {
      user: {"isSignedIn":false,"id":0,"login":"","email":"","name":"","lightTheme":false,"orgCount":0,"orgId":0,"orgName":"","orgRole":"","isGrafanaAdmin":false,"gravatarUrl":"","timezone":"browser","locale":"en-US","helpFlags1":0},
      settings: {"alertingEnabled":true,"allowOrgCreate":false,"appSubUrl":"","authProxyEnabled":false,"buildInfo":{"buildstamp":1529405697,"commit":"a5fe24f","env":"production","hasUpdate":true,"latestVersion":"6.6.2","version":"5.1.4"},"datasources":{"-- Grafana --":{"meta":{"type":"datasource","name":"-- Grafana --","id":"grafana","info":{"author":{"name":"","url":""},"description":"","links":null,"logos":{"small":"public/img/icn-datasource.svg","large":"public/img/icn-datasource.svg"},"screenshots":null,"version":"","updated":""},"dependencies":{"grafanaVersion":"*","plugins":[]},"includes":null,"module":"app/plugins/datasource/grafana/module","baseUrl":"public/app/plugins/datasource/grafana","annotations":true,"metrics":true,"alerting":false,"builtIn":true,"routes":null},"name":"-- Grafana --","type":"datasource"},"-- Mixed --":{"meta":{"type":"datasource","name":"-- Mixed --","id":"mixed","info":{"author":{"name":"","url":""},"description":"","links":null,"logos":{"small":"public/img/icn-datasource.svg","large":"public/img/icn-datasource.svg"},"screenshots":null,"version":"","updated":""},"dependencies":{"grafanaVersion":"*","plugins":[]},"includes":null,"module":"app/plugins/datasource/mixed/module","baseUrl":"public/app/plugins/datasource/mixed","annotations":false,"metrics":true,"alerting":false,"queryOptions":{"minInterval":true},"builtIn":true,"mixed":true,"routes":null},"name":"-- Mixed --","type":"datasource"}},"defaultDatasource":"-- Grafana --","disableLoginForm":false,"disableUserSignUp":true,"externalUserMngInfo":"","externalUserMngLinkName":"","externalUserMngLinkUrl":"","googleAnalyticsId":"","ldapEnabled":false,"loginHint":"email or username","oauth":{},"panels":{"alertlist":{"baseUrl":"public/app/plugins/panel/alertlist","hideFromList":false,"id":"alertlist","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Shows list of alerts and their current status","links":null,"logos":{"small":"public/app/plugins/panel/alertlist/img/icn-singlestat-panel.svg","large":"public/app/plugins/panel/alertlist/img/icn-singlestat-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/alertlist/module","name":"Alert List","sort":6},"dashlist":{"baseUrl":"public/app/plugins/panel/dashlist","hideFromList":false,"id":"dashlist","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"List of dynamic links to other dashboards","links":null,"logos":{"small":"public/app/plugins/panel/dashlist/img/icn-dashlist-panel.svg","large":"public/app/plugins/panel/dashlist/img/icn-dashlist-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/dashlist/module","name":"Dashboard list","sort":7},"gettingstarted":{"baseUrl":"public/app/plugins/panel/gettingstarted","hideFromList":true,"id":"gettingstarted","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"","links":null,"logos":{"small":"public/app/plugins/panel/gettingstarted/img/icn-dashlist-panel.svg","large":"public/app/plugins/panel/gettingstarted/img/icn-dashlist-panel.svg"},"screenshots":null,"version":"","updated":""},"module":"app/plugins/panel/gettingstarted/module","name":"Getting Started","sort":100},"graph":{"baseUrl":"public/app/plugins/panel/graph","hideFromList":false,"id":"graph","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Graph Panel for Grafana","links":null,"logos":{"small":"public/app/plugins/panel/graph/img/icn-graph-panel.svg","large":"public/app/plugins/panel/graph/img/icn-graph-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/graph/module","name":"Graph","sort":1},"heatmap":{"baseUrl":"public/app/plugins/panel/heatmap","hideFromList":false,"id":"heatmap","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Heatmap Panel for Grafana","links":[{"name":"Brendan Gregg - Heatmaps","url":"http://www.brendangregg.com/heatmaps.html"},{"name":"Brendan Gregg - Latency Heatmaps","url":" http://www.brendangregg.com/HeatMaps/latency.html"}],"logos":{"small":"public/app/plugins/panel/heatmap/img/icn-heatmap-panel.svg","large":"public/app/plugins/panel/heatmap/img/icn-heatmap-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/heatmap/module","name":"Heatmap","sort":5},"pluginlist":{"baseUrl":"public/app/plugins/panel/pluginlist","hideFromList":false,"id":"pluginlist","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Plugin List for Grafana","links":null,"logos":{"small":"public/app/plugins/panel/pluginlist/img/icn-dashlist-panel.svg","large":"public/app/plugins/panel/pluginlist/img/icn-dashlist-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/pluginlist/module","name":"Plugin list","sort":100},"singlestat":{"baseUrl":"public/app/plugins/panel/singlestat","hideFromList":false,"id":"singlestat","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Singlestat Panel for Grafana","links":null,"logos":{"small":"public/app/plugins/panel/singlestat/img/icn-singlestat-panel.svg","large":"public/app/plugins/panel/singlestat/img/icn-singlestat-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/singlestat/module","name":"Singlestat","sort":2},"table":{"baseUrl":"public/app/plugins/panel/table","hideFromList":false,"id":"table","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"Table Panel for Grafana","links":null,"logos":{"small":"public/app/plugins/panel/table/img/icn-table-panel.svg","large":"public/app/plugins/panel/table/img/icn-table-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/table/module","name":"Table","sort":3},"text":{"baseUrl":"public/app/plugins/panel/text","hideFromList":false,"id":"text","info":{"author":{"name":"Grafana Project","url":"https://grafana.com"},"description":"","links":null,"logos":{"small":"public/app/plugins/panel/text/img/icn-text-panel.svg","large":"public/app/plugins/panel/text/img/icn-text-panel.svg"},"screenshots":null,"version":"5.0.0","updated":""},"module":"app/plugins/panel/text/module","name":"Text","sort":4}}},
      navTree: [{"id":"dashboards","text":"Dashboards","subTitle":"Manage dashboards \u0026 folders","icon":"gicon gicon-dashboard","url":"/","children":[{"id":"home","text":"Home","icon":"gicon gicon-home","url":"/","hideFromTabs":true},{"id":"divider","text":"Divider","divider":true,"hideFromTabs":true},{"id":"manage-dashboards","text":"Manage","icon":"gicon gicon-manage","url":"/dashboards"},{"id":"playlists","text":"Playlists","icon":"gicon gicon-playlists","url":"/playlists"},{"id":"snapshots","text":"Snapshots","icon":"gicon gicon-snapshots","url":"/dashboard/snapshots"}]},{"id":"help","text":"Help","subTitle":"Grafana v5.1.4 (a5fe24f)","icon":"gicon gicon-question","url":"#","hideFromMenu":true,"children":[{"text":"Keyboard shortcuts","icon":"fa fa-fw fa-keyboard-o","url":"/shortcuts","target":"_self"},{"text":"Community site","icon":"fa fa-fw fa-comment","url":"http://community.grafana.com","target":"_blank"},{"text":"Documentation","icon":"fa fa-fw fa-file","url":"http://docs.grafana.org","target":"_blank"}]}]
    };
  </script> 

<script type="text/javascript" src="public/build/manifest.08803d414d2b7493e032.js"></script><script type="text/javascript" src="public/build/vendor.fea086194b7378a0fa2e.js"></script><script type="text/javascript" src="public/build/app.5372b9bbcb8c6d1c027e.js"></script></body>

</html>

can you tell me what’s wrong?
Thak’s

noppes123 · February 28, 2020, 5:10pm

It redirects to login and seems to succeed in some way (not sure which account is used…). Try first with disable authentication.

Arpagor · March 2, 2020, 9:24am

I have removed the authentication and now I no longer have the Http error message but the image displayed in Habpanel !!!
Thank’s
Now I have to find out why I can’t use authentication !!

noppes123 · March 3, 2020, 8:54am

There is always one more issue to solve…

I’m a bit busy ATM, but will try a small test setup soon to see if I can reproduce and find a root cause.
My setup involves HABpanel and a private openhabcloud setup, so it is somewhat similar, but not exactly…

marco_hoefle · February 23, 2021, 5:39am

Hi Sebastian,
are you still using Grafana or do you use the charts from OpenHAB3.x ?
They look much better now.
I am currently upgrading my installation and I need to make the decision what to use…
In my opinion the Grafana Charts are still a bit advanced.

My current setup was:
Apache Virtual Host for Openhab with LDAP authentication managed by Apache.
Apache Virtual Host for Grafana, LDAP authentication managed by grafana.

However, I didn’t manage to use the authentication token from Apache to get the Grafana data (same account, user/passwd is the same for OpenHAB and Apache).
The very ugly solution was to expose all graphs without authentication.