InfluxDB+Grafana persistence and graphing

The repo for armv6 is now up ( https://dl.bintray.com/fg2it/deb-rpi-1b ) and populated with v4.0.1. The PR is done.

1 Like

I’ve created a JavaScript library that uses the openHAB 2 REST API and server side events (SSE) to resolve items values to Grafana panel parameters. I use it to reduce the number of Webviews in my sitemap. Normally a lot of Webviews are required for user customizable parameters, i.e. using visibility[..] expressions.

openHAB always loads these Webviews, eventhough they are not visible. As a result page load times suffered, especially in combination with my RPi3 and smartphone.

The code including documentation and an example demo is available at:

The following sitemap snippet shows how this library makes it possible to customize all 5 Grafana panel parameters with a single Webview:

Frame label="Combo 2" {
	Switch item=Combo2_Panel mappings=[HUMIDITY="Humidity", TEMPERATURE="Temperature"]
	Switch item=Combo2_Dashboard mappings=[alpha="Alpha", beta="Beta", gamma="Gamma", delta="Delta"]
	Selection item=Combo2_From mappings=[HOUR="now-1h", DAY="now-1d", WEEK="now-7d", MONTH="now-30d", YEAR="now-1y"]
	Selection item=Combo2_To mappings=[NOW="now", HOUR="now-1h", DAY="now-1d", WEEK="now-7d", MONTH="now-30d", YEAR="now-1y"]
	Switch item=Combo2_Theme mappings=[light="Light", dark="Dark"]
	Webview url="/static/demo-combo2.html?dashboardItem=Combo2_Dashboard&fromItem=Combo2_From&toItem=Combo2_To&panelItem=Combo2_Panel&themeItem=Combo2_Theme" height=9
}

So far I’ve tested it with the Basic UI on Chrome (Desktop, Mobile) and Firefox (Desktop). A lot of the library can probably also be reused when you want to use the REST API and SSE in combination with a non-Grafana HTML Webview or web page.

14 Likes

@wborn Very very nice. I’ll have a play later this week, but the examples on your git page look great.

1 Like

Wonderful posting. I’d like to recommend either @ThomDietrich incorporate this into the top tutorial or re-post this as its own independent thread.

I vote for a separate thread because I can see this approach being applicable to other types of webviews.

1 Like

Hi there,
first, thanks @ThomDietrich for thatb great tutorial !
over the last to days i tried several options to get my data to influxdb.
I akm using openHAB2b4, and alos updated tothe latest influx bundle
238 | Active | 80 | 1.9.0.201611012307 | openHAB InfluxDB Persistence bundle

My Issue: No data pulled to InfluxDB.
After a bit o testing, i found that my influxdb.persist is the issue.
It works well when i use

  • : strategy = everyChange, every5Min
    however, this is a bad strategy, as it might goe crazy for bindings like hue ( imagine using a colorheel on 20 Bulbs, that is a lot of changes :slight_smile: )
    while trying with item* or group*, i found none of them are working.
    even group does not work.
    the only syntax that worked is item1,item2,item3:strattegy = [strategies]

i have not tried other persistence bindings so far, but i will test immediately with myopenhab binding to see if the issue is there as well

the easiest way to reproduce is creating a group for Systeminfo Binding and try to get every change strategy on that group … without having *

not sure if that is because it is a influx persistence is a 1.9 Bundle rater than a 2.0

Note that the “*” is not a wild card. It means “all members of this group.” So it makes no sense to use a * with an Item. The only valid way to use it is:

FullNameOfGroup*: strategy = [strategies]

I believe there are other issues with myopenhab right that that makes this a really bad choice to test the behavior.

See the following for an example of using Groups with persistence.

thanks ! that worked for me ! tested various things including tabs before :slight_smile:

Clutch, man. Thanks for your efforts. The graphs look GREAT!

1 Like

Hey @ThomDietrich how can the points be connected? (Value “Ping”)

Hey, you need to select Draw Mode Lines. Your diagram uses Points.

1 Like

@theo, @fremel75, @opus, @FutureCow, @TimO,

I finally got a chance to test this out and the latest build does indeed fix the problem. My queries work again for those of us still on the “default” RP.

Thanks!

1 Like

This is a great, great tutorial. I am creating now several graphs with my data :slight_smile:

2 Likes

I’ve added support for rendering Grafana panels to my Grafana JavaScript libary. The exact width and height of the rendered panels can be added to the Webview url. The library can also automatically compute them based on the available display area. A refresh interval can be used to automatically refresh a rendered panel. :slight_smile:

So a rendered panel with auto width, fixed height and 5s refresh interval would look like this:

Webview url="/static/demo.html?dashboard=wifireception&from=now-1w&to=now&panel=10&width=auto&height=300&render=true&refresh=5000" height=9

People who already used the previous library version may need to update the urlPrefix in smarthome-grafana-user-defaults.js by removing /dashboard-solo/db/ from it. After updating, you can immediately check what rendered panels would look like by adding &render=true to your browser URL. Clicking on a rendered panel will make it interactive.

The examples were also recently improved with some help of @3DJupp!

3 Likes

Hey Wouter,
that’s great work you did there! I didn’t have the time to test your library myself but I’ve just added the reference to your work in the first paragraph. Please let me know of any corrections or additions you would like me to include!

2 Likes

Thanks Thom, the text looks great! It will certainly save people some time when there is a link to the library in the opening post. :slight_smile:

After having this working just great for some months, I now see a problem, which must have come with some update of something some week ago…

The issus can be seen on this picture - the green line (LV VP phase 1) takes shortcuts where it should not. So for example, today at 10:00 the line should be at 0W, but it never goes down to zero, instead jumps to the next update at 11:00 - 500W approx.

In influx, I can see that it has recorded 0W at 09:34

In influx, I can see that it has recorded 0W at 09:34, followed by 372 @ 11:04 (all local time);

1483173238610000000     673.077
1483173251615000000     390.769
1483173264611000000     0
1483178648507000000     372.462
1483178661511000000     641.769
1483178674508000000     575.231

Why is this happening? And why not always, e.g. LVVP Phase 2 also has this here and there.

I have something in the back of my head about an issue with “0” values, but I can’t find it, and maybe this was not related to grafana/influxdb at all. i.e. a dream. :slight_smile:

Any suggestions welcome!

(LV VP is a three phase heat pump)

Hey!
Tricky situation indeed :slight_smile:
There are two interesting things in your graph. This should be mentioned in the first posting.

  1. Why doesn’t the straight line start at 390.769 or 0? That’s because you are using the mean() function. The ~200 we can see is the mean of surrounding measurements. Play around with the aggregation function used (e.g. “last()”)
  2. You have two values over time: 0 and 372. How would you like the graph to look in between them? Currently you tell Grafana to interpolated between them with a straight line. That’s what it does. :wink: Activate staircase mode under “Display” and all should be fine.

Good luck!

:slight_smile: Thank you Thom!

So with your line of questions, I realised that what is different from before, is that 0 is logged only once in influxdb, whereas if I go back a month, I had more “noise”;

1482464303668000000     373.308
1482464316666000000     0.154
1482464329661000000     0.231
1482464342674000000     0.077
1482464355670000000     0.154
1482464368667000000     0.077
1482464381664000000     0.154
1482464407656000000     0
1482464420681000000     0.538
1482464433668000000     0.308
1482464446667000000     0.077
1482464485663000000     0
1482464498663000000     0.154
1482464511664000000     0.231
1482464524666000000     0
1482464537656000000     0.077
1482464576659000000     0.385
1482464589668000000     0.231
1482464602662000000     0.154
1482464628656000000     0.385
1482464641655000000     0.231
1482464654653000000     0.077
1482464667663000000     0.385
1482464680654000000     0.308
1482464693662000000     0
1482464706659000000     0.077

So I need find out why, all of a sudden, I get dead zero from mqtt/btmon/mosquitto/gem.
But even if I find that, I guess the graph is still not perfect, it just hid my real graphing problem. In a way it is actually better not to fill the database with similar values.

I simply want the ‘the last logged value’ to be true over time until theres’ an update. So I tried, and came up with

Cheers,
Micael

Welcome to the ‘staircase’ method :wink:

Hi Wouter,

I very much appreciete your library. I’ve got a question though. I modified (=simplified) your combo1 example so you can only choose between timeframes.

However, when clicking day, week or something else, nothing happens until a page refresh. Is this expected behaviour? Is a limitation of the webview element? Of did I something wrong during my “simplification”.