InfluxDB+Grafana persistence and graphing

influxdb
graphs
grafana
Tags: #<Tag:0x00007fd3111c2848> #<Tag:0x00007fd3111c2708> #<Tag:0x00007fd3111c25c8>

(John Schmitz) #528

Nothing at all in either the openHAB log or the Grafana log? Weird. But you can use Grafana directly and get the graphs you want? Just not in BasicUI?


(Kim Andersen) #529

No nothing in the log… And yes, log in locally directly to Grafana, everything is working perfectly.
After changing Org.role to “Viewer”, it does seem like it´s working in BasicUI. At least I get an image now.
Unfortunatly, it doesnt seem to refresh/update, which makes it rather pointless :frowning:


(John Schmitz) #530

What are you putting in the sitemap? In the 1st post of this thread there is some information on various ways to get it into the sitemap. The refresh should update the image.


(Kim Andersen) #531

This is from the sitemap.

Image refresh=60000 url="http://10.4.28.237:3000/render/d-solo/jwnxXmgRz/marc-temperatur?refresh=5s&orgId=1&panelId=2&from=1529258759230&to=1529269559230&width=1000&height=500&tz=UTC%2B02%3A00"

And this is how it looks. Screenshot is a few minutes old. Grafana picture hasn´t refreshed since 22.42 last night.


(John Schmitz) #532

It’s not going to change because you’ve specified the absolute time for the time range:

You want this to me something like: “&from=now-48h&to=now”


(Kim Andersen) #533

Ahh ofcouse… I thought the link was build from the time periode chosen like the same Grafana UI.

Now the image is updated and it seems to refresh allright. Thanks alot for your help, John!


(Kim Andersen) #534

I´ve got a new weird problem with Grafana…
I have created a dashboard for charting the humidity item from my Netamo indoor module.

This is the latest openhab event update from the item:

2018-06-20 22:59:06.753 [vent.ItemStateChangedEvent] - NetamoIndendRsHjemme_Humidity changed from 50 % to 51 %

But grafana dashboard is empty.

If I change the item, (just make a simple label change, save the item file), Grafana starts to read the data again and draw the chart, but just untill a few hours, then it ends up empty again…

I know for sure there has been several updates picked up by OpenHab during the day. But still, Grafana is empty…

PS, forgot to mention, I have several other dashboards which works just fine. Only differences is, that the Netamo can be rather quite for 2-3 hours with the Humidity measuring,and therefore not sending any updated measuring.

This is Grafana settings. (Screendump)


(John Schmitz) #535

You need to find out if the data was actually written to InfluxDB (implying a write problem) or if the data is written and is therefore a Grafana read problem from InfluxDB. What is your persistence strategy? Does it cover this item? You can turn on logging in InfluxDB (I think it is on by default) and see the data being written - logs to syslog. Do you see the updates being written for this sensor? InfluxDB has a command line interface - you can try the command in green in your screenshot. If it has no data, then Grafana won’t have any data points to display.


(Kim Andersen) #536

Hi John…
My worries is, that it sometimes work, and sometimes it doesn´t.
It seems like Grafana stops charting and “delete” all data for this item, after some hours without any updates. I know for sure grafana has written the chart within the last 24 hours, cause I saw it in the chart. How is it possible to dissapear all of a sudden then?

I will have a look at Influx to see if the data is actualy beeing written. But I would suspect it does. Never than less, it does not explain how this item can dissapear from the chart.


(John Schmitz) #537

You should see the queries Grafana is using in the syslog as it queries InfluxDB. If you can replicate one of those and get data when your chart is empty, then I agree there is a Grafana problem. (And in that case you’d probably want to ask in the Grafana forum for a solution.) If you see the query going to InfluxDB and no data is returned, then there has to be an issue with the query being used or in the InfluxDB itself.


(Kim Andersen) #538

Hmm, I think I may have found something… But I have no idea what I´m looking at, to be honest. It seems like log is only wirtten for the specific item, when I view the Grafana dashboard. Anyway…

First a screendump from the dashboard with the Netamo item right now, going 24 hours back:
According to this screendump, the first data querie is 00:45.

But acording to the syslog, the first entry is 23:10:06

Jun 20 23:10:06 openHABianPi influxd[750]: ts=2018-06-20T21:10:06.559405Z lvl=info msg="Executing query" log_id=08ff~SCl000 service=query query="SELECT mean(value) FROM openhab_db.autogen.NetamoIndendRsHjemme_Humidity WHERE time >= now() - 6h GROUP BY time(15s) fill(previous)"
Jun 20 23:10:06 openHABianPi influxd[750]: [httpd] 10.4.28.30, 10.4.28.30,10.4.28.237 - grafana [20/Jun/2018:23:10:06 +0200] "GET /query?db=openhab_db&epoch=ms&q=SELECT+mean%28%22value%22%29+FROM+%22lille_bad_fugt%22+WHERE+time+%3E%3D+now%28%29+-+6h+GROUP+BY+time%2815s%29+fill%28previous%29%3BSELECT+mean%28%22value%22%29+FROM+%22stort_bad_fugt%22+WHERE+time+%3E%3D+now%28%29+-+6h+GROUP+BY+time%2815s%29+fill%28previous%29%3BSELECT+mean%28%22value%22%29+FROM+%22Nilan_Input_RH%22+WHERE+time+%3E%3D+now%28%29+-+6h+GROUP+BY+time%2815s%29+fill%28previous%29%3BSELECT+mean%28%22value%22%29+FROM+%22NetamoIndendRsHjemme_Humidity%22+WHERE+time+%3E%3D+now%28%29+-+6h+GROUP+BY+time%2815s%29+fill%28previous%29 HTTP/1.1" 200 18704 "http://10.4.28.237:3000/d/_V-4oRggk/fugtighed?refresh=30s&orgId=1" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.87 Safari/537.36" 4e653a92-74ce-11e8-b7bf-000000000000 495457

23:10 is where I messed around with the item, but grafana did not show the data in the dashboard.
I removed it from one dashboard and then created a new one. And it seems like Influx as well as Grafana does indeed have the data…
Now, how come Grafana first show the data in the new dashboard at 00:45?

Strange thing though, according to the screenshot above, it has been running stable ever since. :frowning:


(Kim Andersen) #539

I think I found the reason, well part of the reason.

If I set the dashboard to show the last 6 hours, and an item hasn´t updated in between, then Grafana will not show the item, even though it has been setup til ‘previous’.


(John Schmitz) #540

That’s why I asked about your persistence strategy. Are you only logging every change? You can also log periodically by time such as every hour. This might prevent the problem you are seeing.

If Grafana has no points in the time range you specify, it won’t have any previous value to display. It makes a query to the server with the given time span and tries to display whatever it gets back from the InfluxDB server. See the first post in the persistence section:


(Kim Andersen) #541

You´re absolutly correct John. The persistens file only log every change… Sorry I totally missed that part in your previous message…

Strategies {
 everyMinute : "0 * * * * ?"
 everyHour : "0 0 * * * ?"
 everyDay : "0 0 0 * * ?"
 every15min : "*/15 * * * * ?"
 default = everyMinute
}

Items {
Temperatur* : strategy = restoreOnStartup, everyChange 
Fugtighed*, gChart*  : strategy = restoreOnStartup, everyChange 
nilan_output_ExhaustSpeed, nilan_output_InletSpeed : strategy = restoreOnStartup, everyChange
 }

Guess I could change it to:

Fugtighed*, gChart*  : strategy = restoreOnStartup, everyChange, every15min

Then it should log every change and every 15 minute, correct?


(John Schmitz) #542

maybe this should be?

"0 0/15 * * * ?"

I hope it works! :slightly_smiling_face:


(Jerome) #543

15 Minutes should be

"0 0/15 * ? * * *"

Edit:
Maybe i was too fast now and the syntax is different in rules, where i had this example from :face_with_monocle:


(Kim Andersen) #544

Hmm I have to trust you (both) in some way. I have no idea which is correct. My original was copied from another thread. And I havn´t figured out how to setup these times settings :frowning:


(Jerome) #545

According to http://www.quartz-scheduler.org/documentation/quartz-2.1.x/tutorials/tutorial-lesson-06

“0 0/15 * * * ?”

should fit.

I was shortly a bit confused because my expression has 7 parts and the on from @jswim788 has only 6 parts.
But it seems that the seventh part (stands for the year) is optional.


(John Schmitz) #546

Yes, my first one has a typo, it should have a 0/15 instead of the */15. And the second one is wrong - I’ll remove it from the post.


(Kris K) #547

hi Guys

Ive installed Influx and Grafana. I can login to Grafana but I dont get a proxy option to connect to the Influx DB. I select ‘browser’ but then I get a network error.

Influx/Grafana are on the same host and are running fine

Any suggestions?