InfluxDB+Grafana persistence and graphing

influxdb
graphs
grafana
Tags: #<Tag:0x00007f014c9b5150> #<Tag:0x00007f014c9b42a0> #<Tag:0x00007f014c9b6848>

(Patrick) #629

Hi, I am trying to edit the grafana.ini file with filezilla running on a windwows machine but I always get the answer permission denied. I can’t change anything. I have openhabian and grafana running on a raspberry pi. How can I edit this file?
Thanks.


(Udo Hartmann) #630

Well, you need permission… :slight_smile:

Use putty, login to the raspberry pi, change to the directory with grafana.ini in it, do a ls -l and see, who’s the owner of the file.
Maybe (as you’re already at the right place) use sudo nano grafana.ini to edit the file.Of course, as you now know who’s the owner, you could use filezilla to login as this particular user (haven’t tried this with filezilla yet).
But keep in mind, that this file will only be changed once (or at least on very rare occasions), so the remote access is far easier :wink:


(Patrick) #631

Thanks very much. Using sudo nano worked well :slightly_smiling_face:


(Ron) #632

Quite a long thread (I am posting #625), obviously because it is about the all important look-and-feel of openHAB… ;o)

I use the fantastic JavaScript Library created by @wborn (Thanks!) and it works great. Only issue I have is refreshing when changing to another Grafana view (e.g. from Hour to Day graph). On Chrome or the Android App it won’t update until I manually refresh; on Edge it refreshes automatically. Having installed more extensions in Chrome could have something to do with it, but that certainly doesn’t apply to the openHAB Android App.

Is there a way to ‘force’ a refresh in the App and in the browser whenever an Switch item is changed to change the view?


(Wouter Born) #633

That’s odd since I have it working without any issue with Chrome and in the OH Android App. Maybe your browser debug console (F12 -> Console) shows warnings/errors that help with finding the root cause? In Chrome/Android SSE sitemap events are used by the library. In Edge it uses long polling which might explain why it does work in Edge.


(Ron) #634

Hi @wborn, thank you for the quick response.

I did check the console but at first glance nothing special there after refreshing the page:

GET http://192.168.1.2:8080/icon/webview?format=svg 404 (Not Found)
GET http://192.168.1.2:8080/icon/number?state=22185.34200000&format=svg 404 (Not Found)
GET http://192.168.1.2:8080/icon/number?state=10494764&format=svg 404 (Not Found)
GET http://192.168.1.2:8080/icon/number?state=4927.169&format=svg 404 (Not Found)
  warning rootScope onAppEvent called without localscope
  warning rootScope onAppEvent called without localscope

(EDIT: Next order of business will be to get rid of the webview and number icon errors.)

But this time I also clicked another switch item with the console open and immediately it gives the root cause of the issue:

Uncaught DOMException: Blocked a frame with origin "http://192.168.1.2:8080" from accessing a cross-origin frame.
    at updateFrameSourceURL (http://192.168.1.2:8080/static/smarthome-grafana.js:456:46)
    at onItemUpdated (http://192.168.1.2:8080/static/smarthome-grafana.js:533:13)
    at updateItem (http://192.168.1.2:8080/static/smarthome-grafana.js:183:13)
    at EventSource.<anonymous> (http://192.168.1.2:8080/static/smarthome-grafana.js:225:13)

Using NGINX as reverse proxy (on another machine), I proxy a public domain name (sub.domain.tld/grafana/) to the local openHAB server address (192.168.1.2) and port 3000 where Grafana is listening. Looks like I need some more tweaking on the proxy settings (joining the long line of openHAB/ESH/Jetty users struggling with their proxy setup…). Still, strange that after a manual refresh everything is fine.

UPDATE: After some time, the refresh in Chrome magically works (caching I suppose, because I changed the URL recently…). But the Android App still does not refresh automatically, even after phone reboot.

UPDATE2: Changing the local address in the Android App to the public interface address as well solves it for the App as well.


(Wouter Born) #635

That’s great it’s now solved @noppes123! :slight_smile:
The library readme also contains information about working with and troubleshooting Reverse proxy configurations.


(Ron) #636

Yes, I’ve read that and my configuration is similar, but like I said, it somehow took some time before it started to work. Thanks again.

BTW, This is what it looks like in Basic UI:

Next step is getting it to work with the iOS app as well, for my wife… :grinning:


Variable passed to Grafana not processed until refresh
(Smhgit) #637

@rlkoshak

R u running grafana / influxdb on RPI? If so, which Images / tags do u use (from Dockerhub?)?

As usual, thx for the help!


(Rich Koshak) #638

No, I am running everything in virtual machines running Ubuntu Server 18.04. I really just use the default latest trans for everything I get from Dockerhub.


(Nathan Ellsworth) #639

This is one of the best threads on the site. I’ve had OH+InfluxDB+Grafana working on my RPi for a few months. Thanks for everyone’s input, especially @ThomDietrich for starting the thread.

I have a question about Retention Policies. I know it has been stated pretty strongly that we shouldn’t have to worry about them because the overall data for a typical house isn’t that much.

However, I have recently added circuit-transformer-based power monitoring to my system. I have four-channel Nerio device connected via HTTP service and a five-channel Brultech ECM-1240 connected via MQTT service. Both devices send Watts and Watt-hours for each channel, plus overall Voltage, totaling 19 measurements total. The sampling interval is one second.

Needless to say, this is a lot of data and my little RPi has nearly ground to a halt showing Grafana graphs.

Now I want to get RPs working. I only want the one-second resolution data for about two hours, to see in real-time what is going on. For historical purposes, 10- or 30- or even 60-second data would be fine.

I know this is a perfect use-case for RPs in InfluxDB.

But OH seems to only support a single InfluxDB instance. Aren’t different RPs basically separate instances from OH perspective, since the tables are all “owned” by the RP?

I want MQTT and HTTP power metrics to go into the 1-second RP. But I want my Netatmo and Wemo and other MQTT temperature data to go into the longer-term RP.

Grafana graphs could come from either RP.

RP rule would run in InfluxDB to summarize power metrics from the 1-second to the 60-second RP.

Has anyone gotten this to work before?

Any ideas appreciated.


(John Schmitz) #640

I think you are correct that you are stuck with a single retention policy in openHAB. However, I think you can do what you want with continuous queries since you can have the output of a query go to a different retention policy (see example 2 on this page: https://docs.influxdata.com/influxdb/v1.6/query_language/continuous_queries/). So I believe it should be possible to use a short retention period by default, and use continuous queries to move the data you want to keep into a different, longer retention policy. At least that’s my theory - I hope it can work for you or maybe someone else will have a better idea.


(Burzin Sumariwalla) #641

@ThomDietrich

Although I’m having some problems, great write up. I did however notice a typo in your tutorial

/etc/openhab2/persitence/influxdb.persist should be /etc/openhab2/persistence/influxdb.persist

Finally add a persistence job so openHAB sends your item states to InfluxDB. Add all items or item groups you want to persist 659 including fitting strategies to /etc/openhab2/persitence/influxdb.persist:

Strategies {
    everyMinute : "0 * * * * ?"
    everyHour   : "0 0 * * * ?"
    everyDay    : "0 0 0 * * ?"
}

Items {
    gHeatAct*, gHeatSet*, gHeatValve*   : strategy = everyChange, everyHour
    Presence_Phone                      : strategy = everyChange
}

Regards,
Burzin


(Tom) #642

Very nice tutor. I am just wondering, how you do backup of DB and DB settings. Normally I would backup confing diles *.rules, *.thinks etc and copy it back to clean install OH and it works. But how you backup this DB+Grafana setup?


(Rich Koshak) #643



(Gracjan) #644

I have a problem.
I configured Grafana, InfluxDb and OpenHab and Connect with database is passed but i havet any measurment in Grafana. I dont know why.
Help me please!


(John Schmitz) #645

First step is to convince yourself that data is being written to the Influx database. Check the InfluxDB logs and/or use the SELECT from the influx command line (see https://docs.influxdata.com/influxdb/v1.6/introduction/getting-started/). If you can see the data being written, then you can move on to checking the Grafana configuration.


(Tibor Kovacs) #646

Hi ThomDietrch, could you please advise me how to fix this? I run InfluxDB in Docker container on QNAP NAS. Thanks Tibor


(Yogi) #647

Hey guys, I’m fighting with CPU load while using grafana with influxdb.
I’m using it for temperature graphs, and showing them in habpanel. it looks like storing value into databes works fine. Problem is when I want to display graphs.
My CPU load looks like this:

Grafana:
SELECT mean("value") FROM "Balcony_Humidity" WHERE $timeFilter GROUP BY time(10m) fill(none)

Also when I’m using curl for store graphs into pictures, usualy some of them are empty.
Any idea what I have to look for?
Thanks


(Rich Koshak) #648

Usetop to determine what process is actually using the CPU. That graph says that the CPU use has spiked but it doesn’t tell you what process is using it. Also remember that each CPU on a machine gets it’s own 100%. So a 100% reading means you are using 100% of one CPU but if there are two CPUs that is only half of the available CPU.