InfluxDB+Grafana persistence and graphing

Tutorials & Examples
, ,
81

Data from OpenHAB isn’t populating the InfluxDB. Here’s a the config details;

# The name of the default persistence service to use
persistence:default=influxdb

############################ InfluxDB Persistence Service #############################
#
# The database URL, e.g. http://127.0.0.1:8086 or https://127.0.0.1:8084 .
# Defaults to: http://127.0.0.1:8086
influxdb:url=http://192.168.10.6:8086

# The name of the database user, e.g. openhab.
# Defaults to: openhab
influxdb:user=openhab

# The password of the database user.
influxdb:password=MYPASSWORD

# The name of the database, e.g. openhab.
# Defaults to: openhab
influxdb:db=openhab_db

The data in the database is from the Sine Wave script. So I’m guessing I’ve got a config item wrong.

Here’s my persist details;

Strategies {
everyHour : “0 0 * * * ?”
}

Items {
HRT4_ZW_Temperature* : strategy = everyChange, everyHour
Weather_Temperature, SpeedtestResultDown : strategy = everyChange
}

82

Ah nice! I had no idea the arm binaries are now also available in the repository (were not before). Thanks for pointing that out! :wink: (@KjetilA: this may be interesting to you too)

As said a couple of times in this thread: Do not waste your time with retention policies. Even with a big openHAB setup the stored data will sum up to a few Megabytes per year…

That’s actually what is used in the openHABian steps. @opus obviously something went wrong while you worked through the steps. Can you check every step again?

The linked openHABian lines are by the way not yet part of the menu as I am still in he process of finishing them.

83

Your setup looks good. Please check your openhab.log and the log by influxdb for more details.

84

Wow I actually do pretty much prefer your solution! Very nice with quite a few benefits. I’ll link your answer in the first thread. Thanks :wink:
One request, could you edit the answer and remove the first few lines to only have the explanation of how to set up the Webview? If you prefer/agree I can also copy this part to the first thread.

85

I’ve read all those, thanks.
However, it did need “some” suggestions especialy by @rlkoshak to make me do the switch. I was using RRD4J intentionally and was glad to have understood the concept of archives. For that reason I’ll give it another shot! (wife away, you got to take the chance!)
Will report if (and only if) I get it working!

86

Installation on the Raspberry Pi 2 or 3

These are the special steps to install InfluxDB and Grafana on a Raspberry Pi 2 or 3, the older Raspberry Pi 1 is not suited due to it’s ARMv6 architecture.

The following steps expect a Debian/Raspbian Jessie based OS.

Before we can start, you need to install an apt extension for https capable repositories:

sudo apt-get update && sudo apt-get install apt-transport-https

InfluxDB

InfluxDB can be installed through the official repository. Add the key to your system, then add the repository as an apt source, then install influxdb:

wget -O - https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian jessie stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
sudo apt-get update && sudo apt-get install influxdb

After the installation was successful you can enable (start on boot) and start influxdb as every other program:

sudo systemctl daemon-reload
sudo systemctl enable influxdb.service
sudo systemctl start influxdb.service

Continue with the setup instructions in the first posting.

Grafana

Grafana does not officially support the ARM architecture but binaries are provided by @fg2it. Thanks!

echo "deb https://dl.bintray.com/fg2it/deb jessie main" | sudo tee /etc/apt/sources.list.d/grafana-fg2it.list 
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 379CE192D401AB61
sudo apt-get update && sudo apt-get install grafana

Again, enable and start the program through systemctl:

sudo systemctl daemon-reload
sudo systemctl enable grafana-server.service
sudo systemctl start grafana-server.service

Continue with the setup steps in the first posting.

Good luck!

7 Likes
87

There isn’t anything in the openhab.log other than the usual. InfluxDB doesn’t seem to have created a log.

88

Sorry for chiming in again, but that are the steps I had done and left me with an unusable operating system: I was not able to do an apt-get anymore.
On a standard Raspbian Jessie I had to install apt-transport-https because the influxdb repos are https fetched and not via http.
I had to manually delete influxdb.list, do the apt-transport-https install and then was able to install the influxdb binaries.
Maybe there is a better solution, but sorry, I am not a Linux expert …

89

Arg, of course. You mentioned it above and it’s actually also needed for quite a few repositories, including the Grafana one as well. Just missed to mention it in the summary. Thanks for pointing it out. I’ve also changed apt to apt-get as more know apt-get…

I do not understand however, how the absence of this package would cripple your system. The only problem would be that the one repository can not be read while apt update till the package is installed.

90

I just couldn’t do any apt-<whatever> anymore as long as the influxdb.list was in place …I had to delete it and then was able to proceed.

1 Like
91

(moved to first posting)

3 Likes
92

Do not get me wrong: I would still like to add the possibility of defining a RP to the guide. Which part of the process is making problems? The documentation on that matter seems quite clear: https://docs.influxdata.com/influxdb/v1/guides/downsample_and_retain/

93

That is the reason for me to keep on digging. I may not be the best in discribing such things, but I can try to find out.

It seems I’m running into the same problem as @rlkoshak.
When creating a RP as default, all data seems is lost, and I don’t see any data being persisted anymore.
When creating a RP as non default, in my example duration 1h, I observed that datapoints were logged until I had enough for an hour, then it stopped logging (Checked that only once, so far)

94

I guess I found part of the solution.
When using a NON-DEFAULT RP you need to create it in InfluxDB AND set this RP also in the InfluxDB.cfg in Openhab! Otherwise Openhab will not persist the data into that RP.
As for dropping data out of the RP, I haven’t observed that yet. My RP has a duration of 1h, however data is hold from 2016-10-23T18:06:00.021Z to 2016-10-23T19:09:00.024Z. Maybe I have wait a bit longer. Will check tomorrow.
[Edit 24Oct 11:18] After letting the RP run for more them 12 houres it looks like this:
At least one hour of data is kept, data is dropped in complete hour chunks ( default shard I assume ). The dropping is done approx 15 minutes after the. hour. [/edit]

When checking the entries using REST API, it looks to me as if REST API isn’t using the RP to querry the database. I only get the data that was persited BEFORE the RP was created.

Inside the InfluxDB the query for data in a RP named “test” looks like:

SELECT * FROM “test”.“MyData”

REST API does show the data that can querried by:

SELECT * FROM “MyData”

Is that a bug? Or do we need to use a the default RP in order to make it work for REST API?

The other problem was using a default RP. As reported all the data seems to be lost when creating a default RP. As the default RP doesn’t need to be specified in a querry, the above findings do not explain what was observed.
Further tests needed!
[Edit]
I made the default RP work, I just “ALTERD” the working RP to be default. Nothing was lost (the only difference to previous test was that the RP already worked). However in the influxdb.cfg of OpenHAB the RP needs to be set with the correct name!!,
REST API will return all values, that is no difference to the above reported bug, since now the RP is default and default RPs are not needed to be explicitly set in a query

95

That’s easy :slight_smile: As apt-get update would try to update all lists of packets, but will fail when trying to update without https, it will report errors. However, apt-get should not fail to install apt-transport-https anyway, even without disabling the https-source.
I stumble across this fault regularly :wink:

96

If you got it working I know I and a lot of people on this thread would love to see a step by step to get it working. It probably deserves a new thread.

97

Will try to do that, however I’d like to start from scratch in order to be sure everything is working as I think it does.
I have a question for @rlkoshak :
In case of the Default RP, I’m not really sure anymore what I did when checking during the inital test for entries. My findings were that after a Default RP was created, that there were no more entries in the DB. I can’t say for sure if I checked that only with REST API (in this case I could explain the outcome) or if I did also check doing a direct querry in InfluxDB. How about your tests?

98

I would rather see it as a subsection of this guide but as you wish @opus :wink:

99

I wasn’t sure if someone can edit someone else’s message without an elevated status. My main concern is that the instructions do not get lost in the nearly 100 replies on this thread. I suppose if @opus posts it as a reply you could add it as a new section.

I did most of my tests using the web page for InfluxDB and doing queries that way, which I would suppose means I was really using the REST API. I’m running InfluxDB in a Docker container so it was easier to use the web page than the command line tool to access.

100

No and yes :slight_smile: That’s how I would do it. I think the whole idea behind tutorials should be for everyone to help improve them. Of course everyone helping will be mentioned to honor their work. It’s by the way possible to turn a posting into a “wiki”. I did not try this option yet but I would prefer to have one author responsible for one article rather than many cooks to spoil the broth. :stew: