InfluxDB+Grafana persistence and graphing

Tutorials & Examples
, ,
83

Your setup looks good. Please check your openhab.log and the log by influxdb for more details.

84

Wow I actually do pretty much prefer your solution! Very nice with quite a few benefits. I’ll link your answer in the first thread. Thanks :wink:
One request, could you edit the answer and remove the first few lines to only have the explanation of how to set up the Webview? If you prefer/agree I can also copy this part to the first thread.

85

I’ve read all those, thanks.
However, it did need “some” suggestions especialy by @rlkoshak to make me do the switch. I was using RRD4J intentionally and was glad to have understood the concept of archives. For that reason I’ll give it another shot! (wife away, you got to take the chance!)
Will report if (and only if) I get it working!

86

Installation on the Raspberry Pi 2 or 3

These are the special steps to install InfluxDB and Grafana on a Raspberry Pi 2 or 3, the older Raspberry Pi 1 is not suited due to it’s ARMv6 architecture.

The following steps expect a Debian/Raspbian Jessie based OS.

Before we can start, you need to install an apt extension for https capable repositories:

sudo apt-get update && sudo apt-get install apt-transport-https

InfluxDB

InfluxDB can be installed through the official repository. Add the key to your system, then add the repository as an apt source, then install influxdb:

wget -O - https://repos.influxdata.com/influxdb.key | sudo apt-key add -
echo "deb https://repos.influxdata.com/debian jessie stable" | sudo tee /etc/apt/sources.list.d/influxdb.list
sudo apt-get update && sudo apt-get install influxdb

After the installation was successful you can enable (start on boot) and start influxdb as every other program:

sudo systemctl daemon-reload
sudo systemctl enable influxdb.service
sudo systemctl start influxdb.service

Continue with the setup instructions in the first posting.

Grafana

Grafana does not officially support the ARM architecture but binaries are provided by @fg2it. Thanks!

echo "deb https://dl.bintray.com/fg2it/deb jessie main" | sudo tee /etc/apt/sources.list.d/grafana-fg2it.list 
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 379CE192D401AB61
sudo apt-get update && sudo apt-get install grafana

Again, enable and start the program through systemctl:

sudo systemctl daemon-reload
sudo systemctl enable grafana-server.service
sudo systemctl start grafana-server.service

Continue with the setup steps in the first posting.

Good luck!

7 Likes
87

There isn’t anything in the openhab.log other than the usual. InfluxDB doesn’t seem to have created a log.

88

Sorry for chiming in again, but that are the steps I had done and left me with an unusable operating system: I was not able to do an apt-get anymore.
On a standard Raspbian Jessie I had to install apt-transport-https because the influxdb repos are https fetched and not via http.
I had to manually delete influxdb.list, do the apt-transport-https install and then was able to install the influxdb binaries.
Maybe there is a better solution, but sorry, I am not a Linux expert 


89

Arg, of course. You mentioned it above and it’s actually also needed for quite a few repositories, including the Grafana one as well. Just missed to mention it in the summary. Thanks for pointing it out. I’ve also changed apt to apt-get as more know apt-get


I do not understand however, how the absence of this package would cripple your system. The only problem would be that the one repository can not be read while apt update till the package is installed.

90

I just couldn’t do any apt-<whatever> anymore as long as the influxdb.list was in place 
I had to delete it and then was able to proceed.

1 Like
91

(moved to first posting)

3 Likes
92

Do not get me wrong: I would still like to add the possibility of defining a RP to the guide. Which part of the process is making problems? The documentation on that matter seems quite clear: https://docs.influxdata.com/influxdb/v1/guides/downsample_and_retain/

93

That is the reason for me to keep on digging. I may not be the best in discribing such things, but I can try to find out.

It seems I’m running into the same problem as @rlkoshak.
When creating a RP as default, all data seems is lost, and I don’t see any data being persisted anymore.
When creating a RP as non default, in my example duration 1h, I observed that datapoints were logged until I had enough for an hour, then it stopped logging (Checked that only once, so far)

94

I guess I found part of the solution.
When using a NON-DEFAULT RP you need to create it in InfluxDB AND set this RP also in the InfluxDB.cfg in Openhab! Otherwise Openhab will not persist the data into that RP.
As for dropping data out of the RP, I haven’t observed that yet. My RP has a duration of 1h, however data is hold from 2016-10-23T18:06:00.021Z to 2016-10-23T19:09:00.024Z. Maybe I have wait a bit longer. Will check tomorrow.
[Edit 24Oct 11:18] After letting the RP run for more them 12 houres it looks like this:
At least one hour of data is kept, data is dropped in complete hour chunks ( default shard I assume ). The dropping is done approx 15 minutes after the. hour. [/edit]

When checking the entries using REST API, it looks to me as if REST API isn’t using the RP to querry the database. I only get the data that was persited BEFORE the RP was created.

Inside the InfluxDB the query for data in a RP named “test” looks like:

SELECT * FROM “test”.“MyData”

REST API does show the data that can querried by:

SELECT * FROM “MyData”

Is that a bug? Or do we need to use a the default RP in order to make it work for REST API?

The other problem was using a default RP. As reported all the data seems to be lost when creating a default RP. As the default RP doesn’t need to be specified in a querry, the above findings do not explain what was observed.
Further tests needed!
[Edit]
I made the default RP work, I just “ALTERD” the working RP to be default. Nothing was lost (the only difference to previous test was that the RP already worked). However in the influxdb.cfg of OpenHAB the RP needs to be set with the correct name!!,
REST API will return all values, that is no difference to the above reported bug, since now the RP is default and default RPs are not needed to be explicitly set in a query

95

That’s easy :slight_smile: As apt-get update would try to update all lists of packets, but will fail when trying to update without https, it will report errors. However, apt-get should not fail to install apt-transport-https anyway, even without disabling the https-source.
I stumble across this fault regularly :wink:

96

If you got it working I know I and a lot of people on this thread would love to see a step by step to get it working. It probably deserves a new thread.

97

Will try to do that, however I’d like to start from scratch in order to be sure everything is working as I think it does.
I have a question for @rlkoshak :
In case of the Default RP, I’m not really sure anymore what I did when checking during the inital test for entries. My findings were that after a Default RP was created, that there were no more entries in the DB. I can’t say for sure if I checked that only with REST API (in this case I could explain the outcome) or if I did also check doing a direct querry in InfluxDB. How about your tests?

98

I would rather see it as a subsection of this guide but as you wish @opus :wink:

99

I wasn’t sure if someone can edit someone else’s message without an elevated status. My main concern is that the instructions do not get lost in the nearly 100 replies on this thread. I suppose if @opus posts it as a reply you could add it as a new section.

I did most of my tests using the web page for InfluxDB and doing queries that way, which I would suppose means I was really using the REST API. I’m running InfluxDB in a Docker container so it was easier to use the web page than the command line tool to access.

100

No and yes :slight_smile: That’s how I would do it. I think the whole idea behind tutorials should be for everyone to help improve them. Of course everyone helping will be mentioned to honor their work. It’s by the way possible to turn a posting into a “wiki”. I did not try this option yet but I would prefer to have one author responsible for one article rather than many cooks to spoil the broth. :stew:

101

I’m sorry, that question was based on a false interpreation from my side.

Looking at the problem from InfluxDB:
When creating a RP, the data has to be stored INTO that RP.
In case of a default RP, there is no need to specify the RP

Looking at it from OpenHAB2
OH2 will store the data into the specified RP. Although the default should be set as “autogen”, this doesn’t work. the exact name of the RP has to be set in the .cfg.
When using REST API to read the stored data, the RP is not set in the query! In case of a default RP, the query would work. In case of a NON-default RP the query will NOT work.

Question: Should I report that as a bug in the OpenHAB-section of GitHub?

I have prepared a guide and will post it here soon.

102

Example SetUp of a RetentionPolicy(RP) for InfluxDB and using it from OpenHAB2

OpenHAB2 SetUP

Influxdb.cfg

# The database URL, e.g. http://127.0.0.1:8086 or https://127.0.0.1:8084 .
# Defaults to: http://127.0.0.1:8086
#
#url=http(s)://<host>:<port>

 # The name of the database user, e.g. openhab.
 # Defaults to: openhab
 user=openhab

 # The password of the database user.
password=MyPassword

# The name of the database, e.g. openhab.
# Defaults to: openhab
db=openhab

# The retention policy to be used, needs to configured in InfluxDB
# According to MY findings (using #550, date 25Oct2016) the RetentionPolicy has ALLWAYS to be set 
# even it is set as DEFAULT! 
retentionPolicy=MyRetentionPolicy

influxdb.persist

Strategies {     
                 everyMinute : "0 * * * * ?"
                }    

Items {
           MyItem1, MyItem2    : strategy =  everyChange, everyMinute
           }

InfluxDB SetUP

Using the CLI create the database:

>CREATE DATABASE openhab

Create a Default RP, as an example
with a duration of 1 hour .

> CREATE RETENTION POLICY " MyRetentionPolicy" ON "openhab" DURATION 1h REPLICATION 1 DEFAULT

Now the data will be persisted into the RP named “MyRetentionPolicy” in the database “openhab”.

You can check if data is persisted inside InfluxDB using queries like:

>USE DATABASE openhab
>SELECT  *  FROM  “MyRetentionPolicy”.“MyItem1”

Since “MyRetentionPolicy” was set as default, the query could be shortened to:

> SELECT * FROM “MyRetentionPolicy”.“MyItem1”

You can also check the entries in REST API.

Since the RP has been set as default, REST API will show the entries.
If the RP would not be default, the check using REST API would return nothing. (This might be considered a bug).

Note: The RP is supposed to drop data that is older than the set DURATION. That dropping of data is done on whole hours and in time-steps of the “Shard”-size, in the above example the default “Shard”-size of 1 hour is used (You will see data stored for 1 hour up to 2 hours (plus a couple of minutes)).

2 Likes