InfluxDB+Grafana persistence and graphing

Hi guys,
I’m using InfluxDB and Grafana with good success. But I still have some open points regarding Grafana and how the data is persisted in measurements and finally shown in graphs.

My goal is to have a user friendly dashboard that provides as much information as possible. As Grafana provides functionality like Templating my thought was to use a variable with all the names of my homematic thermostats (i.e. Things) and let the user select either one or multiple of them. Then setup a graph that takes this variable and repeats itself with all of the selected names and displays the respective measurements (i.e. Items)

Before I continue let me give you some more details first. I have:

  • homematic thermostats with 5 Items each that are autogenerated via PaperUI (e.g. homematic_HMIP_eTRV_ccu_xxxxxxxxxxxxxx_0_RSSI_DEVICE, *1_ACTIVE_PROFILE, *1_BOOST_MODE, *1_SET_POINT_TEMPERATURE, *1_ACTUAL_TEMPERATURE)
  • one measurement for each of the items in InfluxDB with pairs of (time, value) data inside

I’ve managed to create a graph that collects data from multiple measuremets based on a variable $Serial like this:

SELECT mean("value") FROM /homematic_HMIP_eTRV_ccu_$Serial.*/ WHERE $timeFilter GROUP BY time($__interval) fill(null)

This graph repeats itself based on $Serial and has dynamic title as well. $Serial is a custom variable with all of the strings (e.g. xxxxxxxxxxxxxx, yyyyyyyyyyyyyy, zzzzzzzzzzzzzz, ...)

But, the result is not as user friendly as I thought. I like to replace the $Serial with something more human readable.

Any thoughts on that how it can be done?

Is there some good advice on how to tame the memory consumption of InfluxDB on a Raspberry Pi?

Over time it starts consuming more and more memory, which then results in swapping memory to disk. Which causes a lot of load and in the end the whole Pi becomes completely unresponsive. Which can only be solved by restarting thet Pi. :cry:

Ofcourse it can be solved with a cronjob that periodically checks the memory consumption and restart it with a: sudo service influxdb restart

But hopefully there is a more elegant solution for this. :smile:

There’s also a bit dated article Memory Adventures with InfluxDB covering this subject. It seems storing non-indexed data helps a lot with this.

I have the same question. Does anyone have experience with getting this up and running on a Synology NAS?

If have used two docker images on my synology. Work perfect.

The way to install is themn reduced on the configuration of variables and folders after the image is installed with docker

Thomas

Hi,
I am running OpenHAB on a Synology.
I have a chroot environment running there, to be able to install stuff using “apt-get”.
I was able to install grafana without an issue (and access the webbased environment).
Regarding influxdb, I am a bit stuck.

After installing influxdb via apt-get, I need to perform the following action

service influxdb start

However, I get “influxdb: unrecognized service” as a response.
I could manually start “influxd” however, but I guess that process does not survive a reboot…

I followed your steps to create the database and users. At a certain stage, you ask to restart the InfluxDB service using “systemctl restart influxdb.service”, but also here I get:

root@DiskStation:/# systemctl restart influxdb.service
Running in chroot, ignoring request.

Does anyone have an idea what I am doing wrong?
I guess it is chroot related? Not sure if other people run influxdb using chroot.

@Dibbler42: Thanks for your reply. I also could have gone the Docker way. But I am considering to have OpenHAB itself also installed via chroot. This allows more easy upgrading (via apt-get) than the standard Syno-package.

1 Like

How did you install influxdb? There should be a .service file to start/stop/restart/status the service.

As I use debian-x64 it may not help, but it may give you an idea how to generate this file, so this is my influxdb.service file:

/lib/systemd/system/influxdb.service:

# If you modify this, please also make sure to edit init.sh

[Unit]
Description=InfluxDB is an open-source, distributed, time series database
Documentation=https://docs.influxdata.com/influxdb/
After=network-online.target

[Service]
User=influxdb
Group=influxdb
# LimitNOFILE=65536
EnvironmentFile=-/etc/default/influxdb
ExecStart=/usr/bin/influxd -config /etc/influxdb/influxdb.conf ${INFLUXD_OPTS}
KillMode=control-group
Restart=on-failure

[Install]
WantedBy=multi-user.target
Alias=influxd.service

You have to do a sudo systemctl daemon-reload and sudo systemctl enable influxdb.service to recognize the file and to enable startup at boot time.

Hi @Udo_Hartmann,

Thank you for your help.
My influxdb.service looks exactly the same as yours. Except that you commented the LimitNOFILE line.

When I try your suggestion, I get again a “running in chroot” message:

root@DiskStation:/lib/systemd/system# systemctl daemon-reload
Running in chroot, ignoring request.
root@DiskStation:/lib/systemd/system# systemctl enable influxdb.service
root@DiskStation:/lib/systemd/system# systemctl enable influxdb.service
root@DiskStation:/lib/systemd/system# service influxdb start
influxdb: unrecognized service

The current workaround I have set up, is to create a script (in /root):

#!/bin/bash
ps cax | grep influxd > /dev/null
if [ $? -eq 0 ]; then
  echo "Process is running."
else
  echo "Process is not running."
  influxd &
  echo "Influxd started"
fi

Make it executable (chmod a+x /root/check_if_influxdb_is_running.sh).
And using the Synology task scheduler I have defined that this script should be executed on boot.
This works fine. I still need to check if this is a stable solution (I just got it up and running for half an hour).

Hi @ThomDietrich,

First thanks for writing this tutorial. I was able to set up InfluxDB and Grafana and have my first few graphs. :slight_smile:

One thing I would add to your opening post is related to the webview. The solution proposed works great in a browser, but not in the Android app. However, this can easily be solved.

In your sitemap, don’t put this:

Webview url="/static/chartsHour.html" height=33 visibility=[weatherChartPeriod=="0"]

But this:

Webview url="http://YOURIP:YOURPORT/static/chartsHour.html" height=33 visibility=[weatherChartPeriod=="0"]

So it needs a fully qualified URL.

I didn’t come up with this myself. All credits to @wborn. Source.

In this variant, the time for displaying the graphs is significantly slowed down in the paperUI. This is normal?

I didn’t have that experience. Both scenario’s seem to have the same loading time (approx 1 second here). However, I must admit that I am still using my local network’s IP address (192.168…), so it does not go via the firewall. I still need to decide what to do about allowing WAN-access to my graphs. I haven’t figured out yet if accessing graphs over the internet is worth having another potential entry point for people with bad intentions (but that is another discussion).

Thanks @Dries, I’ve added the warning to the initial article. Maybe @pahansen wants to also add this information to his original tutorial?

Thanks @Dries the post is amended, including a small hint on how scroll bars can be hidden if they are getting in the way.

1 Like

I have two problems.
1.I can’t connect like openhab user. I must be connect like admin.
2. When I add link to my sitemap so display like on picture.

Here is how to graph minute consumption from counter, and convert kW to W (using math in Grafana)

2 Likes

Hi, Grafana version 4.3 is able to use mysql data.
I am running on raspberry Pi 2 and mysql server on localhost.

2 Likes

InfluxDB+Grafana+Raspi3+Openhabian are working great and installation with the tutorial was easy.
But I don’t understand how to display Switch-States in charts:


How do I have to configure the metrics to interpret ON/OFF as 1/0?

If you want to achieve something similar to the graph below, change the “select” and “group by” options and choose staircase in Display->Mode options

2 Likes

Hi @ThomDietrich
many thanks for tour guide.
I have running grafanda with influxdb into a docker container and thanks to your guide all working file.
Now I would like to add graph to OpenHab2 using this: https://github.com/wborn/smarthome-grafana

I have not experience with this kind of objects and I cannot understand where install code.
Can you help me?

Many Thanks

marco

Follow the Example. You will notice several folders that map to your openHAB conf folder (/etc/openhab2 for apt-get install).

You will notice an html folder which contains the HTML files, some css files, and js files. Make sure you have these files in your conf/html folder.

Use the html files as templates for how to create your own webviews and the sitemap in the sitemap folder as a template for how to add the webviews to your sitemap.

The “smarts” are in the .js and .css files.

2 Likes

hi @rlkoshak
so I don’t have to configure nothing external to penhab, like another web server? Is just enought to copy example on my conf folder?

Great, Thanks