Searching for help with grafana with openhabian

openhab2
Tags: #<Tag:0x00007f0141e98790>

(jasper) #1

Hello

Im looking for an up to date tutorial for making graphs in Openhabian with the RPI3

Is this still valid:


?

But to share my point of knowledge here… Im already stuck at:
Following the installation instructions , start by adding the InfluxData repository and installing InfluxDB on your monitoring-host .
As i dont even know what is a monitoring-host. :worried:

Im trying to get something which Home assistant has right out of the box, some nice history and graphs which show my temperatures in history. I feel that i am just not seeing something as its so easy with HA but with OH i need to do a lot of “coding”

On youtube, nor here i cannot find a simple explanation on how to get graphs to show in OH (which tells more about me than about YT or this community).

Im running Openhab 2.4 on a freshly installed RPI3b
I have 5 zwave devices which all show up in PaperUI and i have been able to get them nicely in HabPanel as well. I have even managed to get a shared network drive on my WIN10 pc that shows the content of the openhab-conf directory and i installed Visual studio code, and added the openhab extension. Thanks to several tutorials though.

But now i cannot find some clear info on how to use influxdb (i dont even know what it is) and how to get it to work with something like Grafana.

I added Influxdb via addon > persistence.
But it does not show up when i want to select the default persistence in System, neither after a reboot.

If anybody would be so kind to get me into the right direction it would be so nice.

Thank you.


(Bob Miles) #2

Hi!
So let’s get started with some basic information:
The way it will work is, that openhab will store values from your devices/sensors/lamps in a database, including a timestamp.
The database used in the tutorial you posted is called influxdb. It is dedicated to data that is in form of a time series (just what we want). It will delete old measurements after some time to make space for new ones.
So openhab already comes with some magic as it will do the storing of values automatically, once you told it where your database is.
To set up the database, just follow the tutorial you posted.
The monitoring host will be your raspberry pi as this is where the database should be hosted. After installing influxdb from paperUI, just open a remote shell or remote shell on your raspberry and execute the command “influx”, it will take you to the influxdb command line interface where you can run all the commands to set up users and so on, just as shown in the tutorial.
Later, grafana will connect to the database (your influxdb) and get the data out.

So best would be you try to work through the tutorial as far as you get and then post the specific point where you’re stuck.


(jasper) #3

Thank you BobMiles for your answer.
ok, i will try to get my issues on paper.

clicking on the installation instructions tells me the information is outdated. Should i click on the InfluxDB v1.7 or not (maybe the OP could update the link in that case)

In PaperUI i installed InfluxDB (V 1.0) persistence (persistence-influxdb - 1.13.0
in the log it shows as:
2018-12-25 22:41:47.769 [thome.event.ExtensionEvent] - Extension ‘persistence-influxdb’ has been installed.

after this is done, there is no posibilty to enable any default persistence in the system settings. Neither after a reboot.
Going with SSH and give command influx or sudo influx. i get Command not found.

So glad for your help already!
thx.


(Bob Miles) #4

Influxdb is not installed when you install the influxdb persistence service in paperUI. This is only the connection that will write and handle the database.
As suggested in the tutorial, you have to install influxdb manually:


Afterwards you can use the influx command and go on in the tutorial.


(jasper) #5

as this is for debian jessie i am wondering…

while searching on Google, i see that it is possible to use the openhabian-config and use #24 InfluxDB-Grafana.

Would this be a safer option for me?


(Bob Miles) #6

Oh yes of course! Sorry I don’t remember exactly how many of the steps is integrated in the openhabian-config…


(jasper) #7

so far:
sudo openhabian-config.
go to #24, install influxdb+grafana (it will tell to go to this post: InfluxDB+Grafana persistence and graphing )

its asking me to install locally or use existing instance.
no idea what to pick so i try the first one. Install locally.

enter passwords for a few times. I enter all the same password for each of them.

Shall influxDB be configured with Openhab? I select Yes
downloading…

Operation Succesful!
OK.

If this is all correct, how can we add this into the tutorial so that others can benefit from this as well?,

[22:52:12] openhabian@openHABianPi:~$ sudo openhabian-config
2018-12-25_22:53:17_EET [openHABian] Checking for root privileges… OK
2018-12-25_22:53:17_EET [openHABian] Loading configuration file ‘/etc/openhabian.conf’… OK
2018-12-25_22:53:18_EET [openHABian] openHABian configuration tool version: [master]v1.4.1-430(67ea550)
2018-12-25_22:53:18_EET [openHABian] Checking for changes in origin… OK
2018-12-25_22:53:23_EET [openHABian] Setting up InfluxDB and Grafana…

Installing InfluxDB…

$ apt -y install apt-transport-https
Reading package lists… Done
Building dependency tree
Reading state information… Done
apt-transport-https is already the newest version (1.4.8).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
–2018-12-25 22:57:34-- https://repos.influxdata.com/influxdb.key
Resolving repos.influxdata.com (repos.influxdata.com)… 2600:9000:2074:1200:7:7790:e740:93a1, 2600:9000:2074:7800:7:779 0:e740:93a1, 2600:9000:2074:ca00:7:7790:e740:93a1, …
Connecting to repos.influxdata.com (repos.influxdata.com)|2600:9000:2074:1200:7:7790:e740:93a1|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 3108 (3.0K) [application/pgp-keys]
Saving to: ‘STDOUT’

  •                         100%[=================================================>]   3.04K  --.-KB/s    in 0s
    

2018-12-25 22:57:34 (76.7 MB/s) - written to stdout [3108/3108]

OK

$ apt update
Hit:1 http://archive.raspberrypi.org/debian stretch InRelease
Hit:2 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Ign:3 http://repos.azulsystems.com/debian stable InRelease
Hit:4 http://repos.azulsystems.com/debian stable Release
Get:5 https://repos.influxdata.com/debian stretch InRelease [4,732 B]
Ign:6 https://dl.bintray.com/openhab/apt-repo2 stable InRelease
Hit:7 https://deb.nodesource.com/node_8.x stretch InRelease
Hit:8 https://dl.bintray.com/openhab/apt-repo2 stable Release
Get:10 https://repos.influxdata.com/debian stretch/stable armhf Packages [918 B]
Fetched 5,650 B in 2s (1,928 B/s)
Reading package lists… Done
Building dependency tree
Reading state information… Done
All packages are up to date.

$ apt -y install influxdb
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following NEW packages will be installed:
influxdb
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 43.0 MB of archives.
After this operation, 93.2 MB of additional disk space will be used.
Get:1 https://repos.influxdata.com/debian stretch/stable armhf influxdb armhf 1.7.2-1 [43.0 MB]
Fetched 43.0 MB in 39s (1,101 kB/s)
Selecting previously unselected package influxdb.
(Reading database … 46236 files and directories currently installed.)
Preparing to unpack …/influxdb_1.7.2-1_armhf.deb …
Unpacking influxdb (1.7.2-1) …
Setting up influxdb (1.7.2-1) …
Created symlink /etc/systemd/system/influxd.service → /lib/systemd/system/influxdb.service.
Created symlink /etc/systemd/system/multi-user.target.wants/influxdb.service → /lib/systemd/system/influxdb.service.
Processing triggers for man-db (2.7.6.1-2) …
Updating FireMotD available updates count … 2018-12-25 22:58:40,519: FireMotD: Error: Template folder doesn’t exist. U pgrading from an older version? Please run make again or run the install function “FireMotD -I -v”.

$ systemctl daemon-reload

$ systemctl enable influxdb.service

$ systemctl restart influxdb.service
OK Configure InfluxDB admin account… {“results”:[{“statement_id”:0}]}
OK Configure listen on localhost only…
$ sed -i -e /^# Determines whether HTTP endpoint is enabled./ { n ; s/^# enabled = true/enabled = true/ } /etc/influxdb/ influxdb.conf

$ sed -i s/# bind-address = “:8086”/bind-address = “localhost:8086”/g /etc/influxdb/influxdb.conf

$ sed -i s/# auth-enabled = false/auth-enabled = true/g /etc/influxdb/influxdb.conf

$ systemctl restart influxdb.service
OK
Setup of inital influxdb database and InfluxDB users… {“results”:[{“statement_id”:0}]}
{“results”:[{“statement_id”:0}]}
{“results”:[{“statement_id”:0}]}
{“results”:[{“statement_id”:0}]}
{“results”:[{“statement_id”:0}]}
OK Installing Grafana…
$ wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_5.3.4_armhf.deb
–2018-12-25 22:59:07-- https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_5.3.4_armhf.deb
Resolving s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)… 52.218.144.72
Connecting to s3-us-west-2.amazonaws.com (s3-us-west-2.amazonaws.com)|52.218.144.72|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 27059330 (26M) [application/x-debian-package]
Saving to: ‘grafana_5.3.4_armhf.deb’

grafana_5.3.4_armhf.deb 100%[========================================================================================================================================>] 25.81M 393KB/s in 42s

2018-12-25 22:59:50 (633 KB/s) - ‘grafana_5.3.4_armhf.deb’ saved [27059330/27059330]

$ apt update
Hit:1 http://archive.raspberrypi.org/debian stretch InRelease
Ign:2 http://repos.azulsystems.com/debian stable InRelease
Hit:3 http://raspbian.raspberrypi.org/raspbian stretch InRelease
Hit:4 http://repos.azulsystems.com/debian stable Release
Hit:5 https://repos.influxdata.com/debian stretch InRelease
Hit:6 https://deb.nodesource.com/node_8.x stretch InRelease
Ign:7 https://dl.bintray.com/openhab/apt-repo2 stable InRelease
Hit:8 https://dl.bintray.com/openhab/apt-repo2 stable Release
Reading package lists… Done
Building dependency tree
Reading state information… Done
All packages are up to date.

$ apt-get install -y adduser libfontconfig
Reading package lists… Done
Building dependency tree
Reading state information… Done
Note, selecting ‘libfontconfig1’ instead of ‘libfontconfig’
adduser is already the newest version (3.115).
libfontconfig1 is already the newest version (2.11.0-6.7).
libfontconfig1 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

$ dpkg -i grafana_5.3.4_armhf.deb
Selecting previously unselected package grafana.
(Reading database … 46262 files and directories currently installed.)
Preparing to unpack grafana_5.3.4_armhf.deb …
Unpacking grafana (5.3.4) …
Setting up grafana (5.3.4) …
Adding system user grafana' (UID 112) ... Adding new usergrafana’ (UID 112) with group grafana' ... Not creating home directory/usr/share/grafana’.

NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server

You can start grafana-server by executing

sudo /bin/systemctl start grafana-server
Processing triggers for systemd (232-25+deb9u6) …

$ systemctl daemon-reload

$ systemctl enable grafana-server.service
Synchronizing state of grafana-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable grafana-server
Created symlink /etc/systemd/system/multi-user.target.wants/grafana-server.service → /usr/lib/systemd/system/grafana-server.service.

$ systemctl start grafana-server.service
OK
Updating Grafana admin password…{“message”:“User password updated”}OK Updating Grafan configuration…
$ sed -i -e /^# disable user signup / registration/ { n ; s/^;allow_sign_up = true/allow_sign_up = false/ } /etc/grafana/grafana.ini

$ sed -i -e /^# enable anonymous access/ { n ; s/^;enabled = false/enabled = true/ } /etc/grafana/grafana.ini

$ systemctl restart grafana-server.service
Connection Grafana to InfluxDB…{“datasource”:{“id”:1,“orgId”:1,“name”:“openhab_home”,“type”:“influxdb”,“typeLogoUrl”:"",“access”:“proxy”,“url”:“http://localhost:8086”,“password”:“xxxx”,“user”:"",“database”:“openhab_db”,“basicAuth”:true,“basicAuthUser”:"",“basicAuthPassword”:“xxxxx”,“withCredentials”:false,“isDefault”:false,“secureJsonFields”:{},“version”:1,“readOnly”:false},“id”:1,“message”:“Datasource added”,“name”:“openhab_home”}Adding openHAB dashboard tile for Grafana… 2018-12-25_23:00:33_EET [openHABian] Adding an openHAB dashboard tile for ‘grafana’… OK
Adding install InfluxDB with database configuration to OpenHAB
$ touch /etc/openhab2/services/influxdb.cfg
2018-12-25_23:01:55_EET [openHABian] Checking for default openHABian username:password combination… FAILED
2018-12-25_23:01:56_EET [openHABian] We hope you got what you came for! See you again soon :wink:


(jasper) #8

after a reboot i see a new UI in my OPENHAB
Grafana.

and now i can indeed enter the influx command.
Well so far that went quite a lot easier than i was reading in the forum. Should we add this into some of the basics of the openhab help pages??
I can contribute…


(jasper) #9

next place where i get stuck.

now when doing the “easy way” of using the Openhabian-config file, i have no idea where to continue in the tutorial but i decided to follow from Connecting Grafana to InfluxDB

here it is told:
_Logged in to the Grafana dashboard, go to “Data Sources” and create a new source pointing to your InfluxDB database, providing the credentials you chose earlier. _

but the only thing i see is this:
Not any Data Sources possible.

Edit,
After finding a small button on the lower left i can log in.
I decided to log in as admin. (not sure if this is wise?)

in data sources it shows openhab_home. Well that seems good i guess.
not sure where to continue now?


(Bob Miles) #10

Hi! Yes loggin in as admin to grafana is correct.
Now you need to tell openhab where the database is and how often values should be written to the database.
This is configured in the files /etc/openhab2/services/influxdb.cfg and /etc/openhab2/persitence/influxdb.persist. Just have a look at the section

Connecting openHAB to InfluxDB


(Vincent Regaud) #11

You openHABmachine


(jasper) #12

next steps done:
in influxdb.cfg i wrote:

# The database URL, e.g. http://127.0.0.1:8086 or https://127.0.0.1:8084 .
# Defaults to: http://127.0.0.1:8086
# url=http(s)://<host>:<port>
url=http://192.168.8.104:8086

# The name of the database user, e.g. openhab.
# Defaults to: openhab
# user=<user>
user=openhab

# The password of the database user.
# password=
password=mysecretpassword

# The name of the database, e.g. openhab.
# Defaults to: openhab
db=openhab_db

# db=<database>url=http://localhost:8086
db=openhab_db
retentionPolicy=autogen

# Enable authentication in the [http] section of the configuration file /etc/influxdb/influxdb.conf:

[http]  
enabled = true  
bind-address = ":8086"      # change to a specific interface if needed 
auth-enabled = true         # will enforce authentication

in persistence i made a new file called influxdb.persist
but how do i know what items i need to add if i have them added in PaperUI and my items map is empty except the readme.txt file?

i went to PaperUI select items and if i want to have my node6 electricity meter i copy the name and paste it in Items like this?
zwave_device_ff0bc9cd_node6_meter_watts
Or something else?

this is what i have now made:

// persistence strategies have a name and a definition and are referred to in the "Items" section
Strategies {
    everyMinute : "0 * * * * ?"
	everyHour   : "0 0 * * * ?"
	everyDay    : "0 0 0 * * ?"

	// if no strategy is specified for an item entry below, the default list will be used
	default = everyChange
}

/* 
 * Each line in this section defines for which item(s) which strategy(ies) should be applied.
 * You can list single items, use "*" for all items or "groupitem*" for all members of a group
 * item (excl. the group item itself).
 */
Items {
	// persist all items once a day and on every change and restore them from the db at startup
	* : strategy = everyChange, everyDay, restoreOnStartup
	
	// additionally, persist all temperature and weather values every hour
	//Temperature*, Weather* : strategy = everyHour
}

after a reboot, i see in the frontail:

2018-12-26 22:24:28.741 [ERROR] [nal.common.AbstractInvocationHandler] - An error occurred while calling method 'QueryablePersistenceService.query()' on 'org.openhab.core.persistence.internal.QueryablePersistenceServiceDelegate@ec1dd': Connection refused (Connection refused)

2018-12-26 22:24:30.048 [ERROR] [ence.internal.PersistenceManagerImpl] - Exception occurred while querying persistence service 'influxdb': Connection refused (Connection refused)


and a lot of


2018-12-26 22:24:34.998 [ERROR] [org.influxdb.impl.BatchProcessor    ] - Batch could not be sent. Data will be lost


2018-12-26 22:07:57.394 [ERROR] [nal.common.AbstractInvocationHandler] - An error occurred while calling method 'QueryablePersistenceService.query()' on 'org.openhab.core.persistence.internal.QueryablePersistenceServiceDelegate@15c7183': Connection refused (Connection refused)

retrofit.RetrofitError: Connection refused (Connection refused)

	at retrofit.RetrofitError.networkError(RetrofitError.java:27) ~[?:?]

	at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:395) ~[?:?]

	at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:240) ~[?:?]

	at org.influxdb.impl.$Proxy127.query(Unknown Source) ~[?:?]

	at org.influxdb.impl.InfluxDBImpl.query(InfluxDBImpl.java:195) ~[?:?]

	at org.openhab.persistence.influxdb.internal.InfluxDBPersistenceService.query(InfluxDBPersistenceService.java:358) ~[?:?]

	at org.openhab.core.persistence.internal.QueryablePersistenceServiceDelegate.query(QueryablePersistenceServiceDelegate.java:47) ~[?:?]

	at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source) ~[?:?]

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:?]

	at org.eclipse.smarthome.core.internal.common.AbstractInvocationHandler.invokeDirect(AbstractInvocationHandler.java:153) [102:org.eclipse.smarthome.core:0.10.0.oh240]

	at org.eclipse.smarthome.core.internal.common.Invocation.call(Invocation.java:53) [102:org.eclipse.smarthome.core:0.10.0.oh240]

	at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:?]

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]

	at java.lang.Thread.run(Thread.java:748) [?:?]

Caused by: java.net.ConnectException: Connection refused (Connection refused)

	at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:?]

	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:?]

	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:?]

	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:?]

	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:?]

	at java.net.Socket.connect(Socket.java:589) ~[?:?]

	at com.squareup.okhttp.internal.Platform.connectSocket(Platform.java:100) ~[?:?]

	at com.squareup.okhttp.internal.http.SocketConnector.connectRawSocket(SocketConnector.java:160) ~[?:?]

	at com.squareup.okhttp.internal.http.SocketConnector.connectCleartext(SocketConnector.java:67) ~[?:?]

	at com.squareup.okhttp.Connection.connect(Connection.java:152) ~[?:?]

	at com.squareup.okhttp.Connection.connectAndSetOwner(Connection.java:185) ~[?:?]

	at com.squareup.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:128) ~[?:?]

	at com.squareup.okhttp.internal.http.HttpEngine.nextConnection(HttpEngine.java:341) ~[?:?]

	at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:330) ~[?:?]

	at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:248) ~[?:?]

	at com.squareup.okhttp.Call.getResponse(Call.java:273) ~[?:?]

	at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:230) ~[?:?]

	at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:201) ~[?:?]

	at com.squareup.okhttp.Call.execute(Call.java:81) ~[?:?]

	at retrofit.client.OkClient.execute(OkClient.java:53) ~[?:?]

	at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:326) ~[?:?]

	... 14 more

this repeats all over.

and i also get a page not available when i go to 192.168.8.104:8086

Any idea what i did not do correct? Thanks again.


(Bob Miles) #13

Have you set the correct password and user name in your configuration file? Seems like it rejects you beacause of wrong or missing credentials…


(jasper) #14

Dear Bob,

Thank you for all your help.
I finely managed to get it to work with following this tutorial.

I would recommend others to follow the same,

Thank you @BobMiles and @jimmyalias who helped me a lot!

main issues for me were

  • my modem separated LAN and WIFI for some reason. Now everything is on WIFI including the RPI3
  • 1 faulty SD card (created some really annoying feature that my changes were not done after a reboot
  • lack of knowledge about programming, but with visual studio code, even i can do it :slight_smile:
  • some sources on the internet are outdated and as this might be obvious for others… Me as a beginner has problems in seeing the differences.
  • too hasty willing to get the best results.

I hope i can start to contribute soon, instead of being a pain.