InfluxDB+Grafana persistence and graphing

@rlkoshak, @ThomDietrich
Could one of you give me an adice on my question concerning the Bug-Report. I’m not sure if I should file the missbehaviour of the InfluxDB-Binding as an issue in the OpenHAB section of Github?

Hallo Jürgen,

Sounds good to me…

This part is unclear to me. Who or what is doing something wrong here?

Maybe the developer of the influxdb addon can say something regarding this. Ping @theo

On the first part, I do agree, altough the documentation pointed me into believing that the binding, set to the default “autogen” would store into the default RP.
I can live with that.

Some more words on the second part.
When using REST API to display the entries, REST API posts a query to the database (at leat that is my understanding). If the RP is not set as default, the query needs to be specified with the RP-name. Obviously this isn’t done by REST API and therefore the query returns empty.
IMHO REST API should be using the RP, default or not, to specify the query.

And thanks for pinging the developer.

And just for me, when you say REST API, that’s the service on port 8086 the openHAB addon is talking to. Did you see the wrong query in a log?

I did assume, the webpage with the heading “openHAB REST API” would be refered to as REST API. Obviously that isn’t correct.
I’m doing the query from that local OH-webpage, as stated there it would be the same as calling:
"http://MyServer:8080/rest/persistence/items/MyItem_1?serviceId=influxdb"
So, I not using a service on port 8086 (which I don’t know).
And no, I did not see the query in the log, mainly because I don’t know which code-package I have to set in karaf to get those messages.
I concluded that from the comparison of the query result by the above mentioned webpage and the ones done directly in InfluxDB using the SELECT statements posted in the setup example.

Hi Jürgen,

I’m quite busy right now. Hopefully I’ve time to look into this this weekend.

Regards,
Theo

Hi All,
Tried to configure it for OH 1.8.3 and of course it doesn’t work. I enabled TRACE level for org.openhab.persistence.influxdb but it didn’t help, i see no errors but also no data is sent.

2016-10-28 21:36:46.900 [INFO ] [runtime.busevents             ] - WaterMeterCounter state updated to 5418
2016-10-28 21:36:46.901 [TRACE] [i.i.InfluxDBPersistenceService] - Tried to get item from item class class org.openhab.core.library.items.NumberItem, state is 5418
2016-10-28 21:36:46.901 [TRACE] [i.i.InfluxDBPersistenceService] - found no fractional part
2016-10-28 21:36:46.901 [DEBUG] [i.i.InfluxDBPersistenceService] - got DecimalType value 5418
2016-10-28 21:36:46.902 [TRACE] [i.i.InfluxDBPersistenceService] - storing WaterMeterCounter in influxdb value 5418, WaterMeterCounter (Type=NumberItem, State=5418)

Ok, time to review influxdb log. First of all i see successful select query at OH start

[query] 2016/10/28 18:35:00 SELECT value FROM openhab_db.autogen.WaterMeterCounter ORDER BY time DESC LIMIT 1
[httpd] 172.17.0.1 - openhab [28/Oct/2016:18:35:00 +0000] "GET /query?db=openhab_db&epoch=ms&p=%5BREDACTED%5D&q=select+value+from+WaterMeterCounter+ORDER+BY+time+DESC+limit+1&u=openhab HTTP/1.1" 200 46 "-" "okhttp/2.4.0" 3bc6c22f-9d3d-11e6-8010-000000000000 6179

I see configured RP in select query. In my openhab.cfg i added it two times just in case

influxdb:retentionPolicy=autogen
retentionPolicy=autogen

Now let’s compare save request from OH and manual from Influx web interface:

  1. Successful manual from Influx web interface
[httpd] 2016/10/28 18:38:33 Write body received by handler: WaterMeterCounter  value=5419.0 1477679749477000000
[httpd] 192.168.99.1 - admin [28/Oct/2016:18:38:33 +0000] "POST /write?db=openhab_db HTTP/1.1" 204 0 "http://192.168.99.100:8083/" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:49.0) Gecko/20100101 Firefox/49.0" bad36a35-9d3d-11e6-801d-000000000000 4417
  1. Failed from OH
[httpd] 2016/10/28 18:35:46 Write body received by handler: WaterMeterCounter value=5418.0 1477679746477000000
WaterMeterCounter value=5418.0 1477679746478000000
[httpd] 172.17.0.1 - openhab [28/Oct/2016:18:35:46 +0000] "POST /write?consistency=one&db=openhab_db&p=%5BREDACTED%5D&precision=n&rp=default&u=openhab HTTP/1.1" 500 72 "-" "okhttp/2.4.0" 57618773-9d3d-11e6-8011-000000000000 4022

The second request contains data 2 times and has additional parameters in request. Maybe problem is there?

P.S. Looked at code of InfluxDBPersistenceService.java for 1.8.3 and don’t understand how i can see RP in select query

Hi @opus,

your right the RP value is missing in the query. Could you please open a GitHub issue and I’ll prepare a PR.

Thanks,
Theo

Will do, however I’m not sure onto which section of Github. “openhab” or “openhab-addons” or???

[Edit] I’ve posted it here.

I’m using the Webview to show Grafana charts. Working on build #550 the switching from one chart to the other works on BasicUI now, However the Grafana part is showing problems.
I do show a single row from the dashboard as one chart for each switch setting . Whenever the chart is newly created the chart is shown correctly, however when just switching to a new chart only the header is shown and I must tap on the header and then select view to show the chart. Which is then shown in a less vertical size.
Does anybody have the same experience, am I using it wrong from the Grafana side,…?

Maybe you should have tagged @pahansen for that problem. :wink:

Hmm, I hadn’t noticed as i’d left that side project and moved to other things.

I think it might be associated with this bug in Grafana. https://github.com/grafana/grafana/issues/2874

Grafana V4 isn’t out of nightly pre alpha builds yet though. You can try and install a nightly build, and it looks like they’re preparing for a beta sometime very soon.

http://grafana.org/blog/2016/10/11/project-update--grafanacon/

Hi Jürgen @opus,

could you test if this bundle fixes the bug:
https://bintray.com/theoweiss/generic/download_file?file_path=rp1%2Forg.openhab.persistence.influxdb-1.9.0-SNAPSHOT.jar

I’ve also changed the default RP to “autogen” therefore users with older influxdb installations have to set retentionPolicy=default in influxdb.cfg.

Regards,
Theo

Will do the test, however I have to admit that I’ve never installed a bundle without PaperUI.
So correct me if I’m wrong, I’ll uninstall the present bundle via PaperUI and then put the posted one into the addons folder.

And for the default PR, as I understood having a “new” InfluxDB install the default RP can be set by using “autogen”?

I’m definitely not sure but I think this should work.

If you don’t set any retentionPolicy then “autogen” is used. If you want something else you have to set retentionPolicy=myRetentionPolicy. The drawback is: users of influxdb installations prior to 1.0 have to set retentionPolicy=default to get it working again?

It works, I’m just in the middle of the test!
[2. Edit] Changes in BOLD!
Now I get it, as I read it in InfluxDB “autogen” is the RP that is used if no RP is created. You can “see” that as a selectable RP in grafana. This one is always there and is not the user created RP. However that is my humble understanding.

As I understood so far in OpenHAB, the setting of “autogen” in the influxdb.cfg should select the user created RP. (As of my present test this works now.) **Wrong, I must have made a mistake here. **
OH will write to the RP that is set in the .cfg. If “autogen” is set, it will be written into it.

What do you expect if there is a default RP in the InfluxDB, however OH doesn’t write into it (Works). Should the query from the REST API website point to that RP or the default one? As I see it here, REST API is still looking for the DEFAULT RP.

[Edit:]
Finished to test so far.
-OH will always write into the RP that is set influxdb.cfg! If no RP is created “autogen” should be set.

-If a non-default RP is configured in the InfluxDB and set in Influxdb.cfg OH is writing into that RP, however when using the REST API website to query data, it will return only the items which are held in the DEFAULT RP!
As of my understanding that shouldn’t be like that.

How did I test:
1.) Existing user created Default RP “test”, influxdb.cfg had setting “retentionPolicy=test”, changed that to “autogen”.
Checked data entries direct in InfluxDB and via REST API. RP “autogen” is getting the data, not the user created DEFAULT RP!
2.) Created new RP “newtest”, changed setting to “retentionPolicy=newtest”.
Checked data entries direct in InfluxDB and via REST API. RP “test” isn’t getting data, all data goes to “newtest”. Different result from REST API is (IMHO) caused by false behaviour of REST API.

Seems like the upgrade to the 1.9.0.201611012307 openHAB InfluxDB Persistence bundle has not worked. I’ll try to figure out how it should be done.

I’ve found no way to upgrade the bundle :frowning: may be @Kai knows a solution? If I try

bundle:install -s 'https://bintray.com/theoweiss/generic/download_file?file_path=rp1%2Forg.openhab.persistence.influxdb-1.9.0-SNAPSHOT.jar'

in the karaf console I get the following error message:

org.apache.karaf.shell.support.MultiException: Error installing bundles:
	Unable to start bundle https://bintray.com/theoweiss/generic/download_file?file_path=rp1%2Forg.openhab.persistence.influxdb-1.9.0-SNAPSHOT.jar
	at org.apache.karaf.shell.support.MultiException.throwIf(MultiException.java:61)
	at org.apache.karaf.bundle.command.Install.execute(Install.java:116)[39:org.apache.karaf.bundle.core:4.0.4]
	at org.apache.karaf.shell.impl.action.command.ActionCommand.execute(ActionCommand.java:83)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:67)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:87)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)[56:org.apache.karaf.shell.core:4.0.4]
	at org.apache.karaf.shell.impl.console.ConsoleSessionImpl.run(ConsoleSessionImpl.java:270)[56:org.apache.karaf.shell.core:4.0.4]
	at java.lang.Thread.run(Thread.java:745)[:1.8.0_65]
Caused by: java.lang.Exception: Unable to start bundle https://bintray.com/theoweiss/generic/download_file?file_path=rp1%2Forg.openhab.persistence.influxdb-1.9.0-SNAPSHOT.jar
	at org.apache.karaf.bundle.command.Install.execute(Install.java:98)[39:org.apache.karaf.bundle.core:4.0.4]
	... 11 more
Caused by: org.osgi.framework.BundleException: Could not resolve module: org.openhab.persistence.influxdb [188]
  Unresolved requirement: Import-Package: org.openhab.core.items

	at org.eclipse.osgi.container.Module.start(Module.java:434)[org.eclipse.osgi-3.10.2.v20150203-1939.jar:]
	at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:393)[org.eclipse.osgi-3.10.2.v20150203-1939.jar:]
	at org.eclipse.osgi.internal.framework.EquinoxBundle.start(EquinoxBundle.java:412)[org.eclipse.osgi-3.10.2.v20150203-1939.jar:]
	at org.apache.karaf.bundle.command.Install.execute(Install.java:96)[39:org.apache.karaf.bundle.core:4.0.4]
	... 11 more

Otherwise we have to test it with the next openHAB snapshot.

I see the following timestamp 201611012307 for the influxdb persistence bundle, isn’t that correct?

How did you get this information?