InfluxDB+Grafana persistence and graphing

NOTORIOUS_VR · July 24, 2018, 10:46pm

Sorry but I followed this fantastic thread to a tee, and as far as I can tell Influxdb is installed, as is Grafana, all the .cfg files are configured.

But none of my logs show anything useful at all… Even adding extra logging info to org.ops4j.pax.logging.cfg does nothing (the log file doesn’t even get created!). It’s as if openhab can’t see influx or can’t use it.

Not sure where to go from here. I’ve double checked everything I can think of from this thread and others. I don’t see anything out of place.

Any assistance would be appreciated!

influxdb.cfg:

# The database URL, e.g. http://127.0.0.1:8086 or https://127.0.0.1:8084 .
# Defaults to: http://127.0.0.1:8086
# url=http://127.0.0.1:8086

# The name of the database user, e.g. openhab.
# Defaults to: openhab
# user=openhab

# The password of the database user.
password={REDACTED}

# The name of the database, e.g. openhab.
# Defaults to: openhab
db=openhab_db

# The retention policy to be used, needs to configured in InfluxDB
# Till v0.13: 'default', since v1.0: 'autogen'
retentionPolicy=autogen

infpluxdb.persist:

Strategies {
    everyMinute : "0 * * * * ?"
    everyHour   : "0 0 * * * ?"
    everyDay    : "0 0 0 * * ?"
}

Items {
    gHeatAct*, gHeatSet*, gHeatValve*   : strategy = everyChange, everyHour
    Presence_Phone                      : strategy = everyChange
}

running SHOW MEASUREMENTS or SERIES doesn’t return anything in the influx cmd line either, so it seems like nothing is actually being written to the DB

jswim788 · July 24, 2018, 10:54pm

Do you actually have Items and Groups with the names you list in influxdb.persist? I think those were meant as examples. You need to customize for your specific items.

NOTORIOUS_VR · July 25, 2018, 1:12am

Damn, I see what you mean now… I’ll check that right now and report back.

Thanks!

EDIT: That’s exactly what it was! Thank you for the tip!

m4rk · August 2, 2018, 12:37pm

Hi,
I have been researching how to setup InfluxDB and Grafana on my Openhab2 Raspberry Pi3b Stretch setup
and I came across this good read:

It wasn’t clear to me from this thread that I could install both InfluxDB and Grafana simply using the Openhab config menu

sudo openhabian-config

and select optional components.

Is there a reason this is not mentioned at the top of thread? Or to put it another way, what is best method to get InfluxDB and Grafana up and running on my Rpi3b Stretch?

I checked the link in this thread to RPi setup and it talks about Jessie. I assume for Stretch I just replace the words ‘Jessie’ with ‘Stretch’ for the correct install. Correct?

Thanks Mark.

rlkoshak · August 2, 2018, 2:41pm

openHABian is not the only way to install and manage an openHAB installation.

When this article was written, I don’t think openHABian supported installing InfluxDB and Grafana. I may be mistaken, but I’m not even sure there was an openHABian at that time.

The instructions are also generic enough such that should users want to host InfluxDB and Grafana on a VM or server separate from openHAB they can do it.

If you are using openHABian, use openhabian-config. If not, use the instructions above to install.

I’m pretty sure there is nothing else different between the two so that should work.

christoph_wempe · August 5, 2018, 8:09pm

This might be interesting for some of you.

Nognog · August 8, 2018, 7:25am

Hello,
is there a way to outsource the infludb to a USB stick?
And if so, how can i do that?

Greetz
Daniel

rlkoshak · August 8, 2018, 2:45pm

Google is you friend.

Note that a USB drive is no better than an SD card. It too will wear out with enough writes.

Hollako · August 10, 2018, 9:23am

Hey All,

A small help is needed please, a message in frontail logs keeps popping up from time to time:

2018-08-10 12:16:41.838 [ERROR] [org.influxdb.impl.BatchProcessor ] - Batch could not be sent. Data will be lost

java.lang.RuntimeException: {“error”:“timeout”}

at org.influxdb.impl.InfluxDBErrorHandler.handleError(InfluxDBErrorHandler.java:19) [205:org.openhab.persistence.influxdb:1.12.0]

at retrofit.RestAdapter$RestHandler.invoke(RestAdapter.java:242) [205:org.openhab.persistence.influxdb:1.12.0]

at org.influxdb.impl.$Proxy127.writePoints(Unknown Source) [205:org.openhab.persistence.influxdb:1.12.0]

at org.influxdb.impl.InfluxDBImpl.write(InfluxDBImpl.java:151) [205:org.openhab.persistence.influxdb:1.12.0]

at org.influxdb.impl.BatchProcessor.write(BatchProcessor.java:171) [205:org.openhab.persistence.influxdb:1.12.0]

at org.influxdb.impl.BatchProcessor$1.run(BatchProcessor.java:144) [205:org.openhab.persistence.influxdb:1.12.0]

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:?]

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:?]

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:?]

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:?]

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]

at java.lang.Thread.run(Thread.java:748) [?:?]

I’ve read the in this topic how to solve it and i added the "retentionPolicy=autogen"in the influxdb.cfg file the restarted the RPi but the problem still remain noting that all the functionality of Influx and Grafana are working properly.

Please advise !!

Thank you

Hollako · August 11, 2018, 6:53am

Hey All,

Problem solved by changing the SD card, a small hint for anyone may face this issue to use a better SD card with faster through put.

Regards,

Kotik · August 21, 2018, 4:18pm

Hello,

Which seems to be the best way of updating Grafana to the latest release (v5.2.2)?

I am using iframes to embed graphs and the Grafana version (v5.1.4) that is installed with Openhabian has a bug with iframes and a vertical scroll bar.

This is fixed in latest release.

Any suggestions are welcome.

P.S. I tried:
sudo apt-get update
sudo apt-get install grafana

and it reports that the latest version is already installed…

nakh_Home · August 21, 2018, 7:57pm

I do not know why but the image is really tiny

                Frame label="Chromecast" icon="chromecast" {
                   Default item=Chromecast_LG_ReleaseDate            	   	
                }
           Image refresh=60000 url="http://192.168.1.110:3000/render/d-solo/WF5DmCigk/a1?panelId=2&orgId=1&&from=now-48h&to=now&width=1000&height=500&tz=UTC%2B03%3A00"
        Frame label="Sensors-A1" icon="temperature" {		 
            Default item=mqtttemperature icon="temperature"
        }	
    }

Kotik · August 22, 2018, 3:43pm

To answer my own question, Grafana now officially supports armv7 and they have the latest version available for it:)

I had to run:
wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana_5.2.2_armhf.deb
sudo dpkg -i grafana_5.2.2_armhf.deb

and my Grafana updated to latest stable release that has the “iframe” scrolling issue fixed.

SmartHomer · August 22, 2018, 6:54pm

Hi Guys,

is there a possibility to build graphs out of openHAB for forecast data. I want to build a forecast graph for a power calculatioin for the next 5 days. In openHAB I have more than one Item. My problem is that I only can persist one Item which persist the past. But I want to show the forecast. And how can I put more than one Item in a line for graphing the forecast data?

Thanks!

5mark · August 25, 2018, 8:21pm

Thanks for this great tutorial. Its really amazing what you can do with grafana and openhab
May I suggest to add the possibility of using grafana’s kiosk mode to add your graphs as webview / frame to a dashboard to the tutorial?

I think its by far the easiest way to display your graph in HABpanel.

just add &kiosk=1 to the url of your dashboard. this will hide all of grafana’s menu items.

m4rk · August 27, 2018, 12:57am

Hi, I am relatively new to Openhab but I thought I’d share something here to inspire others.

A few months ago I really didn’t know anything about Openhab, Javascript or any programming, and still don’t . I don’t know much about Linux either nor am I comfortable with and console commands; but nevertheless with a lot of reading and support here I achieved something I am rather pleased with. So I’d thought I’d share and say thank you.

The screen shot below is taken from Grafana but the dials also shows dynamically in Habpanel. I also have the usual plots and charts for all my room thermostats (Openwebnet binding) but here I show some nice Grafana gauges displaying the dynamic data from my Mobile-Alerts wireless rain ,wind, temperature and humidity sensors. These sensors enable a relatively inexpensive wireless weather station to be built or monitor things like garage, fridges, fish tank, pools and ponds etc. They can be found under various brand names. Discussion: mobile-alerts-and-openhab

The dial pointers are animated and move in a nice slow steady manner to their position. I get email alerts and push notifications if its very windy or it started to rain.

I did it all using Openhabian on a Raspberry Pi 3b.

I got the sensor data from the internet using a combination of Mobile-Alerts REST API query and HTML caching of the Mobile-Alerts sensor web page. Which method I used depends on the sensor type. Only the pro-sensors have the option of the REST API.

In Grafana I used the D3 Gauge panel plugin and implemented Value and Tick mapping for the wind direction gauges. The rain total gauge is, like the charts, also dynamic and adjusts its value depending on the time period selected. To achieve this I used SELECT mean, difference and cumulative sum in the metrics as the mobile-alerts data is just a running total of rain fall since the sensor was activated.

So, thank you again everyone who helped me achieve this.

ps Just noticed an issue with the average wind direction. North directions are around 15, 0, 1 in number terms. So, when the wind is northerly and flipping between 0 and 15 results in average of 8 ie south grrr. Should work for other wind directions but North is a problem. Any ideas how to fix that would be appreciated. …Looks like I have to some maths and convert the wind angle to x and y components. Then average these x and y values and convert back to angle. I could use JavaScript to compute the x and Y using SIN and COS. Persist these values into influx and get influx to transform the average back to angle using ATAN I don’t see ATAN appearing in Grafana metrics options but there is talk of it for Influx … hmmmm.

That’s for unweighted average. For weighted then I need to factor in the speed. I think unweighted would be OK now .

Edited to add … it seems I can’t compute accurately the average wind direction due to Openhab and Influx limitations. Influx can’t do calculations with different measurements and Openhab can’t combine items into one measurement for persistence. I got most working but can’t compute the final part avgX / avgY

nakh_Home · August 27, 2018, 8:43am

Hi

Do someone has an idea why the image is tiny?

rlkoshak · August 27, 2018, 8:14pm

Can’t you create a new Item populated with a Rule? That’s the standard approach.

Often you can also put all the measurements into a Group using

Group:Number:AVG Measurements

The state of Measurements will be the average of all its members. You can then persist Measurements to InfluxDB and chart it normally.

m4rk · August 27, 2018, 8:17pm

Rich, I am often coming across your good ideas. That to me sounds like another one

m4rk · August 28, 2018, 6:17am

Having thought about it I don’t think that’s going to work.

I will explain the problem in more detail.

Ignoring wind speed weighting for now, the way to calculate average wind direction is not to use angles but to break down the wind direction into its component W_E and N-S directions, ie +/- X and +/- Y and then average these. This avoids the problem of for example 0 and 359 degrees both North averaging to 180 ie South. With the average X and Y dynamically calculated I would then need to convert them back into a final average wind direction like this:

atan ( avgX / avgY )

The averages are the dynamically calculated ones that Influx can do. I don’t think that Openhab group Items average is going to work in this case. Influx can also perform the atan function, I tested it, but it can’t divide X measurements by Y measurements. And I can’t get Openhab to persist the x and Y components as one measurement set as Influx fields, tags which could have solved the Influx divide problem. (This part I am learning so I could be wrong)

Thanks to your help elsewhere with type conversations, which I always struggle with, I got the code below to update Items with the latest X and Y wind direction components, and then persist these to Influx.I tried to get Influx to do the finally conversion back to an angle but as expected it didn’t work.

In my case the wind direction angle is in 22.5 degree steps from 0 to 15

logInfo("Mobile Alerts" , ' direction number = '+ MobileAlertsWeatherStationWindDirectionNumber.state)
val Number wdn =  MobileAlertsWeatherStationWindDirectionNumber.state.as Number
logInfo("Mobile Alerts" , ' wdn = '+ wdn)

val radians = Math.toRadians(22.5 * wdn.doubleValue)
val wdx = Math.sin(radians)
val wdy = Math.cos(radians)
MobileAlertsWeathStationWindDirectionX.sendCommand(wdx) 
MobileAlertsWeathStationWindDirectionY.sendCommand(wdy)

The average wind direction gauge is just a bit of fun. I like to see the animation of the dial pointers moving slowly into place The chart below is actually a more informative way to display what the wind has been doing. You can easily see the wind changed to a general new direction or if its just random gusts. You can also see the 0,360 problem as the data appears to split but actually is the same wind direction.