InfluxDB+Grafana persistence and graphing

influxdb
graphs
grafana
Tags: #<Tag:0x00007f51e0aa0a30> #<Tag:0x00007f51e0aa08f0> #<Tag:0x00007f51e0aa0760>

(Rich Koshak) #709

The end result is the same but the explanation isn’t quite right. PhantomJS, being a JavaScript library, cannot throw an Java error. The problem is for some reason PhantomJS consumes so many resources that the Linux kernel freaks out and looks for something to kill to accommodate PhantomJS. For some reason it chooses openHAB, probably because openHAB is the only other process consuming a lot of resources which is running at the same or lower priority level as PhantomJS.

This is honestly the first time I’ve ever seen the kernel freak out and kill some other process like this. The logical approach would be to kill PhantomJS. But if for some reason openHAB is running at a lower priority level than PhantomJS I supposed that would explain it.

The cloud connector only proxies OH’s web server and OH’s REST API. When you use a generated static image chart that image gets served up to your sitemap using openHAB’s web server. When using a webview you are showing content from a different web server (Grafana’s server in this case). The cloud connector cannot proxy the connection to an external web server.

So if you want access to charts outside of your LAN you need to:

  • put Grafana with PhantomJS on some beefy machine, use the static image URL, and hope for the best
  • write your own external script to pull down the charts from Grafana and put the images in your html folder
  • expose your Grafana to the internet

I think the OP is a wiki.


(Kim Andersen) #710

As I recall, my log showed Java was killed, when PhantomsJS threw an error. When Java was killed, openhab stopped as well. But the end result is the same, as you say. Openhab restarts.

Yeah, I had a feeling that would be the case. Thats why I gave up and found another solution…

…which I did.
I installed Grafana on my old WindowsHomeServer2011 last night, (old HP SmartHomeMedia Server with an dual core Intel E6320 and 4GB of RAM). It works, but it sure takes alot of resources on that one as well. CPU goes 100% when rendering. But no crashes of openhab :slight_smile:


(Rich Koshak) #711

The error and the killing came from the kernel in response to what PhantomJS was doing. It was definitely the kernel’s doing.


(bccrew) #712

Hi everyone,

I’m using the broadlink binding to monitor humidity and temperature in a room and I would like to build a nice graph with influxdb and grafana.

As you can see in the events.log the device put data every time the value has changed. So if the temperature stays the same for 20 minutes, no log entry is created. The binding has a parameter that can be set to x seconds for polling the device.

How can I force that a persistence file updates the influxdb every x time? For example, every 5 minutes (even though the temp or humidity is still the same)

==> /var/log/openhab2/events.log <==
2019-01-18 09:59:03.657 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.1 to 42.0
2019-01-18 09:59:34.113 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.0 to 42.1
2019-01-18 10:09:37.200 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.1 to 42.0
2019-01-18 10:12:07.500 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.0 to 42.1
2019-01-18 10:13:07.595 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Temperature changed from 19.299999237060547 to 19.200000762939453
2019-01-18 10:13:37.679 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Temperature changed from 19.200000762939453 to 19.299999237060547
2019-01-18 10:14:37.779 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.1 to 42.0
2019-01-18 10:16:37.956 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Temperature changed from 19.299999237060547 to 19.200000762939453
2019-01-18 10:16:37.961 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.0 to 42.1
2019-01-18 10:20:38.788 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.1 to 42.2
2019-01-18 10:21:08.824 [vent.ItemStateChangedEvent] - BroadlinkA1192168112_Humidity changed from 42.2 to 42.1

(Rich Koshak) #713

(bccrew) #714

@rlkoshak
I have checked that documentation already but I still don’t see how I should achieve my goal.

Just posting a link to a a documentation page is not very (user) friendly imho. If you can help with the specific problem, please do. But if you don’t want to or cannot, don’t just post to post… I know LMGTFY or RTFM but if that still is not clear enough for me to understand, isn’t that what asking for help or a forum is meant for?


(Rich Koshak) #715

Persistence Configuration

The information below allows you to determine which Item states are persisted, when they are persisted, and where they are stored.

Persistence Triggers

The persistence of an Item’s state may be triggered when that Item changes state or when the Item is updated (even if its state did not change). Persistence may also be triggered by a time-related event (see Cron Persistence Triggers below).

Cron Persistence Triggers

openHAB uses Quartz for time-related cron events. See the Rules article for more information.

Below you will find a complete example persistence configuration file:

// persistence strategies have a name and definition and are referred to in the "Items" section
Strategies {
        everyHour : "0 0 * * * ?"   // <--------
        everyDay  : "0 0 0 * * ?"  // <--------

        // if no strategy is specified for an Item entry below, the default list will be used
       default = everyChange
}

/*
 * Each line in this section defines for which Item(s) which strategy(ies) should be applied.
 * You can list single items, use "*" for all items or "groupitem*" for all members of a group
 * Item (excl. the group Item itself).
 */
Items {
        // persist the Item state of Heating_Mode and Notifications_Active on every change and restore them from the db at startup
        Heating_Mode, Notifications_Active: strategy = everyChange, restoreOnStartup

        // additionally, persist all temperature and weather values every hour
        Temperature*, Weather* : strategy = everyHour   // <---------------------
}

Everything I would have typed in to answer your question are right there in the documents.

So I could have spent 10 minutes retyping the exact same thing that is in the docs already, or I could point you to the part of the docs that gives you the exact answer to your question and spend that 10 minutes helping someone who has a question that isn’t already pretty fully documented.

I’ve bolded the most important text and added // <------ to point to the lines in the comprehensive example that specifically answer your exact question.

Just asking a basic question without showing any indication that you have even looked at the docs, done a search of the forum, or even looked at the example in the OP, shows a lack of respect for the time of the volunteers spend on this forum trying to help users, IMHO.

If you DID look at the docs before posting your question, then say so in your original question. Explain what your confusion is and where the docs are not clear. Then we can actually fix the docs to make them more clear.

We don’t have enough volunteers here to spend hours and hours rewriting the docs as a reply every time a user asks a question whose answer is in docs.


(bccrew) #716

My apologies,

A part of my original question was lost. I thought my persistence file was there but clearly it isn’t…

I currently have this:

Strategies {
    everyMinute : "0 * * ? * *"
}
Items {
    BroadlinkA1192168112_Temperature  : strategy = everyChange, everyMinute, restoreOnStartup
}

Seems to be working now, the problem was the refresh rate of the webif of grafana


(Aymeric) #717

Perhaps this tutorial needs to be updated a bit:

Raspberry PI is now officialy supported so I switched to the official repo instead of https://github.com/fg2it/grafana-on-raspberry.

When doing so, anonymous access was not working.
It needed two more changes in the configuration ini file to work with the latest version

[auth.anonymous]
# enable anonymous access
enabled = true
# specify organization name that should be used for unauthenticated users
org_name = YOURORGANIZATION
# specify role for unauthenticated users
org_role = Viewer

I don’t remember when the org name is set, but you can find it in the grafana settings, under server admin menu.

Aymeric


(Martin) #718

A long thread …

I have running influx and grafana on my Raspberry, and was able to see a “live” graph (5s intervall) on a website (local PC), URL like:
http://192.168.:3000/d/Il-rTUzRk/e-power-kwh-live?orgId=1&panelId=2&fullscreen&from=now-5m&to=now&refresh=5s

I will use this “live graph” on a sitemap (actuall no habpanel), i this possible? How :wink:
Or do I need that Grafana create png-files and I have to show these (but: not easy for a 5s intervall …)

Also: On the PC I have also to login into Grafana to see the graph, is there a “good” way that this not needed?


(John Schmitz) #719

The persistence for InfluxDB does impose some limitations: you can’t use the tags and the multiple values and different retention policies in InfluxDB. For example, you could have a measurement “Temperature” and the tags “Inside” and “Outside”. Or maybe you want some measurements saved forever and others for much less time. You can use continuous queries to down sample, but it doesn’t cover every case. Also, you can have a measurement “SensorValues” with values such as Temperature and Humidity all within that measurement. Fortunately there is quite an easy workaround with the HTTP post request. I’ve put a sample below for anyone who is interested. If you’ve used the authentication above, you’ll need to add it to the http val. And of course modify it to your liking.

The InfluxDB write documentation is here

I use a timer to not bog down the rules when the post request is running.

val String INFLUXDB_WRITE = "http://192.168.86.212:8086/write?db=openhab_db&rp=autogen"
var Timer influxWriteTimer = null

rule "End Peak"
when
    Time cron "0 0 20 ? * MON-FRI *"
then
    if (influxWriteTimer === null) {
        influxWriteTimer = createTimer(now, [|
	    sendHttpPostRequest(INFLUXDB_WRITE, "application/json", "KWHPeak value=" + KWHPeak.state.toString, 3000) // 
            influxWriteTimer = null
        ])
    } else {
        logWarn("eagle", "Influx write timer is still running, not creating another")
    }
en

Persistence: Possible to specify both time and value with postUpdate?
(hendrik ) #720

I want to implement a “Direct Link Rendered Image” into my Habpanel.

Problem is, that the background is black from the dark theme or white from the light theme.

In the thread of the matrix theme I saw that it should be possible to create a css file, which I have to implement into the index.html file under directly below the title tag.
I did this, but without success.

Is there somebody who solved this?


(Arne) #721

In Installed influxdb and grafana according to the describtion above, version 1.0.2. was installed (using an Pi 3+).
But i cannot go to the command line interface?

When I run influx, the answer is command not found.
I can start somethin running influxd. Then i get this:

[14:59:28] openhabian@openHABianPi:~$ influxd

8888888 .d888 888 8888888b. 888888b.
888 d88P" 888 888 "Y88b 888 "88b
888 888 888 888 888 888 .88P
888 88888b. 888888 888 888 888 888 888 888 888 8888888K.
888 888 "88b 888 888 888 888 Y8bd8P’ 888 888 888 "Y88b
888 888 888 888 888 888 888 X88K 888 888 888 888
888 888 888 888 888 Y88b 888 .d8"“8b. 888 .d88P 888 d88P
8888888 888 888 888 888 “Y88888 888 888 8888888P” 8888888P”

[run] 2019/02/10 14:59:29 InfluxDB starting, version 1.0.2, branch unknown, commit unknown
[run] 2019/02/10 14:59:29 Go version go1.7.3, GOMAXPROCS set to 4
[run] 2019/02/10 14:59:30 Using configuration at: /etc/influxdb/influxdb.conf
run: open server: listen: listen tcp :8088: bind: address already in use

but why cant i go to the cli?


[SOLVED] MQTT & influx, no datapoints in grafana
(Udo Hartmann) #722

influxd is the daemon (and already started… bind: address already in use)
if there is an influxd, there also should be an influx (which is the connection to the database command line)

Maybe you have to use

sudo influx

instead.


(John Schmitz) #723

I would try this. I recently installed influxdb and got only the server and not the client on Ubuntu 18.04 server. Perhaps they have separated the server and the client in the latest package.

sudo apt-get install influxdb-client

(Andrew Rowe) #724

finding the influx command was what hung me up trying to do this tutorial. Eventually I found info telling me to point my web browser at the web interface which can be found at
localhost:8083
after install
the web interface allows the creation of databases, users and permissions
hope it helps


(John Schmitz) #725

Port 8083 for admin can work, but you need to enable it first in the influxdb.conf file in the admin section as the default is disabled. Also, it says that the interface is deprecated and will be removed in a future release.


(Martin) #726

Hope this is the correct location for my two questions …

I measure few temperatures, store the values every 5 minutes in an influxdb.
Now, the graph for 1 day (or less) view has “steps” because of less values.
But is there an option that Grafana interpolate (create a curve betwen the point) and the graph looks “smoother” ?
(Otherwise I have to save every minute the temperatues into the influxdb …)

And in Grafana under “Legend” I activate “Show” and “As table”:
But this will only show (the first) three temperature sensores I added (in the graph there are six shown, thats correct). See picture
When activating additionally “To the right”, than all six values are shown on the right part from the screen (but than the graph is to much shrinked). Is this a limitation, a Bug or a mistake on my side?


(John Schmitz) #727

For the first question, select “connected” for the null value in the Display tab in Grafana. There’s a picture here.


(Martin) #728

OK, for my second question I found out:
When moving over on the right part, a scroll line appear!
Without move-over there is NO hint that there are more lines!

But in my case it would bether to see all 6 lines directly, is there an option?