Persistence & Graphs - why so complicated?


(Daniel Hermann) #1

Hello,

I am sorry if this is the wrong section, and I am also sorry if this is seeming too much as a rant, but maybe there is something I am just not understanding.

Why is there a modern home automation system without persistance and graphs included as default? It just does not make any sense to me.

Before I used Domoticz, where Graphing for every device (Item here) is done by default. when on the device just click on “history” and it shows you a nice graph. Graphing of different Items is also supported out of the box.

Fast forward a bit, I switched over to Openhab (mainly because of the fast and easy support for Google Home), and what about persistance? Graphs? You have to set them up.

I tried multiple tutorials in setting up persistance, and failed. I am sure that is my error somewhere, and I will get there, but why even make it so complicated and user-unfriendly? If you do not want to ship it by default (modularity and everything) then why not make it a one-klick setup? I do not care if it is logged to a mysqldb or to postgre or influx. I just want it to work. And i think 90% of the potential users want it that way.

I mean you can still use alternative databases and visualization tools, but why not make one way that works with a “one-click” setting, as in my opinion that is one of the core features of any automation system.

Thanks
Daniel


(Sebastian) #2

Hey Daniel,
OH2 is very flexible and can be configred the way everyone needs in his own setup. Basically I do not want to push every single bit & piece into a database since I dont need it. Thus I like the ability to configure it the way I like it. The OH2 docu is under heavy construction right now and lots of tutorials are added already. For the persistance tutorial the page has been created, bu no content yet.
http://docs.openhab.org/tutorials/beginner/persistence.html

Here’s a short list of action items you an try to set it up:

  • PaperUI > Add-ons > Persistance. Install RRD4J Addon

  • add a local config file: /etc/openhab/persistance/rrd4j.persist

  • The content should look like this:
    Strategies {
    // for rrd charts, we need a cron strategy
    everyMinute : “0 * * * * ?”

          default = everyChange
    

    }

    Items {
    * : strategy = everyUpdate, everyMinute
    }
    Remark: The * will persist ALL your item. If you dont want that, take a look at this best practive which I can really recommend

  • Now you will see that RRD databases are populated in /var/lib/openhab2/persistence/rrd4j

  • To see a graph you have to add an item to you’re sitemap like this:
    Chart item=item_name period=D refresh=30000

To have it nice and sexy, I can recommend the following lines for your sitemap:

            Frame {
                Switch item=chart_period label="Graph Zeitraum" mappings=[0="Tag", 1="Woche", 2="Monat", 3="Jahr"]
                Chart item=gOG_temp period=D refresh=30000 visibility=[chart_period==0, chart_period=="Uninitialized"]
                Chart item=gOG_temp period=W refresh=30000 visibility=[chart_period==1]
                Chart item=gOG_temp period=M refresh=30000 visibility=[chart_period==2]
                Chart item=gOG_temp period=Y refresh=30000 visibility=[chart_period==3]
            }

You will need an additional item called chart_period which will be displayed along with the graph. By choosing the Value for chart_period, the graph below will update accoring to your requested time frame.
Here’s the item definition:

// To choose a proper cahrt period in a sitemap
// mappings=[0="Tag", 1="Woche", 2="Monat", 3="Jahr"]
Number chart_period
    "Graph Zeitraum"

Not too difficult, right?
Any questions, let us know.


Visibility is not working as expected
(Matthew Clegg) #3

Just the ticket that. Cheers for sharing


(yves Hanoulle) #4

Just like Daniel, I’m having trouble figuring out charts.
This answer feels like the start of the persistence documentation.

One thing I still can’t get my head around is, where in this example is it defined what to show on the chart?

I probably just look over it, yet where in this example is written to show Essz. Temperatur 1 Essz Temperatur 2 etc


(Jimbob) #5

Hi @yves,

I know the feeling, I’ve got an example like the one above which works, though not completely (but does as I need so I haven’t investigated further).

The item selected is that which follows the
’’‘
Chart item=
’’‘
In this case ‘’‘gOG_temp’’’, there’s a common preference to name items beginning with a lowercase letter denoting their type “g” in this example looks to be a group agroup contain the four temperature items shown in the graph.

The clever bit in the example is showing how to display different graphs of different time frames depending on the value of the item ‘chart_period’ which perhaps over complicates the example.

Hope that helps.
Cheers
James


( ) #6

The basic chart widget is nice for some quick diagrams but I would not even bother and go with a more powerful solution. Just as @waitz_sebastian said, openHAB offers to be flexible. My recommendation would be to use InfluxDB+Grafana from the beginning.

That said setting up InfluxDB and Grafana takes a few minutes.

@yves are you new with openHAB? Persistence is something like the next level and barely needed for the pure home automation aspect of things. On the other hand is what and how data is persisted highly dependent on the user, hence there can’t be a general all-satisfying solution.
Your question regarding the “Essz. Temperatur 1” makes me believe, that you did not yet understand how items and sitemaps work in openHAB.

I’d recommend to first get used to openHAB before looking into the persistence topic. Start here:

Browse through the two documentation articles, then work your way through the astro example and tinker a bit. You’ll learn all you need to know in the process. Good luck!


(yves Hanoulle) #7

I have been using Openhab for little over 2 years. is that new to you?
I know very well how Items and sitemaps work. yet the answers stated that everything was in there. I don’t see a mention of anything of Temperature.

The answer of @waitz_sebastian says that the items are part of the group. That was the crucial information for such an example I needed.

Still does not fix my problem.


( ) #8

Ah, I see! Sorry for that -.-

So what’s not working as expected? Instead of the group item you can just add one normal item to be charted. Maybe this will get you going


(Sebastian) #9

I moved away from RRD as Grafana is much more superior. I would also recommend going that path.

I did not get your issue. Can you elaborate a bit more in detail?


(yves Hanoulle) #10

I might go to grafana, yet if I can’t get a simple example to work with a graph, that would be an unwise decision.

At this moment nothing is shown in the graphs. Also not with one item, which makes me assume the saving part is not working. yet the rdd file does exist…


(yves Hanoulle) #11

I found more out of my problem.

in the logging I see
02:30:06.491 [WARN ] [thome.ui.internal.chart.ChartServlet] - Illegal argument in chart: Null datasource type specified
I had a few rrd file, yet I noticed they had not been updated in a while.
I read somewhere that when types are changed, there might be a problem so I deleted the .rdd files.

It was not until I made changed to the cfg for rdd4j and a reboot of openhab services that the files started to be recreated (I had gauge instead of GAUGE)

Strangly it seems that the rdd service stil tries to launch also using old config data, I see he is using als old dsname with gauge.

Now I can see the real problem with my charts not loading

Now I get the error
03:13:04.223 [WARN ] [thome.ui.internal.chart.ChartServlet] - Illegal argument in chart: Series name cannot be null or zero-length!!!

where this is in my sitemap:

Switch item=Weather_Chart_Period label=“Diagram” icon=“temperature” mappings=[0=“Hour”, 1=“Day”, 2=“Week”]
Chart item=Weather_Temperature refresh=600 period=h service=“rrd4j” visibility=[Weather_Chart_Period==0, Weather_Chart_Period==“Uninitialized”]
Chart item=Weather_Temperature refresh=3600 period=D service=“rrd4j” visibility=[Weather_Chart_Period==1]
Chart item=Weather_Temperature refresh=3600 period=W service=“rrd4j” visibility=[Weather_Chart_Period==2]


(yves Hanoulle) #12

when I changed inside my items file
"Number Weather_Temperature “[%.1f °C]” (gWeather_Chart, gChart) {channel=“yahooweather:weather:4e76379f:temperature” }

to
Number Weather_Temperature “Temperatuur [%.1f °C]” (gWeather_Chart, gChart) {channel=“yahooweather:weather:4e76379f:temperature” }

I no longer get the " Illegal argument in chart: Series name cannot be null or zero-length!!!"
I now get another error

16:44:32.012 [WARN ] [eclipse.jetty.servlet.ServletHandler] - /chart
java.lang.NullPointerException
at sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1264)[:1.8.0_112]
at sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:219)[:1.8.0_112]
at sun.awt.FontConfiguration.init(FontConfiguration.java:107)[:1.8.0_112]
at sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:774)[:1.8.0_112]
at sun.font.SunFontManager$2.run(SunFontManager.java:431)[:1.8.0_112]
at java.security.AccessController.doPrivileged(Native Method)[:1.8.0_112]
at sun.font.SunFontManager.(SunFontManager.java:376)[:1.8.0_112]
at sun.awt.FcFontManager.(FcFontManager.java:35)[:1.8.0_112]
at sun.awt.X11FontManager.(X11FontManager.java:57)[:1.8.0_112]
at sun.reflect.GeneratedConstructorAccessor236.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)[:1.8.0_112]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)[:1.8.0_112]
at java.lang.Class.newInstance(Class.java:442)[:1.8.0_112]
at sun.font.FontManagerFactory$1.run(FontManagerFactory.java:83)[:1.8.0_112]
at java.security.AccessController.doPrivileged(Native Method)[:1.8.0_112]
at sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)[:1.8.0_112]
at java.awt.Font.getFont2D(Font.java:491)[:1.8.0_112]
at java.awt.Font.defaultLineMetrics(Font.java:2176)[:1.8.0_112]
at java.awt.Font.getLineMetrics(Font.java:2283)[:1.8.0_112]
at java.awt.font.TextLayout.fastInit(TextLayout.java:598)[:1.8.0_112]
at java.awt.font.TextLayout.(TextLayout.java:393)[:1.8.0_112]
at org.knowm.xchart.internal.chartpart.Legend.getSeriesTextBounds(Legend.java:302)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at org.knowm.xchart.internal.chartpart.Legend.determineLegendBoxSize(Legend.java:84)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at org.knowm.xchart.internal.chartpart.ChartPainter.paint(ChartPainter.java:112)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at org.knowm.xchart.Chart.paint(Chart.java:102)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at org.eclipse.smarthome.ui.internal.chart.DefaultChartProvider.createChart(DefaultChartProvider.java:228)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at org.eclipse.smarthome.ui.internal.chart.ChartServlet.doGet(ChartServlet.java:255)[136:org.eclipse.smarthome.ui:0.9.0.b5]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)[18:javax.servlet-api:3.1.0]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)[18:javax.servlet-api:3.1.0]
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)[81:org.eclipse.jetty.servlet:9.2.19.v20160908]
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)[81:org.eclipse.jetty.servlet:9.2.19.v20160908]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71)[172:org.ops4j.pax.web.pax-web-jetty:4.3.0]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)[79:org.eclipse.jetty.security:9.2.19.v20160908]
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:287)[172:org.ops4j.pax.web.pax-web-jetty:4.3.0]
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)[81:org.eclipse.jetty.servlet:9.2.19.v20160908]
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80)[172:org.ops4j.pax.web.pax-web-jetty:4.3.0]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.Server.handle(Server.java:499)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)[71:org.eclipse.jetty.io:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[83:org.eclipse.jetty.util:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[83:org.eclipse.jetty.util:9.2.19.v20160908]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_112]

Not sure what this is about.Seems to imply there is a problem with a font. yet I don’t know where that error occurs


(Sebastian) #13

I’m not using Yahoo weather but the item config looks ok to me.

Ok, first you have to make sure your data is persisted properly. Once this is confirmed you can focus on displaying the data.
I guess your RRD.persist is configured for gChart, right?
Then, stop OH, remove the rrdfile and start OH again.
Now the file should be recreated and updated as often as your weather item is configured to update. Of corse in case your persist config is correct (everyUpdate).
Let the RRD update a few times. Keep in mind the size of the rrd file wont change. But “ls -al” will show you the last modified time.
Check the OH logs for errors. In case there are none and the file gets updated, confirm here to do the graphing part.


(yves Hanoulle) #14

I guess I was not clear. I did see that he did not update the database before, and that was fixed when I wrote my last two updates. so yes no errors for logging and multiple rdd files are updated every minute.
So the errors I see it for opening the page with the chart


(yves Hanoulle) #15

As I could not find it, I did set up a separate raspberry PI with influxdb and grafana and that seems to work with same data written out. (And of course a separate dashboard)
I think something might be wrong with dashboard in the latest (2,1) version.


(Jeff Haskel) #16

I had this working or a variation very close before the update to v2.3.0 m any pointers why it may have stopped


(Frank Klein) #17

Guess this logging and graphing discussion is outdated, but glad that the issue is finally overcome.
Finally adding a persistance Add-on results in automatically creation of a default config file and adding the logging for an item is done by clicking “enable logging” per item in PaperUI similar to most other SmartHome Solutions. So finally OpenHab doesn’t only look like a userfriendly successor to FHEM but also caught up to modern OpenSource alternatives like Domoticz , IOBroker etc. Not to mention most commercial solutions - beside Qivicon :wink:

Oh wait, guess I fall asleep some year ago and it was all just a dream :frowning:
OpenHab 2.4 is still stuck halfway with what feels like an alpha version of 2.x with a bugged PaperUI to get Beginners caught and this legacy Sitemap stuff to drive them away again - ok, maybe next version…