New charting solution for OH - simplified installation (modified)

Add-ons Other
1

This post is modified to include these enhancements:

  • no restriction any more about the charting period (the charting has been enhanced) (Aug 04)
  • intchart.rules has been made platform independent (to be confirmed on linux) (Jun 17)
  • the url pattern has changed and it is now possible to access data file through the web (Jun 17)

Hi,

In a previous post I described a new charting solution offered by intchart.
I must admit that the installation procedure was rather complicated.
That’s why below you can find a simplified procedure.

The solution now offers more flexibility :
It can be used alongside all other functionalities in OH (also webviews).
And multiple charts can be drawn, even on the same page.

To have an idea of what the charts looks like, have a look at www.intchart.com.

The charting solution is proposed for OH 1.8.x. , a solution for 2.0 is being looked at.

These are the installation steps.

1. Install the files

The files needed can be found in this zip file :

  • intchart.rules
    To be installed in : ...\openhab\configurations\rules

  • org.openhab.ui.webapp_1.8.x.jar
    To be installed in : ...\openhab\server\plugins

All you need for the logging is included in this one rule file (intchart.rules).

In the jar, the only file changed is : snippets\main.html :

  • An EventListener ‘beginslide’ is added.
  • In the EventListener ‘endslide’ a block is added.

(For now, the webpage including the chart is not refreshed.)

2. Customize the logging rules

In intchart.rules, at the bottom, the following block can be customized.

rule "Logging" // enter unique rule name
when 
	Time cron "0/10 * * * * ?" // every 10 seconds 
	//Time cron "0 0/10 * * * ?" // every 10 minutes
then
	var fileName = 'data.txt' // enter unique file name
	var itemList = newArrayList(Weather_Temperature,Weather_Humidity) // enter items to log	
	var nameList = newArrayList('Temperature','Humidity') // enter legend names
	var round = 1 // enter value decimals

	count = new Integer((intchart_log.apply(fileName,itemList,nameList,round,count)).toString) // function (not to be changed)
end

In this block you should give

  • A unique rule name

  • The logging period

  • The four variables indicated
    As example the items temperature and humidity from the demo are used.

For every chart you need, you should duplicate this logging block.

3. Include frame in Sitemap

In the sitemap, include the following frame:

Frame {
    Webview url="https://www.intchart.com/api/open_HAB/chart?file=data/data.txt&width=1200"
}

This link will create the charts.
The following parameters can be defined:

For every chart, of course, you need a new link.

So, this is, I think, a much more simplified procedure.
If you have any problems with this installation, do not hesitate to contact me.

All other feedback is also welcome.

1 Like
2

Now I’m the first person to push back against the creation of new bindings. But this looks a lot like a good candidate for a binding (persistence binding?), or at least implemented as a servlet to run alongside the native charting servlet. Is there any reason why you chose to implement this almost in rules?

Is the webapp binding jar something you wrote or something more generic? If it is something you wrote you should probably name it something a little less generic.

Have you considered hosting this on github or some other similar place where the community can help improve and enhance it?

Would it be possible to use the logging persistence binding rather than the logging rule and lambda? That seems a more appropriate way to get the data logged out.

If we use the “openHAB” user ID I assume that means that all the data we send to intchart using this solution becomes publicly accessible. Is it possible to lock the information down so only I can access it? Many on this forum are hesitant to trust their information with a cloud service at all, let alone one that anyone can see the data.

I really like this and everyone knows that OH needs better charting and this is the first concrete and complete example I’ve seen posted of a working solution to use a third party charting service.

If I have time I’ll root around in that zip file and see how it works and maybe have some more concrete comments and suggestions as opposed to just questions.

3

Hi, I like very much your idea of a binding. The reason why I chose for an implementation in rules, is that I’m more familiar with these rules, and it was a faster first proposition. But, of course, I am open for any other (more efficient) solution.

The webapp jar (org.openhab.ui.webapp_1.8.x.jar) is an existing jar used for the Classic UI in openHab 1.8.x , modified to enable the presentation of charts simply by using a Webview and url. Only a few lines were added, which I expect to change nothing to the other functionalities of the Classic UI. This was tested, but of course it would be nice to have some confirmation.
Meaning that, for an eventual permanent solution, this should be provided by default in the installation of OH 1.8.x. For now, these files are included in the zip file, so that it is easily implemented by anyone who want to try these charts.

About your github question, there is nothing secret about the practical implementation solution for OH and the way how to integrate the charts. The way the charts themselves are generated are propriety of intchart.

There is a reason why a lambda function is used instead of a persistence binding. The charts are generated from a simple text file in a certain format which should be created (in no matter what way). However, in my first proposition, when I used the logging with the logback.xml, I experienced that this file was locked permanently by the logging, and therefore the chart couldn’t be refreshed (by refreshing the browser). With the lambda function, the file is locked only temporarily during writing, and now when refreshing the browser, or by going back and forward with the screens, the chart has the latest data. This was the easiest solution I could think of, but of course, if someone has a better idea this should certainly be tried.

About the “openHAB” user ID, perhaps some clarification is needed.
First of all, for the implementation for OH, no data is transferred, the generation occurs in the browser with the data found by the local OH host. So, no data is leaving OH : this is the easiest and most accessible solution, and this way a public user ID can be used without problem.
Of course, if you are registered and have your own user ID, you get your own private area at intchart. Encrypted data can be transferred back and forth by using the API of intchart, and you can watch your charts on the site. You can also integrate this chart in whatever website just by using a link similar to the one used in the OH webview (without setting the path parameter) data stored at intchart is used. But again, this an upgrade from the OH implementation.

Any further comments are certainly welcome.

4

Just for information, while I haven’t read through this fully, a binding is probably not the right thing to do. When the chart servlet was originally written, it was written with expandability in mind, so I added an interface that can be used to allow other charting systems to be employed.

The default chart system was designed to fit within OH/ESH rules for the use of libraries etc with open licenses, but there’s nothing to stop you adding a new chart servlet that replaces this. The servlet will be called with the parameters required for the chart, and responds with an image file that can be sent to the UIs.

From memory (and it’s been a couple of years since I wrote this!) the configuration is defined in the documentation (probably the sitemap docs since it was related to changing the chart servlet) and you should find the source for the current servlet in the ESH source.

5

Chris, thanks for this info. It’s certainly useful to have some ideas about the way to proceed.
It’s mainly for the logging part that some optimization is needed. In above procedure I proposed a lambda function to be used in rules to log the data to a text file. I was wondering if this could be done with some kind of binding in a ‘persistence style’. The variables for the logging would be:

  • the file name
  • the items to log
  • the names for these items
  • (eventually the precision)

An important feature would be that this logging shouldn’t lock the file (that’s why the logback isn’t used). That way the chart can be updated.

6

My reference for the binding was primarily for the writing the data out to a file part, not necessarily the actual charting part. Having the logging of the data in a lambda doesn’t feel right to me for something that would be released to the broader community.

However, if a new charting servlet can just query the persistence to send the data on to the charting service, so much the better. @Vincent, perhaps there is something you could do within the servlet to generate the file that gets passed to the service on demand, or create something akin to the logging persistence binding that does not keep a lock on the file.

Concerning the locking, I admit it has been a long time since I’ve messed with files like this in programming, but I thought that when a service locks a file it only prevents further writes but others can still read. This is why I can, for example, tail -f openhab.log. Is intchart opening this file RW? If so that is the real root of the problem. They should not be writing to these files IMO.

7

Shouldn’t that be implemented as a persistence service then?

8

I’m probably misusing the word “binding”. Indeed, I do mean a persistence service.

9

Does there exists in OH a persistence service logging to a text file that could eventually be used for this?

10

This is what I originally asked in my initial reply as to why you were not using it.

It is based on logback logging service and it should give you the ability to format the files however you need. There is still the file locking issue but the more I think about it the more it smells fishy to me. The only reason intchart should have a problem with the lock is if it is opening the file RW instead of just R and intchart should not be writing to these files.

11

Intchart is certainly not writing to the files. It’s based on an ajax call to the files stored at OH, and apparently this logback didn’t allow a refresh. I didn’t go into detail about the why of this.
Also,when I looked at the logback some weeks ago, I didn’t manage to get the right format for the file (like in my first propostion), and therefore I had then a mixture of rules and logback.

12

Hi Vincent,

I’ve implemented intchart as per your instructions and have created a new user account on IntChart as it is saying that the OpenHab user doesn’t exist anymore.

When I try to view the chart in OH 1.8.3 I receive:
File data.txt is not found with url : https://www.intchart.com/api/g7Ztjc0D/chart/data/data.txt?_=1466596567369

I can see the text file exists in webapps/data/data.xml so must be missing the step where the api knows where to look for the file.

13

Hi Rob,

First of all, I would try to use the same file name : in the url and the physical file.
In the url you have data.txt
And the physical file data.xml

I would suggest to use data.txt, because it should be a simple text file.
Can you say if it worked this way?

Vincent

14

Sorry, I don’t know why I typed data.xml - the file is definitely called data.txt!

15

Since you have this message, it means that the chart functionality is loaded correctly, but the ajax request to the file is not succesful (error code 404 : not found)

Could you have a look in the browser console (F12), to see if there are error messages?
And perhaps see if there is some data in the file : data.txt?
Which browser are you using, and OS?

I would perhaps restart OH and empty the browser history, to see if this has an influence.

Meanwhile, I’m trying to reproduce this message to see what else could be the cause.

Ok, there is a difference with the message I get when the file is not found.
I get the following message:

File data.txt is not found with url : http://localhost:8080/data/data.txt?_=1578457854754.
Are you sure you put the right url from the error message?

Remark : if you are keen to have privacy on your data, I would change your API user id on the website.

16

Thanks for getting back to me.

I am using OH on Raspian Linux. I amended the rule so that the path to data.txt was in the correct format for Linux. So:
var String path = new File("").getAbsolutePath()+'\\webapps\\data\\'+a
became:
var String path = new File("").getAbsolutePath()+'//webapps//data//'+a

and:
// check folder var File dir = new File(path.replace("\\"+a,""));
became:
// check folder var File dir = new File(path.replace("//"+a,""));

The data in data.txt is being populated as expected.

My url in sitemap is:
Frame { Webview url="https://www.intchart.com/api/g7Ztjc0D/chart/data.txt?path=data&width=1200&smooth=0.98" }

Seeing your example above, I’m not sure if that should be localhost instead of intchart.

Looking in the Chrome Console I can see the following error:

Failed to load resource: the server responded with a status of 404 (Not Found)intchart_launch @ VM108:12

I hope this helps with your investigations.

Rob

17

Good that you pointed out that I forgot to make the path applicable for all OS. I will correct this in the rules.

Your sitemap url is correct, and since your file is being populated, the rules are working, and since everything is tested in Chrome, there should be no problem there.

But the error message you get should imperatively point to the localhost instead of intchart, since in this configuration the data is logged to a local file. And to get the data your bowser is requesting it from your local sever. So I suppose this could be the cause.

It could be caused by your OS. Maybe something with the path. I will investigate this now.
For the moment I haven’t a Linux environment ready for testing (I will probably install one soon), but I will try now to make some changes based on above assumptions.