Slow charting

Hi !

I have a problem with slow charts. I am using both rrd (with everyminute strategy) and db4o + default chart provider (just use Chart in sitemap). All of charts which I want to display on GUI (it doesn’t matter from which persistence or quantity of items) , when I enter it I must wait 10-20 seconds until they will appear :unamused:.
It is on LAN, on VPN usually I got that image is broken and after few times I can see it.
What should I check ? Where can be problem located ?

Is there any chance that your openHAB server is very resource-constrained? Is it an older Raspberry Pi or similar?

It is on Raspberry 2. System dosn’t look overloaded:

red - Processor usage
green - memory
blue - openhab process (in fact Java), it shows more than 100% probably because it scale up to one core

Those pics are during GUI usage, so it looks gui occupied some resources, but not 100% The rest it is working quite well, no LAG. I have also other light www server which provide me other temp charts and it work great.

Open your browser’s developer console, and look at the Log and Network tabs to see if you can determine if there are any Javascript errors, or if the HTTP request to retrieve the chart is really taking all that time.

Do you mean habmin ? If so there is nothing there. I have checked Openhab’s log and there is no problem.
Maybe it is because of persistence data … I have about 50 items. I have restart it and from hardware overload it looks ok.
I am out of ideas :frowning:

I mean your web browser, as its built-in developer interface can report when the request for the chart is made and when the chart image is returned. I doubt that HABmin is involved.

There is nothing suspicion there. It is simply waiting for data to being delivered. Of course there is no issues on 1gig LAN the rest of gui works fine. Problem is openhab do not deliver chart immediately (within few seconds), sometimes it is ca 30 seconds.

For one of my charts, it takes 278ms waiting for a reply to the request for the chart, and 379ms to download the chart image, so it takes less than one second to request and receive the chart (over wifi, served from a Raspberry Pi 2). So the fact that it takes your setup several seconds, up to 30 seconds, for the chart image to be delivered, suggests either a performance or configuration issue. I’m afraid that I would need more information in order to be able to offer more assistance.

Ok I have analyzed in detaills:

Request:

GET http://192.168.1.106:8080/openhab.app?sitemap=moj&sitemap=moj&w=0400&poll=true&__async=true&__source=wa0400

Request header:

Host: 192.168.1.106:8080
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,pl;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://192.168.1.106:8080/openhab.app?sitemap=moj
Cookie: ext-graphWindowSizer=o%3A; ext-configWindowSizer=o%3A; ext-automationWindowSizer=o%3A
Connection: keep-alive

Request answer:

Content-Type: image/png
Date: Sat, 19 Mar 2016 10:36:13 GMT
Server: Jetty(8.1.3.v20120522)
Transfer-Encoding: chunked

Request parameters:

groups=CPU
period=D
random=1

Answer:
<root><part><destination mode="replace" zone="timeout" create="false"/><data/></part></root>
waiting 10 682 ms

After few such gets I received chart:
connecting: 1ms
Waiting: 111 129ms
Receiving: 270ms

Chart has 18kB.

So If I compare time with you only download time is similar ca 300ms, but waiting for an answer is dramatically diff :frowning:

what do you suggest:

  • disable most of persistence ?
  • try to launch it on diff platform ?
  • change a config to simple one which I used few months ago on the begging ?

thanks for help !

The request you documented is the long-polling request from the client, not the request for the chart image. That request can block for several seconds.

The chart image request would look more like:

http://server:8080/chart?groups=CPU&period=D&random=1

Do you see requests of that sort in the Network tab of your Developer Tools window?

There is a bug in 1.8.1 and previous, fixed in the upcoming 1.8.2, where chart refresh is broken. You might want to follow the instructions there to see if this issue is resolved.

Im sorry my mistake, probably I have copied get url from previous request, should be:

http://192.168.1.106:8080/chart?groups=CPU&period=D&random=1

Rest is ok, as I wrote.

I don’t see refresh of this chart, so I can’t say if it is problem too. for sure problem is first first request. Do you think problem with refresh can affect requests in general ?

Not sure. The problem with the &random=1 is that an old chart could be cached in the browser, since 1 isn’t very random. The change in 1.8.2 makes sure that every chart requested by the Classic UI is fresh, both the initial chart returned and all subsequent charts.

Have you tried the above URL directly, and also by making the random value unique? Does it still take many seconds for the chart image to appear?

Yes I have tried directly to get this image/chart. Same problem, browser is waiting quite long for date.

I am just thinking if it is not due to 2 DBs, which I use, for all items I use db40 with strategy everychange and for few items I use rrd4j with everyminute strategy. Problem occurs with charts from both DBs.

Does it help if you explicitly provide the persistence service name in the URL?

http://192.168.1.106:8080/chart?groups=CPU&period=D&service=db4o&random=453423234

Nothing help :frowning:
I have to wait 6 minutes.

One more thing; Is it any method to check how big is my persistence DB ? Any way to reset it ?

I don’t know where the delay is in the code path for delivering your chart. There are multiple possibilities, anything from DNS resolution problems, overloaded persistent stores, extremely slow image creation code, very slow disk I/O, and on and on. It’s possible that RRD4J or DB4O is having some problem, but there are other possibilities as well. I don’t use either of those persistence services, but you should be able to find where it keeps the files by using find or similar tools on your openHAB server system.

I have found it in /var/lib/openhab/persistence
db4o files has ca 240MB/day :astonished:
Is it not too much for Raspberry ?

That definitely looks like it could be a problem! Unless db4o is very fast at queries across such a massive datastore, that would be a good area to dig into. And if this is being stored on an SD card, it’s likely going to be worn out quickly and stop working altogether. Even if it’s being stored on an external disk, that might be too much to ask of db4o.

Bingo !

I have delete db4o file (over 200MB) and charts are working fine :slight_smile:

thank for help !

BTW
Is it any way or tool to clear db4o files ? Ie. delete old data ?

I have collected data since 3 months and db grow up to 200MB

1 Like

I don’t know personally, but it looks like @rlkoshak just answered this question!