InfluxDB measurement adding every couple seconds on it own?

influxdb
Tags: #<Tag:0x00007f213491ce48>

(Maciej Eckstein) #1

I have this strange problem with influxDB.

One of my measurements should be created once a DAY and I do that with cron rule.
However if I do a select in the DB it is populated by thousands of the replicated result.

> drop  MEASUREMENT  B_Power_energy_daily_log

> select * from B_Power_energy_daily_log
name: B_Power_energy_daily_log
------------------------------
time                    value
1484750628004000000     1.85
1484750629004000000     1.85
1484750630006000000     1.85
1484750631004000000     1.85

Why is that? my influx persist is following: And I guess that does not explain why there is one value per second there.

Strategies {
    everyHour :   "0 0 * * * ?"
    everyMinute : "* * * * * ?"
    default = everyChange

}

Items {
    *   : strategy = everyChange, everyMinute, restoreOnStartup
    B_Power_energy_daily_log : strategy = everyChange
}

( ) #2

You’ve got two things wrong.

Your every minute is actually “everySecond”

everyMinute : "0 * * * * ?"
everyHour : "0 0 * * * ?"
everyDay  : "0 0 0 * * ?"

* includes everything, including B_Power_energy_daily_log. That’s why your item is currently persisted every second as you defined it.

What you should do is use the above definitions and you should not persist everything. You should only persist specifically what you want to graph or analyze later.


(Maciej Eckstein) #3

WOW! thats why influx is killing my RPI3! Thanks.
Is there a way to use default persist every minute and only take away one two sensors.
I want to persist 90% every minute as I graph almost everything with Grafana and it has some problems when the value does not change within the graph window. Setpoints or motion sensors are the most often ofenders that brake the graphs.


( ) #4

No it’s not possible. You should create groups and persist these. Here’s part of my config:

Strategies {
    everyMinute : "0 * * * * ?"
    everyHour : "0 0 * * * ?"
    everyDay  : "0 0 0 * * ?"
}

Items {
    gHeatAct*, gHeatSet*, gHeatValve*, gHeatMode*                   : strategy = everyChange, everyHour
    gPower*                                                         : strategy = everyChange, everyHour
    gRSSI*                                                          : strategy = everyChange
    Presence_Phone                                                  : strategy = everyChange
}

Persisting every minute is really not a good idea. You should use everyChange to catch actual changes and maybe everyHour to have support points for only seldomly updated items. As you can see I’m not even doing that for all items. I can’t imagine you really have the need for minutely accurate graphing!?

How do they do that? No change will normally not be of any harm.


(Maciej Eckstein) #5

Ok I fixed my install and now my energy readout does sum() properly as there is no dupes. but it did break Grafana graphs in the thermostat section when I have valve states and setpoints.

So I have now manualy add those to persist every minute to fix the graphs.


( ) #6

Your graph looks just fine to me :smiley:

You might want to play around with the Selection/Aggregation function as described in the tutorial: InfluxDB+Grafana persistence and graphing

For “Water Setpoint” you might want to use the last() selector.


(Maciej Eckstein) #7

As you can see the water setpoint is missing both ends on the graph. last() selector is not fixing that.
fill (previous) is fixing the missing end near the now() (right part) but for the mising part on the left I could not find a fix other then provide more granular data into DB
Also if the value did not change over 24h it will be completely missing from the graph.
I had to open some valves by hand and change the setpoints to make them into the graph as they were totaly omitted despite the fact the metric on the graph was still there.

But I do have much more concerining issue related to the orginal post.
I have croned ruled that on 21:06 should write daily value to the DB. it is simple as possible

	rule "Update max daily energy value"
when 
	Time cron "0 6 22 1/1 * ? *"
then
	var Number tmp
	var Number today_max = B_Power_energy_today_float.state
	tmp=today_max.floatValue
	logDebug("TERMO","---------Daily solar max sent to db:" + tmp )
	postUpdate(B_Power_energy_daily_log, tmp)
end

In the log I see it gets called twice 4ms apart from each other and that results in 2 values in the DB.

2017-01-19 22:06:00.032 [DEBUG] [eclipse.smarthome.model.script.TERMO] - ---------Daily solar max sent to db:5.951
2017-01-19 22:06:00.036 [DEBUG] [eclipse.smarthome.model.script.TERMO] - ---------Daily solar max sent to db:5.951

Why is that?


( ) #8

Ah that… Yes that’s “okay” as in expected. That is why I mentioned to go the middle way and add hourly points. This is the way Grafana draws diagrams and yes, that could be improved but it really doesn’t matter much if you look at the diagram with a range of one day or one week. Here’s an example from my system (hourly values, pay attention to the tiny space at the left):

At the end I believe for your Temperature diagram an hourly rule is okay. For other kinds of data or devices other settings might be beneficial.

Another Example: One of my “sensors” is sending data only periodically every 10 minutes and values in between can be linearly interpolated. I am only using the onChange strategy because a cron based strategy would actually inject false data.

I’m not sure. Maybe @rlkoshak can jump in.


(Rich Koshak) #9

I’ve no idea. Its as if the rule is triggered twice. I see nothing wrong with that cron expression that would cause that though.


( ) #10

Same ruling.

I remember somebody once reporting rules triggering twice in some constellation… do not remember. Did a restart of your system change something?


(Maciej Eckstein) #11

rebooted, waited nicely whole day and kaboom

2017-01-20 22:06:00.029 [DEBUG] [eclipse.smarthome.model.script.TERMO] - ---------Daily solar max sent to db:12.539
2017-01-20 22:06:00.058 [DEBUG] [eclipse.smarthome.model.script.TERMO] - ---------Daily solar max sent to db:12.539

It trigers twice! how can I debug that?


Rules executed twice
( ) #12

This could be a real bug… If we can find the cause we should file a bug report for it.

You are using a RPi3. Which way did you install openHAB and which revision are you on? WHat is the name of your rule file, are there any other rule files present that do not work correctly or empty? Please check the rule file for formatting error or similar…

Stop openHAB, clear out cache and tmp and start again.

Good luck! :slight_smile:


(Rich Koshak) #13

There are other threads discussing this issue add well. I might be seeing this too but am not sure as I’ve been doing a lot of changes in how my rules are triggered.


(Maciej Eckstein) #14

I did openhabian on RPI3, this night I updated from 653 to the latest snapshot 743.
I only have single rules.file called default.rules
I only have a single crone rule in the rules file, all other trigger on the sensor or input changes and generaly work ok. I’ll clrear tmp and wait until today trigger at the end of day.


( ) #15

Hey, seems you are not alone. Check the suggestions over there. After you found the problem, please report back so we can file the bug report :wink:


(Maciej Eckstein) #16

GOT IT.
It was a typo in the rule file.
A trailing space after the “when” clause was the problem!


(Rich Koshak) #17

That shouldn’t have caused a problem. Are you sure it was a space and not some sort of special character?


(Maciej Eckstein) #18

Now That I deleted it I cant check that back. But I’m quite positive it looked like a space. I only edit rules files on the RPI one way. And that is with Notepad++ on the SMB share on the OH2 machine.
I can see that it was set to the Windows style of the line ends CR LF and not to the unix LF only

so that would be a “space CR LF”


(Rich Koshak) #19

Trailing white space should not cause an error and openHAB should be able to handle Windows style CR/LF endings.

If you add the space back in does the error return?

If so this needs to be filed as a bug.


(Maciej Eckstein) #20

I’ll add that space and report tomorrow night when the rule kicks in.