Again i am working on my energy management system. One simple task is to track energy consumption on a hourly base. So i created a rule that calculates the consumption evry hour on the hour. This value is send to an item and the item should be persisted. Simple enough.
Starting to analyse the values, i got strange results. The values are nocht displayed on the hour. Often very far away from the hour.
Based on that i set up a rule that is updating every five minutes a number item with the current Minite value. Additionally i set up a postgesql database to persist these value.
As you could see the timestamps sometimes differ significantly. My expectation on the timestamp was that the timestamp holds the time when the item was updated/changed and not the time the value was written to the database.
Am i wrong, is this the intended behaviour, does anyone have the same problem?
There’s at least potential for problems here.
What’s your strategy in .persist and how does postgres inner workings/config affect timestamps ?
Consider using standard rrd4j, I have not heard about any similar problem there and there will be many more people that can help with their knowledge.
PS: and sorry for advertising but when you work on an OH based EMS, have a look at https://storm.house/energiemanagement-system it might already provide a lot of what you’re looking for
How is your everyHour cron expression defined? It might not be defined to save the value on the hour. That’s unlikely to be the case but it’s a possibility.
I don’t think that’s documented anywhere. Looking at the code for PostgreSQL I don’t see anywhere that the timestamp is actually passed when the entry is created so, assuming I’m reading the code correctly I think your expectation is incorrect and OH is relying on PostegreSQL to generate the timestamp instead. Here’s the SQL:
sqlCreateNewEntryInItemsTable = "INSERT INTO items (itemname) SELECT itemname FROM #itemsManageTable# UNION VALUES ('#itemname#') EXCEPT SELECT itemname FROM items";
``
@mstormi
typically i persist every value in the mapdb on every change with a restore on startup. The disired value are persisted with a everyupdate strategy.
To be honest i have no idea how pregesql’s internal handling is.
ny problem is that with rrd4j i am not able to store exact data over a long time.
You advertising is no problem and i really like the idea to earn money with openhab. EMS is like Scenes one of the things that openHAB is missing
The cron is done from HABApp, so a little different from the typical cron trigger. But i have check the time stamps in the logs. HabApp Log shows the right time (plu some millies) and the openhab Eventlog is showing the right time as well (plus some more millies), but the database vary the tomestamp sometime a little some times a lot. With the hourly trigger i found differences up to half an hour.
Tanks for checking the code. I think that is really an issue on persistence and i think all other binding will do it in the same way.
I know there is a way to insert values into the database through openHAB’s REST API that does allow you to set the timestamp yourself instead of relying on system time. You might be able to do something with that.
I think that there is an issue open on that, because i aked to implement that into the persitence funktion of HABApp. There we have found out that this feature is not implemented yet in the bindings
I have to check for the issue. But this would be great, because you could then write the right data at any time you want.
the latter you mean in postgres.persist ? mapdb persistence is irrelevant / unrelated to the problem.
Well, that’s often overestimated as a need. And even if you really need it there’s means to deal with that like to generate exports for off-line analysis.
And you can actually tweak rrd4j’s compression. Don’t remember off my head exactly where but saw it’s documented so you can increase the intervals before compression applies. IIRC it was some setting in /etc/openhab/services/<namesdontmatterhere>.cfg (remember persistence is essentially still OHv1 hence configured through text files like v1 bindings).
I suggest you add rrd4j persistence in the first place (well if you don’t have any rrd4j.persist it’ll be there by default anyway) and then see if it shows the same problem you see with postgres.
Yep. But there’s one major difference:
an EMS implementation is way more complex and much more work to get it right and these efforts are greatly underestimated. I know, I invested several months.
That’s even more so true for ‘generic’ implementations that need to be flexible enough such that users can adapt (i.e. parametrize, no coding) it to their needs without implementing all the common basic parts themselves.
There’s so many more use cases in energy management, plus most are also device dependent.
Just a quck one on that:
With rrd4j you do need to specify the whole timeframe for which you want to keep the data!
Knowing that, customize rrd4j to have only one archive that stores the data for the above mentioned timeframe. As it is only one archive no compression will be done. That’s it.
