Persistence and scripts for rolling periods

I’m running OH3 OpenHabian on a pi4. One of my smart sockets has an energy monitor, and I want to periodically check how many units it has consumed.

This for electricity cost estimates and there are two tariffs; Night (9pm - 7am) and Day, so I need to persist two items - night units and day units.

From this I want to:

  1. Trigger (on event) an email reporting the night and day units consumed since midnight today, and the night and day units consumed so far this month.

  2. Be able to report daily, weekly and monthly totals for both units up to a year in the past.

OH3 configures rrd4j by default so I can see the values in a chart. I can get the value at midnight using

SmartPlug_EnergyUsage.historicState(ZonedDateTime.now().with(LocalTime.MIDNIGHT),"rrd4j").state as Number

I’m not sure how to get the state from 9pm and 7am but even if I do, I don’t think there’s any guarantee that the rrd4j values will be correct due to the averaging that takes place.

What is the best way to approach this?

I recognise that I need to reset the monthly night and day unit count at midnight on the first of the month, and the daily night and day unit count at midnight every day, but I’m not sure what the most efficient way would be. Perhaps move the item persistence to some other service, then simply query that?

Cheers

That’s probably less of an issue than you think, even for going back a year.

As you’ve understood, ease of use of stored data later depends on what you choose to store now.
A fairly fundamental choice is to store an ever-increasing “meter reading” or “per-short-period consumption”. You can calculate what you want from either approach, but you might find one simpler for charting etc.

Don’t forget you can persist on demand, from a rule,not just rely on automatics.
Example, at 23.59 each day you might calculate the day’s consumption in a rule, and persist that.
I do this, and it’s not difficult to extract and sum up previous months or years totals. It mostly makes it easier for me to think about without a headache, than any mathematical advantage.

This was really helpful, thank you.

This is the rule I ended up with; I added this in the UI via schedule but because I set it to trigger whenever the power socket sends an update, the UI (sensibly) placed it in Rules.

This sort-of works. I ran into the odd exception and memory error in the log, OH slowed right down and there were ‘failed to respond in 5000ms’ entries too. It would be OK after a reboot for a few hours, then the same thing.

I commented out all the postUpdates and Persists, leaving the rest, and it seemed to settle down. However, all Things disappeared and the DB JSON file was at 0KB so I restored that from a backup - looks possible I need to replace the SD card. I’ll do that, but meantime is there anything in this script that might be leading to problems with memory?

Thanks!

var Number hour = now.getHour
val Number nMidnight = SpaTPLink_EnergyUsage.historicState(ZonedDateTime.now().with(LocalTime.MIDNIGHT),"rrd4j").state as Number
val Number nNow = (SpaTPLink_EnergyUsage.state as Number).floatValue
val Number nConsumptionToday = nNow - nMidnight

logInfo("spaTest", "Hour: " + hour)
logInfo("spaTest", "Spa consumption midnight: " + nMidnight)
logInfo("spaTest", "Spa consumption now: " + nNow)
logInfo("spaTest", "Spa consumption today: " + nConsumptionToday)
  
if ((hour >= 0) && (hour <= 07))
{
  // set night units to now value
  logInfo("spaTest", "Set NightUnits to " + nConsumptionToday)
  SpaNightUnits.postUpdate(nConsumptionToday)
}
else if ((hour >= 7) && (hour <= 21))
{
// set day units to diff betweeen 07:00 value and now
   val Number nSeven = SpaTPLink_EnergyUsage.historicState(ZonedDateTime.now().with(LocalTime.MIDNIGHT).plusHours(7),"rrd4j").state as Number
   logInfo("spaTest", "Spa consumption 7am: " + nSeven)
   logInfo("spaTest", "Set DayUnits to " + (nNow - nSeven))
   SpaDayUnits.postUpdate(nNow - nSeven)
}
else
{
  // increment night units by diff between 21:00 and now  
  val Number nSeven = SpaTPLink_EnergyUsage.historicState(ZonedDateTime.now().with(LocalTime.MIDNIGHT).plusHours(7),"rrd4j").state as Number
  val Number nNine = SpaTPLink_EnergyUsage.historicState(ZonedDateTime.now().with(LocalTime.MIDNIGHT).plusHours(21),"rrd4j").state as Number
  val Number nightAdd = nNow - nNine
  logInfo("spaTest", "Spa consumption 7am: " + nSeven)
  logInfo("spaTest", "Spa consumption 9pm: " + nNine)
  logInfo("spaTest", "Increment NightUnits: " + nSeven + " + " + nightAdd + " = " + nSeven + nightAdd)
  SpaNightUnits.postUpdate(nSeven + nightAdd)      
}
  
SpaDayUnits.persist
SpaNightUnits.persist

The mere existence of UI entered DSL rules is enough to cause performance issues at OH 3.01 release.
This is fixed in milestone releases etc. already.

Or put the rule in a conventional xxx.rules file while awaiting next stable release.

That rules memory hogging should not of itself cause that, but you could be really unlucky at the moment of a crash.

I’ll move it to a rules file, thanks! I noticed the error is still present today with OH not responding.

[WARN ] [ache.cxf.phase.PhaseInterceptorChain] - Application {http://internal.id.core.openhab.org/}UUIDResource has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Java heap space
	at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162) ~[bundleFile:1.0.9]
	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128) ~[bundleFile:1.0.9]
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201) ~[bundleFile:1.0.9]
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104) ~[bundleFile:1.0.9]
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) ~[bundleFile:1.0.9]
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) ~[bundleFile:1.0.9]
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) [bundleFile:1.0.9]
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) [bundleFile:1.0.9]
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:216) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:301) [bundleFile:1.0.9]
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:225) [bundleFile:1.0.9]
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) [bundleFile:3.1.0]
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:276) [bundleFile:1.0.9]
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:544) [bundleFile:9.4.20.v20190813]
	at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:71) [bundleFile:?]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:536) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307) [bundleFile:9.4.20.v20190813]
	at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:293) [bundleFile:?]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) [bundleFile:9.4.20.v20190813]
	at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:80) [bundleFile:?]
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.Server.handle(Server.java:494) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:367) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782) [bundleFile:9.4.20.v20190813]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918) [bundleFile:9.4.20.v20190813]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.OutOfMemoryError: Java heap space

I have exactly the same error. Do you have a Z-Wave network? The error is reproducible for me:
As soon as I open up the detail view of the Z-Wave Serial Controller (a Aetoec Z-Wave Stick Gen 5), I get this error after a few moments (maximum CPU usage) and the UI page only shows “ONLINE” in the view (no details).
Same error appears, when I try to open the Network Map in an already configured Z-Wave thing.

If required, I will write a full bug report with logs and what else is need. But I guess it would be interesting if it might be the Z-Wave addon.

Interesting.

Yes, I have the same Aeotec Z-Wave stick Gen 5 and these log entries appeared at the same time as I started using it, but I hadn’t made the connection since I made lots of other changes at the same time. I’ll try disabling the Z-Wave for a bit.

I still run into the problem, however I can open the Zwave Network Map via a Z-Wave thing. Were you able to resolve the issue @SonicS?