InfluxDB OpenHAB2 stops storing data after reboot

hio77 · January 7, 2019, 10:23am

Hoping someone here can work this out, I’ve searched and searched again.

Platform information:

Hardware: Intel® Atom™ CPU C2750 / 6GB / 16GB (Sits on Esxi node, so dedicated diskspace etc)
OS: Debian 8
Java Runtime Environment: 1.8.0_191
openHAB version: 2.4.0
InfluxdB plugin: persistence-influxdb - 1.13.0

reinstalled, reconfigured and i just can’t quite work it out.

Logs don’t show any error on startup

2019-01-07 22:48:15.485 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'influxdb.persist'
2019-01-07 22:48:31.749 [DEBUG] [org.openhab.persistence.influxdb    ] - BundleEvent STARTING - org.openhab.persistence.influxdb
2019-01-07 22:48:31.761 [DEBUG] [org.openhab.persistence.influxdb    ] - ServiceEvent REGISTERED - {org.osgi.service.cm.ManagedService}={service.id=359, service.bundleid=204, service.scope=singleton} - org.openhab.persistence.influxdb
2019-01-07 22:48:31.793 [DEBUG] [.internal.InfluxDBPersistenceService] - influxdb persistence service activated
2019-01-07 22:48:34.255 [DEBUG] [org.openhab.persistence.influxdb    ] - ServiceEvent REGISTERED - {org.openhab.core.persistence.PersistenceService, org.openhab.core.persistence.QueryablePersistenceService}={password=openhab, service.id=360, service.bundleid=204, service.scope=bundle, retentionPolicy=autogen, user=openhab, service.pid=org.openhab.influxdb, component.name=org.openhab.persistence.influxdb, db=HAB, component.id=220} - org.openhab.persistence.influxdb
2019-01-07 22:48:34.258 [DEBUG] [org.openhab.persistence.influxdb    ] - BundleEvent STARTED - org.openhab.persistence.influxdb

Logs confirm OpenHAB2 and influxDB are communicating

2019-01-07 23:02:47.126 [TRACE] [.internal.InfluxDBPersistenceService] - adding historic item BreadboardTest_Light_Light: time Mon Jan 07 19:58:48 NZDT 2019 value 30.0
2019-01-07 23:02:47.127 [TRACE] [.internal.InfluxDBPersistenceService] - startEntryNum 0
2019-01-07 23:02:47.126 [TRACE] [.internal.InfluxDBPersistenceService] - columns [time, value]
2019-01-07 23:02:47.133 [DEBUG] [.internal.InfluxDBPersistenceService] - objectToState found a NumberItem
2019-01-07 23:02:47.134 [TRACE] [.internal.InfluxDBPersistenceService] - adding historic item BreadboardTest_Pressure_Pressure: time Mon Jan 07 19:58:59 NZDT 2019 value 995.87
2019-01-07 23:02:47.129 [DEBUG] [.internal.InfluxDBPersistenceService] - query string: select value from "autogen"."BreadboardTest_Vcc_Voltage" where  time > 1546768870s  and time < 1546855367s  limit 2147483647
2019-01-07 23:02:47.154 [TRACE] [.internal.InfluxDBPersistenceService] - series Series [name=BreadboardTest_Vcc_Voltage, tags=null, columns=[time, value], values=[[1.546844358067E12, 3.29]]]
2019-01-07 23:02:47.155 [TRACE] [.internal.InfluxDBPersistenceService] - columns [time, value]
2019-01-07 23:02:47.157 [DEBUG] [.internal.InfluxDBPersistenceService] - objectToState found a NumberItem
2019-01-07 23:02:47.159 [TRACE] [.internal.InfluxDBPersistenceService] - adding historic item BreadboardTest_Vcc_Voltage: time Mon Jan 07 19:59:18 NZDT 2019 value 3.29
monitor@MonitorHost:/var/log/openhab2$ tail -f openhab.log
2019-01-07 23:02:47.126 [TRACE] [.internal.InfluxDBPersistenceService] - adding historic item BreadboardTest_Light_Light: time Mon Jan 07 19:58:48 NZDT 2019 value 30.0
2019-01-07 23:02:47.127 [TRACE] [.internal.InfluxDBPersistenceService] - startEntryNum 0
2019-01-07 23:02:47.126 [TRACE] [.internal.InfluxDBPersistenceService] - columns [time, value]
2019-01-07 23:02:47.133 [DEBUG] [.internal.InfluxDBPersistenceService] - objectToState found a NumberItem
2019-01-07 23:02:47.134 [TRACE] [.internal.InfluxDBPersistenceService] - adding historic item BreadboardTest_Pressure_Pressure: time Mon Jan 07 19:58:59 NZDT 2019 value 995.87
2019-01-07 23:02:47.129 [DEBUG] [.internal.InfluxDBPersistenceService] - query string: select value from "autogen"."BreadboardTest_Vcc_Voltage" where  time > 1546768870s  and time < 1546855367s  limit 2147483647
2019-01-07 23:02:47.154 [TRACE] [.internal.InfluxDBPersistenceService] - series Series [name=BreadboardTest_Vcc_Voltage, tags=null, columns=[time, value], values=[[1.546844358067E12, 3.29]]]
2019-01-07 23:02:47.155 [TRACE] [.internal.InfluxDBPersistenceService] - columns [time, value]
2019-01-07 23:02:47.157 [DEBUG] [.internal.InfluxDBPersistenceService] - objectToState found a NumberItem

influxdb.persist

Strategies {
    everyHour : "0 0 * * * ?"
    everyDay  : "0 0 0 * * ?"
    default = everyChange
}
Items {
      * : strategy = everyChange, everyDay, restoreOnStartup
}

influxdb.cfg

user=*snip*
password=*snip*
db=HAB
retentionPolicy=autogen

As noted, After a reboot or any edits to items, it dies.

If i delete the measures and series in influxdb, Remove all things, items and links from openhab and configure they start to work until the next reboot.

BrutalBirdie · January 7, 2019, 10:27am

You dump all values stored and all openHAB configs?
Did you try to remove the restoreOnStartup option? I remember people having problems with this.

Ohhh and ps: you still published your user and password for your influxdb
The DEBUG log shows it all

ServiceEvent REGISTERED - {org.openhab.core.persistence.PersistenceService, org.openhab.core.persistence.QueryablePersistenceService}={password=openhab, service.id=360, service.bundleid=204, service.scope=bundle, retentionPolicy=autogen, user=openhab, service.pid=org.openhab.influxdb, component.name=org.openhab.persistence.influxdb, db=HAB, component.id=220} - org.openhab.persistence.influxdb

hio77 · January 7, 2019, 12:18pm

Yes, Everything i pipe into openhab i wish to store in a persistant db.

I’ll try removing restoreOnStartup

vzorglub · January 7, 2019, 1:04pm

It is recommended to use mapDB for restoreOnStartup
You can use several persistence services together
I use mapDB for restoreOnStartUp
Influx for general long term storage and fancy graphs
and rrd4j for quick UI graphs

See the first line:

hio77 · January 8, 2019, 6:36am

so, removing restoreOnStartup seemed to be quite happy with my test device, persisting for over an hour through reboots. (after resetting everything again)

However, soon as i added a second list of items for another device, the first stopped persisting again.
the second has happily continued along.

Rebooting devices seems to fail to recover the first device. MQTT messages are still flowing through.

I’ve skipped over the mapdb suggestion fornow, as i’m trying to keep it simplified while troubleshooting this.

hio77 · January 8, 2019, 9:06am

Hmm… Just noticed the following.

Would this explain what’s going on here?

2019-01-08 22:07:10.728 [INFO ] [org.quartz.core.QuartzScheduler     ] - Quartz Scheduler v.2.2.1 created.
2019-01-08 22:07:10.736 [INFO ] [org.quartz.simpl.RAMJobStore        ] - RAMJobStore initialized.
2019-01-08 22:07:10.754 [INFO ] [org.quartz.core.QuartzScheduler     ] - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'openHAB-job-scheduler' with instanceId 'NON_CLUSTERED'
  Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
  NOT STARTED.
  Currently in standby mode.
  Number of jobs executed: 0
  Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 2 threads.
  Using job-store 'org.quartz.simpl.RAMJobStore' - **which does not support persistence**. and is not clustered.

2019-01-08 22:07:10.760 [INFO ] [org.quartz.impl.StdSchedulerFactory ] - Quartz scheduler 'openHAB-job-scheduler' initialized from specified file: '/usr/share/openhab2/runtime/etc/quartz.properties'
2019-01-08 22:07:10.767 [INFO ] [org.quartz.impl.StdSchedulerFactory ] - Quartz scheduler version: 2.2.1

hio77 · January 9, 2019, 4:20am

Further testing, seems to be stable with a single ‘thing’

Soon as i add another thing though, the first stops working.

Both Homie devices report themselves differently, so it’s not overlapping. the pushes from one isn’t appearing on the other etc.

In terms of MQTT logs, both devices report their stats nicely.

Is there a way to log on mqtt receive? I’m starting to wonder if that’s where my issue is.
MQTT is simply setup with the MQTT Broker addon, MQTT broker added through UI then Homie devices added via detection (inbox)

hio77 · January 9, 2019, 1:50pm

Right, so going off the theory that it’s not to do with persistence has been confirmed.

Completely removed influxdb from openhab2, reset everything and cleared all cache.

Adding the things, items and linking all happily worked.
i left this working for a few hours, soon as i changed the “Location” it stopped updating just like i was seeing with influxdb persistence.

For now, I have two homie sensor nodes happily running into openhab2.
That’s more success than before… I’ll try adding influxdb back in tomorrow and see if it still happily stays.

if so, i suspect the issue i’m seeing comes down to editing a Homie thing after being added via autodetection? if so, that seems like a bug?

Euphi · January 10, 2019, 10:33am

amm009uk · April 18, 2019, 9:32am

I am in the midst of moving from OH 2.3 to 2.4 and noticed the restoreOnStartup is not working with influxdb. If you restart openhab after everything is up then restoreOnStartup works. I added “sleep 120” in /etc/init.d/openhab2 within the startup function. All good now.

I actually want a short history of events and adding one more service isn’t what I wanted.