OH, InfluxDB, KNX: no scheduled read on KNX bus

[knxhab (Carsten Wegner)]

Hi there,

finally reading here isn’t enough anymore. I am stuck in a persistence (or OH?) problem and need some help.

What I’m trying to do, is to store data from KNX in InfluxDB, either on change or from time to time, if there isn’t a change.

The problem: values are read on OH startup, but then never again - until next restart.

My config - to keep it simple and repeatable I reinstalled everything, but reduced the persistence to only one value:

1. Ubuntu 16.04 LTS, distro-upgrade 2018-02-13, full-update 2018-02-25
2. Java is Zulu 8 from azulsystems via apt
3. InfluxDB from influxdata via apt
4. OH 2.2.0 from bintray via apt
5. KNX binding and InfluxDB persistence via Paper UI

knx.cfg:

type=ROUTER
ignorelocalevents=true
busaddr=1.1.4

influxdb.cfg:

url=http://127.0.0.1:8086
user=openhab
password=myPWD
db=openhab_db
retentionPolicy=autogen

WP_Sensoren.items file for persistence:

Number    WP_HK2_IST    "WP HK2-Ist"    {autoupdate="true",knx="13/7/5+<13/7/10"}

influxdb.persist:

Strategies {
	everyMinute : "0 * * * * ?"
	everyHour   : "0 0 * * * ?"
	everyDay    : "0 0 0 * * ?"

	every05Min  : "0 */5 * * * ?"

	default	= everyChange
}

Items
{
	WP_HK2_IST	:	strategy = everyChange, everyMinute, restoreOnStartup
}

What I see in openhab.log:

2018-02-26 20:47:59.412 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'WP_Sensoren.items'
2018-02-26 20:47:59.428 [DEBUG] [el.item.internal.GenericItemProvider] - Read items from model 'WP_Sensoren.items'
2018-02-26 20:47:59.432 [TRACE] [el.item.internal.GenericItemProvider] - Created item 'WP_HK2_IST' of type 'Number'
2018-02-26 20:47:59.445 [DEBUG] [el.item.internal.GenericItemProvider] - Processing binding configs for items from model 'WP_Sensoren.items'
2018-02-26 20:47:59.450 [TRACE] [el.item.internal.GenericItemProvider] - Created item 'WP_HK2_IST' of type 'Number'
2018-02-26 20:47:59.455 [TRACE] [el.item.internal.GenericItemProvider] - Couldn't find config reader for binding type 'knx' > parsing binding configuration of Item 'WP_HK2_IST (Type=NumberItem, State=NULL, Label=WP HK2-Ist, Category=null)' aborted!
2018-02-26 20:48:05.491 [TRACE] [el.item.internal.GenericItemProvider] - Created item 'WP_HK2_IST' of type 'Number'
2018-02-26 20:48:05.492 [DEBUG] [el.item.internal.GenericItemProvider] - Start processing binding configuration of Item 'WP_HK2_IST (Type=NumberItem, State=NULL, Label=WP HK2-Ist, Category=null)' with 'BindingConfigReaderDelegate' reader.
2018-02-26 20:48:05.689 [DEBUG] [x.internal.bus.KNXBusReaderScheduler] - Datapoint 'WP_HK2_IST':  one time reading scheduled.
2018-02-26 20:48:05.697 [DEBUG] [al.bus.KNXBindingDatapointReaderTask] - Autorefresh: got new item WP_HK2_IST in reader queue
2018-02-26 20:48:05.698 [DEBUG] [al.bus.KNXBindingDatapointReaderTask] - Autorefresh: Trying to read from KNX bus: state DP 13/7/10 WP_HK2_IST, DPT main 0 id 9.001, low priority
2018-02-26 20:48:05.698 [DEBUG] [al.bus.KNXBindingDatapointReaderTask] - Autorefresh: Sending read request to KNX for item 'WP_HK2_IST' DPT '9.001'
2018-02-26 20:48:05.742 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Added event (item='WP_HK2_IST', type='29.3') to the ignore event list
2018-02-26 20:48:05.766 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Processed event (item='WP_HK2_IST', type='29.3', destination='13/7/10')
2018-02-26 20:48:05.768 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Added event (item='WP_HK2_IST', type='29.3') to the ignore event list
2018-02-26 20:48:05.768 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Processed event (item='WP_HK2_IST', type='29.3', destination='13/7/10')
2018-02-26 20:48:05.770 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Added event (item='WP_HK2_IST', type='29.3') to the ignore event list
2018-02-26 20:48:05.770 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Processed event (item='WP_HK2_IST', type='29.3', destination='13/7/10')
2018-02-26 20:48:05.771 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Added event (item='WP_HK2_IST', type='29.3') to the ignore event list
2018-02-26 20:48:05.772 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Processed event (item='WP_HK2_IST', type='29.3', destination='13/7/10')
2018-02-26 20:48:05.788 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Received update (item='WP_HK2_IST', state='29.3')
2018-02-26 20:48:05.825 [TRACE] [.binding.knx.internal.bus.KNXBinding] - We received this event (item='WP_HK2_IST', state='29.3') from KNX, so we don't send it back again -> ignore!
2018-02-26 20:48:05.890 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Received update (item='WP_HK2_IST', state='29.3')
2018-02-26 20:48:05.891 [TRACE] [.binding.knx.internal.bus.KNXBinding] - We received this event (item='WP_HK2_IST', state='29.3') from KNX, so we don't send it back again -> ignore!
2018-02-26 20:48:05.936 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Received update (item='WP_HK2_IST', state='29.3')
2018-02-26 20:48:05.936 [TRACE] [.binding.knx.internal.bus.KNXBinding] - We received this event (item='WP_HK2_IST', state='29.3') from KNX, so we don't send it back again -> ignore!
2018-02-26 20:48:06.018 [TRACE] [.binding.knx.internal.bus.KNXBinding] - Received update (item='WP_HK2_IST', state='29.3')
2018-02-26 20:48:06.019 [TRACE] [.binding.knx.internal.bus.KNXBinding] - We received this event (item='WP_HK2_IST', state='29.3') from KNX, so we don't send it back again -> ignore!
2018-02-26 20:49:00.246 [TRACE] [.internal.InfluxDBPersistenceService] - storing WP_HK2_IST in influxdb value 29.3, WP_HK2_IST (Type=NumberItem, State=29.3)
2018-02-26 20:49:00.384 [TRACE] [persistence.internal.PersistItemsJob] - Storing item 'WP_HK2_IST' with persistence service 'influxdb' took 141ms
2018-02-26 20:50:00.745 [TRACE] [.internal.InfluxDBPersistenceService] - storing WP_HK2_IST in influxdb value 29.3, WP_HK2_IST (Type=NumberItem, State=29.3)
2018-02-26 20:50:00.745 [TRACE] [persistence.internal.PersistItemsJob] - Storing item 'WP_HK2_IST' with persistence service 'influxdb' took 2ms

But I don’t see the scheduled reading of this value on the KNX Bus - only the initial reading at starting OH:

#	Zeit	Quell-Adresse	Quell-Name	Ziel-Adresse	Ziel-Name	Rout	Typ	DPT	Info
5	26.02.2018 20:48:05,753	1.1.4	openHab	13/7/10	HK2 Ist	5	GroupValueRead		
6	26.02.2018 20:48:05,759	1.0.2	ISG KNX IP	13/7/10	HK2 Ist	5	GroupValueResponse	  9.001 Temperatur (°C)	0D B9 | 29,3 °C
***removed other KNX telegrams***
14	26.02.2018 20:50:23,989								Aufzeichnung wurde beendet

Looks like persistence strategy kicks in on the OH side, but never makes it to the KNX bus. BTW, scheduled values are written to InfluxDB, which means, the WP_HK2_IST measurement serie gets filled with the same value, 29.3, over and over.
Also other items, like lights, and related sitemaps with switches for these lights are working.

Kind regards

[rlkoshak (Rich Koshak)]

@knxhab

But I don’t see the scheduled reading of this value on the KNX Bus - only the initial reading at starting OH:

OH persistence only deals with Items. It is not designed nor intended to do anything with the bindings. OH persistence:

• saves the state of Items based on the configured strategy
• can be queried from rules for historic data
• can restore the state of an Item on startup; note that this ONLY restores the state of the Item, the restored state does not get sent out to the binding or the device

If you need to send this data back out on the KNX bus, you will need to write a Rule and sendCommand to the appropriate Items.

For example, to set the lights to their restored state when OH restarts one would do something like:

rule "Restore lights"
when 
    System started
then
    Thread::sleep(1000) // give persistence a chance to come up and restore all the Items
    gLights.members.forEach[light | light.sendCommand(light.state)]
end

The above rule sends the current state of all the members of gLight, which have been initialized from persistence with their previous state, to the light as a command.

[knxhab (Carsten Wegner)]

@rlkoshak

@knxhab

But I don’t see the scheduled reading of this value on the KNX Bus - only the initial reading at starting OH:

OH persistence only deals with Items. It is not designed nor intended to do anything with the bindings. OH persistence:
…
If you need to send this data back out on the KNX bus, you will need to write a Rule and sendCommand to the appropriate Items.

Thanks for your answer,

I think my words are misleading: I don’t want to send data back on the KNX bus. “Schedule read” means that the item WP_HK2_IST, defined in WP_Sensoren.items, should be read at the interval specified in the Items section of influxdb.persist according to the strategy which is also defined in influxdb.persist.

Such reading of the items state from OH should result in a GroupValueRead on the KNX bus. But as you see from the logs, on the OH side the system is pretending to read WP_HK2_IST according to the schedule/strategy, but in fact it is not reading from the KNX bus, as there is no indication of this reading. And in fact, if it read from the KNX bus, the value should change (I can see it by reading the WP_HK2_IST value through ETS from the KNX bus) and by the UI of my heating system. But OH is stuck with the one value it reads on startup.

Hope I’ve explained better now.

[rlkoshak (Rich Koshak)]

@knxhab

should be read at the interval specified in the Items section of influxdb.persist according to the strategy which is also defined in influxdb.persist.

Persistence and OH do not work like that. influxdb.persist just deals with Items and it only saves the state of Items. Saving the state of the Items or even reading the state of Items does not go out to the binding and to the device. It only deals with the state of the Item.

To go out to the KNX bus you have to configure that in the binding, if it is supported. Or write a rule that sends a command that causes the binding to return the value. Persistence is not involved.

[knxhab (Carsten Wegner)]

Oh, so I’ve been on a wrong track!

What I thought: if I have defined an item, defined a strategy in a persist file and bond this item to the strategy in the persist file (like I did), OH would do a scheduled read of the item - in my case send a read request for that item on the KNX bus.

But as I understand, persistence in OH will only persits state values from items, as they (the values) already are in OH. The persistence strategy only says how often the state is written to the persistence service.
If I like to persist an updated state value of an item, I will have to trigger a regular read of the items state myself or the current state will get updated in OH by other means (e.g. as is the state of lights by turning them on or off and the new state being “published” on the KNX bus, so OH gets it by just listening through the binding (given that an appropriate item is defined).

Is this correct?

[rlkoshak (Rich Koshak)]

I think you have it right.

Put another way, Persistence just sits there passively reading the Item State registry and the Event Bus and storing the state of the Items according to its configured strategy.

In order for an Item to communicate with the actual device, one needs to sendCommand (via the UI or Rules) which gets processed by the binding and sent to the device, or the binding receives an update/change in the device’s state that it processes and updates or commands the linked to/bound to Item with the new value.

[knxhab (Carsten Wegner)]

Thank you for the explanation!

Now I will have to think of my own “strategy” to update values. KNX devices can send some of them after the values change a certain amount or in regular intervals. Others will have to read by a rule, for they are not likely to be command receivers, such as counters of energy consumption.

[binderth (Thomas Binder)]

@knxhab

What I thought: if I have defined an item, defined a strategy in a persist file and bond this item to the strategy in the persist file (like I did), OH would do a scheduled read of the item - in my case send a read request for that item on the KNX bus.

To make it a bit clearer. The persistence strategy never will reach out to the binding, which in turn reaches out to the KNX bus in your case and read the item’s state - that’s what @rlkoshak is saying all along.

In case of KNX you have to consider this:

1. why do you want to read the item’s state in intervals in the first place? if the item changes - the sensor/actuator will send a KNX telegram on the bus. openHAB will pick up the telegram and update the item accordingly. KNX is active, you don’t need to pull.
2. if you really, really, really like to have intervals on the KNX bus, you have to define your GA (in the ETS) to send the value in intervals - every KNX device (the actuators at least!) can be configured to do just this

There is no way to my knowledge to tell openHAB to read KNX GAs. If you configured your KNX correctly (I hope, your electrician did an awesome Job for it), there is actually no need for polling GAs in openHAB. As I said earlier - KNX will send all changes on the bus on itself and openHAB will get them.
What could be an issue is, if your GAs lack the readable flag. if openHAB starts, it will indeed read all the GAs it knows for the actual state - if they are not readable (as configured in the ETS again), openHAB will fail to update the state until this is either changed - or the GA sends the interval update.

Problem solved?

PS: even if you had everyMinute as your InfluxDB-strategy - if the item value is the same, this will only result in having 1440 identical items in your database per day. #justsayin!

[knxhab (Carsten Wegner)]

Thank you also, Thomas.

Yes, there are item states, which have to be read at intervals. For the KNX device can’t be configured to send them on change. If you’re curious: getting from a Stiebel Eltron Wärempumpe WPF07 wether it is working for heating or warming up water for the showers. Of course, you could compare temperatures, but I find this awkward, if I can get what I want by querying this state directly. Also, some runtimes can only be read this way.

Regarding the reading of GAs, well, it gave me some headache and I am not sure, if I have it done right now. As you probably know, there are quite some threads about OH and KNX. One thing is, that reading a GA results in a GroupValueWrite on the KNX bus, which is not how it should be. This is what I have experienced myself. So I went to the KNX binding docs and studied them again. I ended up with defining (in OH items file) a non-existent GA as the main GA for read-only KNX devices and a second GA, which is bound to the device channel in ETS. Now I get a correct GroupValueRead on the KNX bus.

Wether my electrian would have done an awesome job I can’t tell. I decided to do all the KNX stuff (and really a lot of other electrians stuff) myself. Has been my first contact with home automation and I think it’s done “OK” ;-)) ETS needs some getting used to, but that’s only my humble opinion.
During my time in ETS I tried to set an Read flag on every device channel where it was missing, so I hope not to run into the problem you mentioned.

And yes: problem solved