Proposal: Micrometer Persistence Service

Add-ons Persistence Services
, , ,
1

Hi All, looking or opinions, thoughts, feedback, etc on an idea that I have for creating a Micrometer persistence service for Item Status metrics.

A few years back the Metrics Service was added, with a focus on openHAB internal/core metrics - the health of the service itself.

This included adding a Micrometer registry to collect said metrics, and a Metrics service to expose those metrics, either for scraping by Prometheus or being sent to InfluxDB.

My thought is that we (I) can build on top of this by adding a Persistence service that creates Micrometer Gauges for Item States, which will be registered with the already existing Micrometer registry that is part of openHAB core. Doing so will automatically expose these Item statuses to Prometheus (my reason for this), but would also allow these to be sent to InfluxDB along with the core internal metrics.

This would be slightly different from the current InfluxDB Persistence service, which also supports InfluxDB v2, and (more importantly?) is able to be queried for Item status to restore states on restart, query historic data, etc. This Micrometer Persistence service would be uni-directional - data goes in, but cannot be pulled back out for use within openHAB itself.

Thoughts? Worth doing and could serve a purpose? I have seen a number of folks asking how to do this, but there has not been much chatter on those topics.

I have not contributed to openHAB yet, other than a few ZWave device definitions, but I am a senior Java engineer / architect for a living, so am comfortable on that side of things.

Thanks in advance for any feedback!
ā€“Scott

Tagging some folks that I think might be interested or have some input: @friesenkiwi, @pravussum, @utdrmac, @peterhoeg, sorry for the noise if I have overstepped.

2

What would those additional metrics be?

3

Item states.

I edited my original post to be explicit about this.

4

As a matter of principle, the more metrics we make available the better. I have no immediate personal need for it, but Iā€™d definitely say ā€œgo for itā€.

1 Like
5

Frankly I donā€™t understand the point of your need and what you would aim to implement. Any OH persistence service records item states that is its primary purpose and works with any service implementation be it the default rrd4j or influxdb. And if you want more metrics just create items for them.
Now as you say youā€™re able and willing to contribute, thatā€™s great! Iā€™d suggest you check out open GitHub issues on openhab-core that are persistence related to spend your valuable time on. Those issues affect potentially many people, making any contribution that improves useabilty an even more valuable one.

6

Thanks @mstormi, the goal for me is to make it possible to have Prometheus pull Item state metrics.

There is currently not a Persistence service that will expose an endpoint for Prometheus to pull Item state metrics from. A straightforward way to do this would be to use Micrometer Meters, and to expose an endpoint that accesses the data from those Meters, since Micrometer has native support for outputting metrics in Prometheus format.

However, the Metrics Service addon already does all that, except that it only has Micrometer Meters for core / internal metrics. So rather than duplicate what the Metrics Service is already doing, it makes sense to me to just add (through a new Persistence service) Meters for the Item states to the existing and working framework.

Should others choose to extend this to other metrics systems that Micrometer supports, that would also be quite simple to do.

ā€“Scott

7

Hmm I donā€™t know Prometheus and all I know is thereā€™s not many users so see my response as a generalized one from an OH maintainerā€™s perspective rather than as meant to help solve your pretty specific use case.
From a software architecture POV if you need anything in your 3rd party addon (Prometheus) you should not access OH internals (a specific persistence implementation) but use the REST API. That in turn is supposed to work with any persistence implementation.
One thing about the OH project is itā€™s already so complex that we should not add singular solutions for specific use cases when the existing system can be tweaked to provide the desired functionality.
For sure I donā€™t want to stop you from anything but you asked for opinions and I have the feeling that your talent and appreciated willingness to contribute is more efficiently spent elsewhere.

8

@mstormi, I am wondering if my description of the proposal is not very clear :frowning: Prometheus is a metrics server, very similar in scope and purpose to InfluxDB. There is an existing official Persistence Service add on - InfluxDB (0.9 and newer) - Persistence Services | openHAB.

What I am proposing is an implementation of the Persistence Service interface, to sit on top of openHAB using the official exposed interfaces, I donā€™t believe that I would be touching any of the internals of openHAB any more than the other Persistence Services. I donā€™t think that this is a very specific to me use case, this feels like something that would be useful to many others as well.

Thanks for your patience!
ā€“Scott

9

Prometheus collects data in a different way than existing persistence API usually work on openHAB end. For sure querying will be possible, yet on writing partā€¦ prometheus is a bit specific and will require metric (item) scrape endpoint.

I had done something around this regard, and you definitely do not need micrometer or jakarta for that. Prometheus format can be generated out of basic i puts. Main problem I see is normalization of units so prometheus sees them same as OH intents them to be.

See this: lokahi/minion/minion-observability/observability-metrics/src/main/java/org/opennms/horizon/minion/observability/metrics/internal/MetricsServlet.java at develop Ā· OpenNMS-Cloud/lokahi Ā· GitHub (warning all opennms code is licensed under AGPL).

10

@splatch , definitely, Micrometer is not needed, there are many other ways, from completely DIY to libraries and frameworks other than Micrometer.

I was leaning towards using Micrometer as it already is part of openHAB, and exposed for Prometheus and/or InfluxDB through the Metrics Service addon (endpoint implementation is here), so I did not want to re-invent the wheel with a parallel implementation of a second endpoint for Prometheus to pull from.

ā€“Scott

1 Like
11

Ran into Metrics service not reporting thing properties Ā· Issue #11088 Ā· openhab/openhab-addons Ā· GitHub, which I believe my proposal would be following the recommendations there.

I am proposing to create a Persistence addon that creates Micrometer meters, which can then be exposed through the Metrics Service endpoint.

CC: @Kai

12

Persistence services are mostly about items and not things, so you might not be allowed/permitted to pull thing metadata.

13

I am looking to provide item states. The Metrics Service already does thing data.

The terminology used on the referenced issueā€™s title says thing properties, but in the body of the description, what is being asked for is item properties.

Expecting to be able to see the values of the channels provided by my zwave devices (temp, on/off state, kwh, etc) so i can scrape them into prometheus for grafana viewing.