I am just thinking about replacing all of my extra items (just holding the timestamp of opening a door/window) with some metadata direkt at the contact-item.
I know that I may solve such problems with another persistance (like influx), but I want to discuss it in general.
So my question is just: Will there be bad sideeffekt by using metadata write-actions very often?
Like direct persistance on the filesystem. Or will metadata cached / hold in memory in some kind?
Greatings
Item metadata is data that describes the item in more detail e.g. like unit which describes the unit of the data the item holds. As such it’s configuration data which should be static. It’s like if you would be asking if you should save the timestamp e.g. as an item label. Mixing configuration and runtime data will always lead to issues down the line and is always a bad idea.
There is a configurable aggregation time for the jsondb writes and by default it’s set to 5secs. So changes will always be written every 5 secs.
Ok, thanks for explanation.
In this case I stay with my seperate item.
patterns like:
myHouse_myRoom_myWindow1
myHouse_myRoom_myWindow1_opensince
makes it also easy to grab them in depency during code execution.
(as mentioned until I feel confortble using influx for such timeshifts)
100% agree
Not quite as on board with this.
For my system, I use a lot of metadata. Much of it is static. On the other hand, a good deal of it is dynamic, because some “details” about an item are inherently context dependent. For example, as with many OH users, I have numerous items with expire metadata. For most of those that never changes. However, for some, such as the master bathroom lights, that value is different in the morning and in the evening.
For me the decision between whether or not to make some information a separate item vs. item metadata is a more complex decision tree. Obviously, information that should be persisted needs to be an item, as trying to persist metadata would just be reinventing the wheel. Information that is accessed or utilized by some process external to the item itself, such as a timestamp, needs to be its own item. Information that relates to how the item is used or the logical path that an item takes through a general process, however, is metadata.
Here’s another example with custom metadata that is probably the one that is the most dynamic in my setup: Many of my lights triggered by motion sensors are not just directly connected, but are connected via a complex rule. In this case, the behavior of the light is different if the light is turned on manually vs if it is turned on automatically. These lights have a metadata that records this description of the light switch: previousControl. When the sensor timer expires whether the light also switches off automatically or not depends on the value of that metadata. Could this also be done with an item? Of course, but from a philosophical standpoint, it makes more sense to me that this bit of information is metadata about the switch itself.
Can’t speak to the general dangers here. This certainly sounds like a voice with far more experience and wisdom in these matters than I have. I can only say that I have not noticed any issues arising from any of my dynamic metadata since I implemented several of these systems at the beginning of OH3.
How often?
No, metadata will get written to org.openhab.core.items.Metadata immediately, so it will increase the writes to disk. If you don’t need this information to be persisted on an OH restart you might consider using the cache if you don’t want to use an Item. However you’ll need a rule or script transform to do that so won’t be saving much in terms of configuration.
I would not save this info as Item metadata based on the description so far on how these timestamps are used and generated.
I like this idea!
I am currently using metadata to store several hundred kilobytes of data and treating it as a persistent json data storage.
But does the metadata describe the value that the switch item currently has? You are mixing the “mode of operation” of your light which is independent from the current state of the light into one item.
As you said it’s more of a philosophical question and if it works it works but I would have intuitively thought otherwise.
I would have expected three items:
One item for automatic mode, one item for manual control and a third item that decides which item to forward to the switch. That way automatic and manual are truly independent from each other.
HABApp provides the third item as a building block (MultiModeItem) and it’s how I do most of my rules now which can be influenced manually.
I dunno, seems to me that the majority of the “common” metadata namespaces are about function more than value, so this feels to me like it fits right in. In addition, the light sensor is one of the more straightforward examples. For a more complex example, here’s the metadata for one of my items related to timed alerts:
value: query
config:
message: Boy's bathroom light is on. Turn it off?
responseRule: 0a12633458
users:
- Toby
- Justin
Four additional items to track (one of which would require parsing the state back into an array from a string as well) seems like a lot of extra overhead. Now this metadata isn’t dynamic, but it is 100% about the mode of operation related to this particular item.
That’s a fascinating way of handling it. Occasionally I’m sorry that you rolled HAPApp out AFTER I migrated from jython to js. #NotChangingAllMyRulesAgain.
So essentially you are using the item metadata as a json storage because openHAB lacks a proper complex item type?
I feel your pain though - I’m using lot’s of HABApp internal items which can hold any arbitrary data structures. E.g. I have one item which holds the complete weather forecast for the next couple of days. When this item changes I then programmatically calculate e.g. if I have to do shading or frost protection.
It’s also much easier to create complex rules which seamlessly work together.
I have an automatic mode which opens and closes my roller shutter. Then I have an additional mode which does shading based on sun and weather forecast. Then a third one which does frost protection. A fourth one which doesn’t open the shutters when the automatic close from the automatic mode is less then half an hour away. And a fifth mode which is the manual control. All these get aggregated into a single state. Creating one complex rule would be way to difficult and error prone.
You can still play around with HABApp and/or use it (or the docs) as an inspiration to make your js rules different (ideally better 
). Automation rules are not an end in itself - it it works why change?
Thanks. That was the info I needed to put inside my OH-Codingguideline.
(My Server runs with an SSD but I want to reduce disk interactions if possible: Event *.log seens having destroyed some SD-cards.)
I use metadata for something similar. I set a flag on the Item to disable controlling the lights based on how cloudy it is if it’s been manually toggled. @JustinG’s approach is much more comprehensive though.
That’s one approach and a good one. It’s not the only possible approach though and it’s not even the only good approach.
And if metadata only described the value that the Item currently has, what about expire? That wouldn’t be allowed because that is functionality. Auto-update too has to do with functionality.
I’m completely on board with metadata being also for controlling how an Item behaves and/or is treated by a rule or rules. Most of my rule templates use metadata on the Item to control how that Item is treated. For example, Time Based State Machine uses metadata to identify the type of day the timestamp is for and what time STATE the timestamp represents. Debounce uses metadata to identify the proxy Item to debounce to and how long to debounce for. Threshold Alert can customize all the properties supported by the rule on an Item by Item basis (e.g. give a different timeout for individual Items, have a different do not disturb window, call a different alerting function, etc).
These are all functionality related, not just describing the current state of the Item. They define behaviors.
In case you haven’t seen, Jython has a new maintainer! It’s no longer marked as deprecated and is being actively worked on again. The developer is also looking into GraalVM Python so we can eventually get Python 3 support.
Persistence is probably a bigger issue. Depending on the strategies and the back end database/add-on persistence can cause more writes than events.log. For example, events.log doesn’t show updates by default.
I wouldn’t say that (at least for the cases I outline above, @JustinG does have differing uses which could benefit from more complex Item types). The types of the current set of Items is sufficient. What the metadata helps define is how that Item behaves (does it change on command or wait for the binding to update, does it remain in a given state forever or expire to some default state, does the rule treat the Item one way or a slightly different way, how does this Item appear to 3rd party integrations which often has a functional component?, etc.).
Adding in more expansive Item types wouldn’t help in those cases and right now Item metadata remains an excellent way and really the only way to define and apply these behaviors on an Item by Item basis.
How do you do this ? I need to change towel rails expire time depending on the season.
See Expire Updater [4.0.0.0;4.9.9.9] for how to change the expire metadata (or you can just use install and use the template. The rest is just triggering a rule at the right time to change the metadata (you could use Time Based State Machine [4.0.0.0;4.9.9.9] or Timeline picker to setup heating, light and so on to control the schedule).
Expire and (or at least) autoupdate should have been a core functionality which is directly set on the item and not in item metadata. A value for autoupdate is always assumed even when it’s not set and it changes fundamentally how an item works so there should have been a dedicated field for it.
You are using item metadata to parametrize your generic rules because currently there is no other way to do it. But this is data that is rule specific and thus should also be attached to the rule. It would be much nicer if you could click on “Debounce” under rules in the UI and it would show all debounce rules and how they are parametrized. And then you could change the parameters directly in the overview. Or click on a 
and add another debounce rule for an item and edit the parameters directly in the UI.
In HABApp creating multiple rules of the same type is cheap and easy. You just pass in the arguments to the constructor:
FadeInAlarm(
alarm_item='MobileAlarm1', output_mode='SleepingRoomRGB',
log_alarm=logging.getLogger('SleepingRoom.Alarm'),
log_mode=logging.getLogger('SleepingRoom.RGB'),
)
That’s the reason why I’m hardly using any metadata because my rule parameters are defined where my rule instantiated (or in a parameter file when I want to be able to change the value without having to re-instantiate the rule).
But I think this discussion is getting rather academical and off topic.
And as I said - if it works it works.
Sort of. Rule templates do have parameters which apply to the whole rule. It works pretty much as you show for HABApp.
But to define properties at the Item level we have metadata.
The only other way to handle it would be to instantiate a separate rule for each individual Item which is completely unmanageable and frankly rediculous in some cases. I’d have somewhere around 100 instances of my Threshold Alert rule if I had to instantiate a separate one other Item. I’d have 30 Denounce rules. And Time Based State Machine would be impossible in its current form (possibly not implementable at all).
The only reason I continue to push on this is because such “academic” discussions had already cost us YAML based file configs. I don’t want the perfect to override the good again.
No, I am creating a rule instance in the example above. Parametrizing a rule template is not the same as creating a rule instance.
Yes - that’s the idea behind a rule instance. And currently you’re also creating a pesudo instance by parametrizing the rule through item metadata and applying a generic rule on the item.
In reality you now already have the 100 Threshold Alert rules by defining 100 different metadata entries on the item.
I figured you’re still salty at me but that’s uncalled-for and unfair.
I’ve created multiple github issues, poured an enormous amount of thought into a good file format, thought of rules which make the format easy to understand and easy to work with.
And when I comment on a PR that is essentially the json.db just as a yaml that this file format is not very user friendly that should be what killed the new file format?
My brought up points could also have been addressed and I’ve signaled multiple times that I’m willing to provide input and ideas. All that got ignored and the tone was like: file based user want to suffer anyway so here’s something to suffer with.
I’m aware everyone is working on stuff in their free time and I’m thankful for that.
But between take it or leave it and I’ll implement it exactly like you suggested is a huge place for compromise and also a huge chance for improvement.
And that’s what one does with a rule template too. Role templates don’t do anything on their own. You need to instantiate a rule based on a template after setting the parameters. Same as you demonstrate. The only difference is the parameters are “passed” into the rule through string substitution instead of function arguments.
And yet when there is an error I have one place to look. I have one ruleuid to know. I have one entry in settings → rules. If I had 100 it would be unmanageable.
I’m not saying instantiating a separate rule per Item is wrong by any means and that is possible today if that’s how you prefer to do it both in rules files (not Rules DSL) and through the UI.
I am saying that making it so that’s the only way is wrong. It cuts off whole avenues of ways of using OH and so far the only benefit I can see is it becomes more “pure” by one measure.
Jan said stop or I quit. You didn’t do he did. Yes I am salty about that.
All your concerns could have been addressed individually after a first minimal viable version was released by you, or other devs (maybe even me).
And yet you persisted despite the warnings because, all I can surmise, it was better to not have it at all than to not have a perfect (by your individual measures) first release. There was even a way to provide backwards compatibility so these future changes wouldn’t have to be breaking changes.
That’s what I’m still miffed about. And after killing it, no one on that pr has picked it back up. (note I’ve been trying to get a working dev environment so I can but it’s really slow going but I do plan on giving it a go when/if I’m able).
And this thread is kind of the same thing. You prefer to keep functional stuff or if item metadata for reasons, so OH should force everyone to do so too or else it’s wrong.
I do appreciate all you do for OH in all ways. I think HABApp is great and promote it where it makes sense. But stuff like this I think is damaging to OH over all and I have to push back.
Absolutely true. What’s missing from this description is the mechanism by which that compromise is achieved. Your comments here and the repo are always thoroughly thought-out, well-articulated and supported with clear data/examples. They are also, however, occasionally presented with an air of authority and vehemence that misses the mark on that compromise middle ground (i.e., leaving zero indication that you consider compromise acceptable on some particular point). Not intended, I’m sure; I work very hard against the same tendency myself and occasionally miss as well (which my wife will be more than happy to verify).
This is where, to an outside observer, Rich’s assessment that you pushed for “the perfect to override the good”, in fact, looks fair and doesn’t follow your own identification of that middle ground.
The PR in question is one of those times when your aim was off. You had some valid points, but none of them worthy of elevation to “die on this hill”, however, many of them were presented that way. I don’t blame Jan for simply deciding that his time could be better spent elsewhere; he wants to code and move OH forward for everyone not argue every fine philosophical point. I also agree with Rich that discouraging him from working on that PR (whether intentional or not) was a huge loss for OH progress.
You even began this thread with similar no-room-for-discussion phrasing:
There’s no qualification there. There’s no IMHO. I was happy to engage with my own viewpoint because I know from experience that you will more often than not be reasonable and see compromise in the face of other well thought-out and well presented view points. Sure enough:
But even then, you couldn’t resist the implication that yours was the only “correct” interpretation and that the flaws were in OH and others’ implementations:
If you want to put the lie to Rich’s assessment, maybe revive the PR thread and try apologizing to Jan
and explicitly stating a willingness to work with, not against, him on getting a trial new file format going. It’s clear that is something you also want, and that’s a good way to get there for eveyone.
I think you are misunderstanding what I am suggesting and I think it’s mainly because we use instantiating in a different context. Additionally I think you are misunderstanding that I am suggesting that this should be the only way to create rules.
I’ll try again to describe what I meant:
Let’s say there is a new rule type “Parametrized Rule”. When you create it you have to describe the parameters it takes e.g. the type (string, date or even custom types), the name of the parameter and an optional description.
Parametrized Rule doesn’t do anything by itself, you have to instantiate it to make it work. When you instantiate it you have to supply the non optional parameters and they are bound to the instance of the rule - just like a class and a class instance. The internal variables of Parameterized Rule are bound to an instance so each instance is a rule of it’s own.
So you still have only one Parametrized Rule and one place to look at if things go wrong but a quick and easy way to apply this rule multiple times. If you modify the Parametrized Rule all the instances get modified, too.
In the UI you would click on the Parametrized Rule and then see all the instances of the rule with the parameters and a + button to add another, etc.
The main benefit is that there is a structured and fail proof mechanism of providing the rule parameters and instance specific variables. No more parsing and validating the item metadata in each rule because that’s done by the Parametrized Rule functionality. The UI can immediately show an error message if the parameters are wrong.
The UI can provide hints and have a nice dialogue where one can enter the parameters. No more typos and no more errors in logs because of wrong metadata.
Also it would be easy to create multiple instances because one could provide a table where one could enter the parameters or select items from a dialog and create instances for them. That would be much more user friendly then tediously creating metadata for each item.
Additionally it would be very easy to get an overview for e.g. which items there is a ThresholdAlarm.
Thanks for the hint - that’s not my intention and I’ll try to pay more attention to it. If you see some posts from me again where this is the case it would be nice if you could just pm me so I can readjust.
I’m not really sure what I should apologize for. Do you feel that I have attacked him or mistreated him in any way? If so please point out because that is and has been by no means my intention.
I signaled multiple times that I would love to provide conceptual input and work together for a new file format, too. I’ve even tried to create a sample file format to get some feedback. Unfortunately I have not been taken up on that offer on working on a file format together nor have I gotten much feedback. I’m not sure that if I write it again it’ll change something.
But there has been no indication that changing the file format in a way that addresses any of my issues will be feasible in the future. There has been a clear statement that textual configuration is an advanced approach which should be “close to code” thus more or less locking the file format.
Of course I then try to improve my argument in case there is a slight chance that at least some points will be addressed.
Please stop phrasing it that way because it implies that I have some kind of decisional power over which PR gets merged and which doesn’t. I have never even written once that this PR can not or should not be merged.
So far this is exactly what a rule template is.
Again, still sounds like a rule template with the only minor difference being that the parameters are bound to the template through string substitution. If I have a parameter named “item” then everywhere in the rule where {{ item }} appears it will be replaces with the value of the item parameter.
The parameters chosen are also recorded in the instantiated rule.
But I don’t. I have 100 instantiations of the parameterized rule. And each of those need to be identifiable somehow so I know it’s failing with this set of parameters instead of that set of parameters. And since that’s what ruleUIDs are for…
This is the only part that isn’t yet supported but I’m OK with that becoming supported at a later date because rule templates are incredibly useful even without it. For now, to change the parameters of an instantiated rule you must modify the rule or delete and reinstantiate.
But setup and configuration becomes much more tedious. For example, my threashold alert rule template has somewhere around 20 parameters. You are suggesting I much set all 19 parameters for each and every individual Item because each Item gets it’s very own rule.
Right now I can define all 20 parameters once on the rule for all Items and override just the one or two parameters for those rare cases where an individual Item needs something different (e.g. a different timeout for a door open reminder). If you take that ability away, with 100 Items I’m looking at 1900 parameters I need to set instead of maybe 25-30 parameters once they are all added up.
When/if this exists I’m sure it would be useful and used. But it doesn’t negate the fact that is doesn’t make sense in all cases where setting a parameter at the Item sometimes also makes sense.
I don’t create metadata for each Item in all cases. See above.
But let’s talk about Time State Machine for a moment because this parameterized rule approach basically makes that template impossible.
And clearly there is demand for the template as it’s one of my most downloaded and used templates despite the fact that creating one rule per Item is already possible in MainUI today. All Time Base State Machine does is set a String Item to a value at the indicated times in the Items, with the added quirk that a different set of Items is used based on the type of the day according to Ephemeris. The metadata in this case is used to identify what type of day a given Item belongs to and what string to send to the state Item.
I’m not arguing that what you propose is not a good idea. It is. But it’s not a replacement for all the ways Item metadata is used now just by me and I’m certain many other users as well, and insisting that is will break a lot of stuff and/or make my end user’s experience worse, not better.
And of course the whole conversation is moot until something like you propose actually exists.
A built in versioning so that OH can support different versions of the config file isn’t strong enough indication that the format is expected to change in the future? Why have versioning at all if it’s expected to remain unchanged forever.
It is true that you may not get everything you are asking for ever or it may take some time, but it was very clear to me that support for changing it in the future was built in from the start.
In addition, what else in OH comes out fully formed in the first PR and never changes? We couldn’t get anywhere if that were the case.
That’s not how I interpret that. And frankly, I’m not sure that would apply here if the main thing I want out of it you appear to fight against (perhaps it’s a misunderstanding) where the YAML you see in the UI is identical to the YAML you see in the file.Then it’s the same format for everyone and there is no more file based users vs. managed config users dichotomy. The only big difference would be where the YAML is viewed and how it’s edited. And those who want to go to a managed config or from a managed config can easily do so since a full import and export capability would become possible.
But this is the part where it went wrong. Jan made it clear he didn’t want a repeat of the UoM PR. If this turned into that he was out. By persisting the argument you essentially vetoed any chance that the PR would get worked. Even if we assume that the format will never change, is it really better that we don’t get any YAML file support at all? Because by persisting in arguing on that PR that’s what you guaranteed. And that’s why I’m salty about it.
But in this case you did. Jan clearly stated if the arguments continue he was going to quit it. So either you thought it was a bluff on his part, or you were OK with the PR being dropped if you didn’t get your way. In this case you did have that power and you exercised it with great effect.