I dunno, seems to me that the majority of the “common” metadata namespaces are about function more than value, so this feels to me like it fits right in. In addition, the light sensor is one of the more straightforward examples. For a more complex example, here’s the metadata for one of my items related to timed alerts:
value: query
config:
message: Boy's bathroom light is on. Turn it off?
responseRule: 0a12633458
users:
- Toby
- Justin
Four additional items to track (one of which would require parsing the state back into an array from a string as well) seems like a lot of extra overhead. Now this metadata isn’t dynamic, but it is 100% about the mode of operation related to this particular item.
That’s a fascinating way of handling it. Occasionally I’m sorry that you rolled HAPApp out AFTER I migrated from jython to js. #NotChangingAllMyRulesAgain.
So essentially you are using the item metadata as a json storage because openHAB lacks a proper complex item type?
I feel your pain though - I’m using lot’s of HABApp internal items which can hold any arbitrary data structures. E.g. I have one item which holds the complete weather forecast for the next couple of days. When this item changes I then programmatically calculate e.g. if I have to do shading or frost protection.
It’s also much easier to create complex rules which seamlessly work together.
I have an automatic mode which opens and closes my roller shutter. Then I have an additional mode which does shading based on sun and weather forecast. Then a third one which does frost protection. A fourth one which doesn’t open the shutters when the automatic close from the automatic mode is less then half an hour away. And a fifth mode which is the manual control. All these get aggregated into a single state. Creating one complex rule would be way to difficult and error prone.
You can still play around with HABApp and/or use it (or the docs) as an inspiration to make your js rules different (ideally better ). Automation rules are not an end in itself - it it works why change?
Thanks. That was the info I needed to put inside my OH-Codingguideline.
(My Server runs with an SSD but I want to reduce disk interactions if possible: Event *.log seens having destroyed some SD-cards.)
I use metadata for something similar. I set a flag on the Item to disable controlling the lights based on how cloudy it is if it’s been manually toggled. @JustinG’s approach is much more comprehensive though.
That’s one approach and a good one. It’s not the only possible approach though and it’s not even the only good approach.
And if metadata only described the value that the Item currently has, what about expire? That wouldn’t be allowed because that is functionality. Auto-update too has to do with functionality.
I’m completely on board with metadata being also for controlling how an Item behaves and/or is treated by a rule or rules. Most of my rule templates use metadata on the Item to control how that Item is treated. For example, Time Based State Machine uses metadata to identify the type of day the timestamp is for and what time STATE the timestamp represents. Debounce uses metadata to identify the proxy Item to debounce to and how long to debounce for. Threshold Alert can customize all the properties supported by the rule on an Item by Item basis (e.g. give a different timeout for individual Items, have a different do not disturb window, call a different alerting function, etc).
These are all functionality related, not just describing the current state of the Item. They define behaviors.
In case you haven’t seen, Jython has a new maintainer! It’s no longer marked as deprecated and is being actively worked on again. The developer is also looking into GraalVM Python so we can eventually get Python 3 support.
Persistence is probably a bigger issue. Depending on the strategies and the back end database/add-on persistence can cause more writes than events.log. For example, events.log doesn’t show updates by default.
I wouldn’t say that (at least for the cases I outline above, @JustinG does have differing uses which could benefit from more complex Item types). The types of the current set of Items is sufficient. What the metadata helps define is how that Item behaves (does it change on command or wait for the binding to update, does it remain in a given state forever or expire to some default state, does the rule treat the Item one way or a slightly different way, how does this Item appear to 3rd party integrations which often has a functional component?, etc.).
Adding in more expansive Item types wouldn’t help in those cases and right now Item metadata remains an excellent way and really the only way to define and apply these behaviors on an Item by Item basis.
Expire and (or at least) autoupdate should have been a core functionality which is directly set on the item and not in item metadata. A value for autoupdate is always assumed even when it’s not set and it changes fundamentally how an item works so there should have been a dedicated field for it.
You are using item metadata to parametrize your generic rules because currently there is no other way to do it. But this is data that is rule specific and thus should also be attached to the rule. It would be much nicer if you could click on “Debounce” under rules in the UI and it would show all debounce rules and how they are parametrized. And then you could change the parameters directly in the overview. Or click on a and add another debounce rule for an item and edit the parameters directly in the UI.
In HABApp creating multiple rules of the same type is cheap and easy. You just pass in the arguments to the constructor:
That’s the reason why I’m hardly using any metadata because my rule parameters are defined where my rule instantiated (or in a parameter file when I want to be able to change the value without having to re-instantiate the rule).
But I think this discussion is getting rather academical and off topic.
And as I said - if it works it works.
Sort of. Rule templates do have parameters which apply to the whole rule. It works pretty much as you show for HABApp.
But to define properties at the Item level we have metadata.
The only other way to handle it would be to instantiate a separate rule for each individual Item which is completely unmanageable and frankly rediculous in some cases. I’d have somewhere around 100 instances of my Threshold Alert rule if I had to instantiate a separate one other Item. I’d have 30 Denounce rules. And Time Based State Machine would be impossible in its current form (possibly not implementable at all).
The only reason I continue to push on this is because such “academic” discussions had already cost us YAML based file configs. I don’t want the perfect to override the good again.
No, I am creating a rule instance in the example above. Parametrizing a rule template is not the same as creating a rule instance.
Yes - that’s the idea behind a rule instance. And currently you’re also creating a pesudo instance by parametrizing the rule through item metadata and applying a generic rule on the item.
In reality you now already have the 100 Threshold Alert rules by defining 100 different metadata entries on the item.
I figured you’re still salty at me but that’s uncalled-for and unfair.
I’ve created multiple github issues, poured an enormous amount of thought into a good file format, thought of rules which make the format easy to understand and easy to work with.
And when I comment on a PR that is essentially the json.db just as a yaml that this file format is not very user friendly that should be what killed the new file format?
My brought up points could also have been addressed and I’ve signaled multiple times that I’m willing to provide input and ideas. All that got ignored and the tone was like: file based user want to suffer anyway so here’s something to suffer with.
I’m aware everyone is working on stuff in their free time and I’m thankful for that.
But between take it or leave it and I’ll implement it exactly like you suggested is a huge place for compromise and also a huge chance for improvement.
And that’s what one does with a rule template too. Role templates don’t do anything on their own. You need to instantiate a rule based on a template after setting the parameters. Same as you demonstrate. The only difference is the parameters are “passed” into the rule through string substitution instead of function arguments.
And yet when there is an error I have one place to look. I have one ruleuid to know. I have one entry in settings → rules. If I had 100 it would be unmanageable.
I’m not saying instantiating a separate rule per Item is wrong by any means and that is possible today if that’s how you prefer to do it both in rules files (not Rules DSL) and through the UI.
I am saying that making it so that’s the only way is wrong. It cuts off whole avenues of ways of using OH and so far the only benefit I can see is it becomes more “pure” by one measure.
Jan said stop or I quit. You didn’t do he did. Yes I am salty about that.
All your concerns could have been addressed individually after a first minimal viable version was released by you, or other devs (maybe even me).
And yet you persisted despite the warnings because, all I can surmise, it was better to not have it at all than to not have a perfect (by your individual measures) first release. There was even a way to provide backwards compatibility so these future changes wouldn’t have to be breaking changes.
That’s what I’m still miffed about. And after killing it, no one on that pr has picked it back up. (note I’ve been trying to get a working dev environment so I can but it’s really slow going but I do plan on giving it a go when/if I’m able).
And this thread is kind of the same thing. You prefer to keep functional stuff or if item metadata for reasons, so OH should force everyone to do so too or else it’s wrong.
I do appreciate all you do for OH in all ways. I think HABApp is great and promote it where it makes sense. But stuff like this I think is damaging to OH over all and I have to push back.
Absolutely true. What’s missing from this description is the mechanism by which that compromise is achieved. Your comments here and the repo are always thoroughly thought-out, well-articulated and supported with clear data/examples. They are also, however, occasionally presented with an air of authority and vehemence that misses the mark on that compromise middle ground (i.e., leaving zero indication that you consider compromise acceptable on some particular point). Not intended, I’m sure; I work very hard against the same tendency myself and occasionally miss as well (which my wife will be more than happy to verify).
This is where, to an outside observer, Rich’s assessment that you pushed for “the perfect to override the good”, in fact, looks fair and doesn’t follow your own identification of that middle ground.
The PR in question is one of those times when your aim was off. You had some valid points, but none of them worthy of elevation to “die on this hill”, however, many of them were presented that way. I don’t blame Jan for simply deciding that his time could be better spent elsewhere; he wants to code and move OH forward for everyone not argue every fine philosophical point. I also agree with Rich that discouraging him from working on that PR (whether intentional or not) was a huge loss for OH progress.
You even began this thread with similar no-room-for-discussion phrasing:
There’s no qualification there. There’s no IMHO. I was happy to engage with my own viewpoint because I know from experience that you will more often than not be reasonable and see compromise in the face of other well thought-out and well presented view points. Sure enough:
But even then, you couldn’t resist the implication that yours was the only “correct” interpretation and that the flaws were in OH and others’ implementations:
If you want to put the lie to Rich’s assessment, maybe revive the PR thread and try apologizing to Jan
and explicitly stating a willingness to work with, not against, him on getting a trial new file format going. It’s clear that is something you also want, and that’s a good way to get there for eveyone.
I think you are misunderstanding what I am suggesting and I think it’s mainly because we use instantiating in a different context. Additionally I think you are misunderstanding that I am suggesting that this should be the only way to create rules.
I’ll try again to describe what I meant:
Let’s say there is a new rule type “Parametrized Rule”. When you create it you have to describe the parameters it takes e.g. the type (string, date or even custom types), the name of the parameter and an optional description. Parametrized Rule doesn’t do anything by itself, you have to instantiate it to make it work. When you instantiate it you have to supply the non optional parameters and they are bound to the instance of the rule - just like a class and a class instance. The internal variables of Parameterized Rule are bound to an instance so each instance is a rule of it’s own.
So you still have only one Parametrized Rule and one place to look at if things go wrong but a quick and easy way to apply this rule multiple times. If you modify the Parametrized Rule all the instances get modified, too.
In the UI you would click on the Parametrized Rule and then see all the instances of the rule with the parameters and a + button to add another, etc.
The main benefit is that there is a structured and fail proof mechanism of providing the rule parameters and instance specific variables. No more parsing and validating the item metadata in each rule because that’s done by the Parametrized Rule functionality. The UI can immediately show an error message if the parameters are wrong.
The UI can provide hints and have a nice dialogue where one can enter the parameters. No more typos and no more errors in logs because of wrong metadata.
Also it would be easy to create multiple instances because one could provide a table where one could enter the parameters or select items from a dialog and create instances for them. That would be much more user friendly then tediously creating metadata for each item.
Additionally it would be very easy to get an overview for e.g. which items there is a ThresholdAlarm.
Thanks for the hint - that’s not my intention and I’ll try to pay more attention to it. If you see some posts from me again where this is the case it would be nice if you could just pm me so I can readjust.
I’m not really sure what I should apologize for. Do you feel that I have attacked him or mistreated him in any way? If so please point out because that is and has been by no means my intention.
I signaled multiple times that I would love to provide conceptual input and work together for a new file format, too. I’ve even tried to create a sample file format to get some feedback. Unfortunately I have not been taken up on that offer on working on a file format together nor have I gotten much feedback. I’m not sure that if I write it again it’ll change something.
But there has been no indication that changing the file format in a way that addresses any of my issues will be feasible in the future. There has been a clear statement that textual configuration is an advanced approach which should be “close to code” thus more or less locking the file format.
Of course I then try to improve my argument in case there is a slight chance that at least some points will be addressed.
Please stop phrasing it that way because it implies that I have some kind of decisional power over which PR gets merged and which doesn’t. I have never even written once that this PR can not or should not be merged.
Again, still sounds like a rule template with the only minor difference being that the parameters are bound to the template through string substitution. If I have a parameter named “item” then everywhere in the rule where {{ item }} appears it will be replaces with the value of the item parameter.
The parameters chosen are also recorded in the instantiated rule.
But I don’t. I have 100 instantiations of the parameterized rule. And each of those need to be identifiable somehow so I know it’s failing with this set of parameters instead of that set of parameters. And since that’s what ruleUIDs are for…
This is the only part that isn’t yet supported but I’m OK with that becoming supported at a later date because rule templates are incredibly useful even without it. For now, to change the parameters of an instantiated rule you must modify the rule or delete and reinstantiate.
But setup and configuration becomes much more tedious. For example, my threashold alert rule template has somewhere around 20 parameters. You are suggesting I much set all 19 parameters for each and every individual Item because each Item gets it’s very own rule.
Right now I can define all 20 parameters once on the rule for all Items and override just the one or two parameters for those rare cases where an individual Item needs something different (e.g. a different timeout for a door open reminder). If you take that ability away, with 100 Items I’m looking at 1900 parameters I need to set instead of maybe 25-30 parameters once they are all added up.
When/if this exists I’m sure it would be useful and used. But it doesn’t negate the fact that is doesn’t make sense in all cases where setting a parameter at the Item sometimes also makes sense.
I don’t create metadata for each Item in all cases. See above.
But let’s talk about Time State Machine for a moment because this parameterized rule approach basically makes that template impossible.
And clearly there is demand for the template as it’s one of my most downloaded and used templates despite the fact that creating one rule per Item is already possible in MainUI today. All Time Base State Machine does is set a String Item to a value at the indicated times in the Items, with the added quirk that a different set of Items is used based on the type of the day according to Ephemeris. The metadata in this case is used to identify what type of day a given Item belongs to and what string to send to the state Item.
I’m not arguing that what you propose is not a good idea. It is. But it’s not a replacement for all the ways Item metadata is used now just by me and I’m certain many other users as well, and insisting that is will break a lot of stuff and/or make my end user’s experience worse, not better.
And of course the whole conversation is moot until something like you propose actually exists.
A built in versioning so that OH can support different versions of the config file isn’t strong enough indication that the format is expected to change in the future? Why have versioning at all if it’s expected to remain unchanged forever.
It is true that you may not get everything you are asking for ever or it may take some time, but it was very clear to me that support for changing it in the future was built in from the start.
In addition, what else in OH comes out fully formed in the first PR and never changes? We couldn’t get anywhere if that were the case.
That’s not how I interpret that. And frankly, I’m not sure that would apply here if the main thing I want out of it you appear to fight against (perhaps it’s a misunderstanding) where the YAML you see in the UI is identical to the YAML you see in the file.Then it’s the same format for everyone and there is no more file based users vs. managed config users dichotomy. The only big difference would be where the YAML is viewed and how it’s edited. And those who want to go to a managed config or from a managed config can easily do so since a full import and export capability would become possible.
But this is the part where it went wrong. Jan made it clear he didn’t want a repeat of the UoM PR. If this turned into that he was out. By persisting the argument you essentially vetoed any chance that the PR would get worked. Even if we assume that the format will never change, is it really better that we don’t get any YAML file support at all? Because by persisting in arguing on that PR that’s what you guaranteed. And that’s why I’m salty about it.
But in this case you did. Jan clearly stated if the arguments continue he was going to quit it. So either you thought it was a bluff on his part, or you were OK with the PR being dropped if you didn’t get your way. In this case you did have that power and you exercised it with great effect.
There is no direct attack, of course, but sometimes apologies are about what you didn’t do, instead of what you did. In this case, I think Jan would have a legitimate claim that you disreagrded/didn’t pick up on his obvious frustration. Rich’s description of the events is colored by his frustration with the outcome as well, but he is essentially describing the same thing. Jan said more or less directly, “the time I would put into this feature isn’t worth this drama,” and your next post did not attack him personally or professionally, but did fail to acknowledge that what you were posting was the source of that drama he was referring to.
100% valid, and I agree with many of the points you put forth in that issue and follow-ups that you put in Jan’s PR.
What I’m saying is that the when Jan signaled exasperation there’s no evidence that you took the time for introspection. It doesn’t really look like you tried to find that “huge place for compromise”. I read many of your comments as saying, “let me help you get your contribution to match exactly what I see as the only solution.” Not many would truly interpret that as help. And that comes back to what Rich is saying, I think. What you posted should, instead, have summed to “let’s get this to a place that, even if not exactly what I was expecting, sets everyone off in the right direction, with the chance to reach (eventually) something special in the long run.”
Edit:
I’ve only been on these boards extensively for maybe 5 or 6 years. But even in that time alone, I can think of a couple of situations where clashes that started out as reasonable “philosophical differences” such as this escalated to sufficient bad blood to cost OH excellent contributors. I’d hate to see that happen here in either the case of you or Jan.
I have never suggested that. Parameters that have a default value are optional because if omitted the default is used. So you would have to set only one parameter and additionally only those that changed:
It’s exactly the same amount of configuration you have to do.
Just that instead of spreading the configuration over several items it’s in a central place and you get the benefit that not every rule has to do the metadata validation and a nice user facing parameter input dialogue.
Really - the UoM discussion? Where people were essentially commenting “I didn’t read the whole thread and didn’t understand the issue in detail but how about this and that?” Where I tried my best to keep every one on the same page and up to date and in the end everybody (including me) was extremely annoyed that the discussion was running in circles? Because yet another person asked a question or made a statement that had already been answered multiple times. That UoM discussion?
Also his resentment was not clear at all. We’ve had a discussion shortly before on another issue where I suggested that metadata should not exist alone but be bound to the item. I’ve attributed his comment “If it’s that discussion again, no.” to the merge of item and metadata because one could understand the suggested configuration format in that way. But just because the file format provides an option to configure it in a certain way (a way that it already provides now) does not mean the underlying data structure has to be the same as the file format. That’s what I tried to make clear with the next comment.
I fully agree that this is how things should work and have stated that multiple times. It would be a great benefit for all users. I even suggested that it should be a shared core functionality that should do the (de-)serialisation of the yaml so the core, UI and file based is always in sync.
I feel like we’re running in circles again and these discussions are extremely exhausting and stressful for me. I think all has been said and nothing new or good will come if we continue.
It’s been made clear that my ideas and suggestions are an active hindrance for the openHAB development so I’ll refrain from making them in the future.
No, no, no, no no. Please don’t. Ideas and suggestions are never a hindrance.
In that case, I apologize for any part in causing stress. I agree this is approaching running in circles again, but I had some hope we would escape that for a while there.
This all comes back to the old saying:
Communication isn’t about what you say. It’s about what they hear.
If you are hearing that your ideas are not appreciated, then that is on us. Your ideas have driven many wonderful advancements to OH. Your contributions stand on their own. I’m trying to work towards a solution where your ideas again flow and move OH.
Jan’s not blameless here either (but he’s also not on this thread, or at least hasn’t made his presence known), and could have done a better job with the “what they hear” part. On the other hand, my impression from interacting with him over these years is that he is a pretty easy-going, level-headed guy and the fact that he got to the point where he lashed out about killing file config altogether is a huge red-flag about just how far off course communication had gotten in that thread.
It only takes one person so say “let’s find the points we all agree on…” instead of “I agree, but you’re wrong…” for everything to get back on track. It could be any of you…
Edit…
If I had a nickle for every time I’ve heard “I agree, but you’re wrong…” in faculty meetings…
No, they are not and please do not refrain from making them.
However, I think the way you present them can be problematic at times. As @JustinG indicated, you don’t leave room for compromise in how you state things and when your suggestions are rejected, if you don’t agree, you continue to argue for it.
That in itself also causes stress and and exhaustion. And that stress and exhaustion is why Jan quit the YAML PR. No one wants that. I don’t want that. I don’t want you to have that.
In general I like your ideas and they move OH forward. I only push back when the idea breaks my stuff.
Yes, and despite the efforts of you, and me, and others, the discussion went off the rails. It didn’t go off the rails because of you and you did great work trying to keep it on track yet it was pretty much a nightmare.
And if you recall, he almost gave up on that PR too. I can’t blame him for not wanting a repeat of that.
I’m not sighting your behavior on that UoM PR. I’m sighting the overall experience which you were trying to mitigate on that PR and for which I’m grateful. I did not intend to mean your behavior on that PR was anything but exemplary.
At the end of it I think your idea to make rule parameterization in the UI is a good one. It should be pursued. I only disagree with that replacing the way I use Item metadata.
I think you vastly overestimate how much is actually going into the Item metadata and vastly underestimating the amount of work that would be imposed on the end users of the Threshold Alert rule template if they were forced to instantiate a separate rule per Item.
But that’s OK. Just don’t break my stuff by insisting I can’t use Item metadata the way I currently do and it’s all good.
I spent a lot of time on the Threshold Alert rule template. I implemented it at least a dozen different ways. I don’t reject your idea because I don’t understand the idea of parameterizing on the rules. I don’t reject your idea at all. In fact I always implement my rule templates that way first. But it invariably either makes the template essentially impossible to implement, or makes it become too much work for the users of the template and I need to resort to Item metadata.
I’ll say it again, I’m not against the idea of parameterization of rules. I am only against making my rule templates worse by not allowing them to use Item metadata. Your proposal is a good one but not sufficient in my use cases.
Just to put some real numbers to it, lets take the most extreme of my instances of Threshold Alert.
Number of Parameters: 16
Number of Parameters that are not the default: 11 (3 are always required, 6 would be required for the rule template to do anything useful)
Number of Items: 18
Number of Items with metadata: 0
Total parameters defince 11.
If I had to create a separate rule for each Item given the non-defaults I’d have to set 198 parameters.
Those 16 parameters on the rule do have a nice user facing dialog.
I’ve not tagged him and I’ve been uncomfortable both ways (tagging him or not tagging him). I feel like it’s unfair to talk about him but at the same time I don’t want to rope him back into a discussion he clearly doesn’t want.
I just would like to make sure that there is no misinterpretation regarding what happened to the mentioned PR. @Spaceman_Spiff and me disagreed on the way YAML should look like. I prefer “closer to code” while @Spaceman_Spiff prefers “closer to a good model”. This is a conceptual disagreement, but has nothing to do with “bad blood” or something along these lines. I do appreciate his contributions and we had good and close collaboration in the past (e.g. regarding websockets).
Changing from DSL to YAML has no personal benefit for me. I don’t use textual configuration and most of you probably know that my personal opinion is that textual configuration should be completely removed. Therefore my suggestion/PR was purely “service” and the amount of time and effort I am willing to spend on something like that is limited. I had (and still have) the impression that the overall interest was not very high (counting by the number of participants in the PR) and if 50% don’t agree with my proposal it’s not worth the effort.
I was busy with other projects at the time and didn’t catch up with the PR until after the fact.
If it helps, I’m strongly in Rich’s camp on this one. I probably wouldn’t use the feature either but the possibilities it opens up are tremendous and I’m really excited to see the number of incredible features it would lead to eventually.
I’d just add, having the support for YAML by itself isn’t what I’m excited about. I’m probably not going to change over to them either.
But it’s the stuff YAML files would enable that gets me excited and it’s all stuff that would come later.
what you see in the UI is what you see in the files
import/export (move between file based and managed easily)
UI widgets/pages in files?
building “modules” that include Items, Things, UI Widget, transformations all in one file (makes it easier to share on the forum and elsewhere), maybe even supported on the marketplace
Multiple rule templates in one file
support for rule templates from files (no need to publish to the marketplace first)
making changes don’t necessarily lead to breaking changes with the versioning; upgrades become much easier for end users
no more FUD about file support going forward
easier to support changes to the file formats for the developers
YAML file support is a necessary first step for many (but not all) of these I think.
). Automation rules are not an end in itself - it it works why change?
and add another debounce rule for an item and edit the parameters directly in the UI.