So I finally upgraded our core system to OH4 from OH3. A few days ago. All seemed good - although I finally noticed one minor thing due to the units now getting normalized in OH4.
So the blank part 08/20 00:00 is when the servers were offline for disk imagining - prior to the backup - just in case snapshots failed, etc… no issue.
Then there’s a blip where it kicks in with OH3 before the OH4 upgrade is run.
Now there’s the fun bit. The VeSync binding I initially did porting of the protocol from pyvesync, originally mapped the units as what I would call pass-through. From what the binding specified through the entire system, so it was reflective of what the unit displayed. With the upgrade it changed the units to kg/m3, which effectively zeroed out the units and would have made the rules invalid I believe.
I have added another column to the VeSync binding Readme.md called units and specified the explicit units against the channel, where the normalized mappings would impact the binding.
I really hate suggesting anything that creates work for us developers, but my experiences makes me think for OH 4 all bindings may need a doc revisit, and a unit column added if they don’t have it.
I say this as I wouldn’t have a clue what to use, and luckily from the context of writing the binding could figure out what to google until I found the correct unit’s def. to use and test. Now at least users can see it in the documentation when that gets merged - I would have been stumped however as a end user, who didn’t know the units of the protocol. (Also its still a pig to find what is the correct unit string entry to use - I see other UI suggestions elsewhere to fix this ).
In the meantime it makes me think that anything that maps to a non standard unit, need’s this column and we unfortunately need to check for it on PRs as well?
Any thoughts - the above is hopefully just useful input based on my experience.(Maybe there is already a unit column in other bindings I’ve never seen?).
Did you apply unit metadata to these Items prior to or immediately after the upgrade to 4 if at all?
If not, did the Item’s State Description Pattern include units (not including %unit%)?
How was the upgrade performed?
The way it’s supposed to work is:
.items files per the release notes, all Number:X type Items should have unit metadata added. If not updating all, at a minimum those Items that have a State Description Pattern that shows the Item as a different unit from the default or from what the Channel provides must have unit metadata added.
Managed Items mostly get the unit metadata added automatically during the upgrade process. If the Item has a unit in the State Description Pattern, that is used for the unit. If it has %unit% or doesn’t have that metadata, the unit advertised by the Channel is used (I’m not sure how that is detected, maybe. If that doesn’t exist, the system default for that unit is used.
So the first step is to determine if for you something failed in either 1 (i.e. you didn’t manually add the metadata) or 2 (something failed during the upgrade or there is an edge case not handled).
Note, if the unit on the Item doesn’t match the unit provided by the Channel, the value will be converted to the unit on the Item before the Item’s state is updated, meaning the Item will always carry the user specified unit regardless of what the add-on delivers or what the state description pattern uses to display it.
I do think that add-on authors should always document what units a Channel delivers if it publishes values in units. If for no other reason, because users can now choose to link those Channels to just a Number (throwing away the units) in which case they’ll need to know what units the value is coming over as just for their knowledge on how to use it. But now with the unit metadata the user has a bit more control over the state that the Item actually stores, and therefore what units the value gets saved to persistence.
Okay so to try and explain the binding maps the value in question using this:
updateState(DEVICE_CHANNEL_AIRQUALITY_PM25,
new QuantityType<>(purifierStatus.result.result.airQualityValue, Units.MICROGRAM_PER_CUBICMETRE));
In channel-types.xml for that channel it has:
Number:Density
Air Quality PPM2.5
Indicator of current air quality
Now the item line was:
Number:Density LoungeAPAirQuality “Lounge Air Purifier Air Quality [%.0f%]” { channel=“vesync:airPurifier:vesyncServers:loungeAirFilter:airQuality” }
In 3.4 this showed and persisted the units as µg/m³
Upon upgrading to v4. The units became kg/m³. So the units no longer pass through what the binding specifies in code, but go “may be incorrectly named by me” a normalized base unit. Hence it gave the persisted data as shown in the original post.
Of course when it was µg/m³ changing to kg/m³ means rules using µg/m³ won’t work as they are now evaluating against a different persisted value.
The documentation I have updated to:
Number:Density LoungeAPAirQuality “Lounge Air Purifier Air Quality [%.0f%]” { unit=“µg/m³”,channel=“vesync:airPurifier:vesyncServers:loungeAirFilter:airQuality” }
and added a unit definition against the channel details.
As for the display / use of this parameter the units need to be specified to µg/m³ in order to match the scales shown by the unit.
Does that make sense?
I think your first point is the key bit:
.items files per the release notes, all Number:X type Items should have unit metadata added
However it used to pass-through and wasn’t in the binding doc’s until my new PR goes in. So as a user I would struggle to find the right “units” to set for the channel or any other ones, is my main gripe - not with the normalization that OpenHab does as such.Hence the doc’s should have unit’s specified otherwise it it can result in what looks like a fault, where actually the data is just re-based to a different unit. P.S It doesn’t help grafana / influx combo, appears to show the units from the latest persisted value, so its not immediately clear the units changed.
P.S at the end of the original graph on the first post (the values started coming in as unit has then been set as updated in the docs).
Right. So no unit is specified. Therefore the default unit for that unit of measurement type is what gets applied.
Previously, given no other information, the unit in the state update was applied so what came from the Channel was passed through. Now the value is always converted (if necessary) to the unit of the Item. When unit metadata isn’t defined that base unit is going to be the system wide default for that UoM.
Persistence saves the state of the Item without units. It just saves the number. One of the problems that arose with the old way that was fixed in 4.0 was the following scenario (among others). Let’s say you have a Channel that publishes °C. The Item’s State Description Pattern is set to show the Item as °F. In this scenario the Item’s state is °C so that’s what gets saved. However, on restoreOnStartup the only unit that is known at that point is the State Description Pattern so that 22 °C gets restored to the Item as 22 °F and until that Item gets updated from the Channel again, it’s going to be wildly off.
So yes it is important that binding authors specify the unit in the docs for their Channels. But there is a nuance there because even if the unit published is documented to be °C (for example), if my default units is set to imperial, my Item will be set to °F, given no other information (i.e. unit metadata is not defined).
Ultimately, I think the rule should be “always define the unit metadata to be what you want for the Item. Never rely on the defaults.” If you follow that rule, it doesn’t matter what units the Channel publishes. Either the unit will match or it will be converted automatically. In either case, the end user gets the units they want.
However, because it’s possible to link a Channel to a Number Item too, it’s important to document the units so that those users know what values they are receiving because no conversion will take place in that case. The units are just dropped.
All great points, and don’t disagree with any of it. As a user however I do want the binding usually setup or the ability to set it up, without researching units for the device, and you’ve said also the units should be documented, in which case this problem goes away a sensible unit “suggestion” is documented. I think that’s the key ask is can we ensure units are specified not as a mandatory suggestion but starting point, so users know how to get a reasonable starting representation of the data pulled in (where possible).
Yeah your right about the units I could have sworn I saw kg/m3 on the UI interface - but I can see its coded into the UI, so must have been mistaken.
Part of my point is it doesn’t matter what units the device reports. If it reports a density, the user can and should choose the density unit that makes sense to them. If none of the options make sense to them, well they need to research the units anyway so they can even interpret the value in the first place.
That doesn’t mean that binding authors should not document what the unit is or what a sensible unit should be. But ultimately, the user should choose what makes sense to them regardless of what the device reports. Or, if they don’t want units they have that choice as well.
Yeah completely agree, I think this is the point I was trying to make
That doesn’t mean that binding authors should not document what the unit is or what a sensible unit should be.
Your completely right it should be the users choice, and all of the logic to persist in that selected format makes perfect sense. Just figuring out the correct units if the modules doc’s don’t get that information, I can see making more noisy threads. Just trying to think if there’s any way to suggest improvements to the units page, to make life easier (I may need to read up more on the whole units system first however).
There really is no “correct” unit. There are a collection of valid units that each type supports. But any one of them is correct. If you, for example, have a Number:Temperature, °C, °F, and °K are all valid choices. Any one of those is “correct”.
What units the binding author choose isn’t the “correct” unit. It just happens to be one of the valid units to represent that value. One can find the the list of valid units for each Number Item type in the UoM docs. The binding docs definitely indicate what type of Item a Channel is expected to be linked to. And with that information one can look at the table on the UoM page to see what units are possible.
The only additional complication is that some units support prefixes while others don’t. For example you can have kWh but not k°F. The latter makes no sense. So a little bit about what is being measured needs to be understood by the end user.
New to this topic but I’m also a bit puzzled since OH4 and since I’m currently actively looking into an existing binding’s code I’m wondering if there is anything which could or should be done to make it “more usable”.
In this example there are channels providing a percentage number or a length unit:
updateChannel → maintenance#distanceToInspection = 28600 km
So as to what I understand the Thing binding is updating the channel with “28600 km”.
The item in the end shows “286000000 m” instead.
Now I know that I could define the unit metadata to “km” and it would be ok again (after OH restart).
But it means (as per situation) means a lot of work compared to just clicking to link all thing channels and for some cases it’s really quite obvious to use km instead of m.
Another example in the same thing is a percent value. Very rarely this should show up in the item as 0.x but people want to see x % instead.
So is there really no way to suggest or recommend OH (from a binding) to use a certain unit by default?
I don’t think a binding can/should suggest a unit in all cases, but the UI perhaps could.
As an example, the Tesla API provides distances in miles, and this is used to update relevant channels. What unit should be suggested depends on your configured region (for example km for Europe, miles for US).
In other cases you might have a point, but it quickly becomes complicated. For example, for a washing machine measuring water consumption, the unit m³ doesn’t make sense as default unit. Suggesting litres might be better, but I don’t know if gallons are preferred unit in the US. In this case, what should be suggested? The binding would need to provide a list of suited units and regional settings would then pick from this list?
Number:Dimensionless might be a more straight-forward example where the binding could probably suggest something, for example % for relative humidity.
A restart is not required. However, an Item update is. The unit on the Item won’t change until the Item’s state changes.
Shows where? In MainUI, the Model view, and I think in the sitemap, the Item will show what ever unit is defined on the State Description Pattern which is settable from the binding (with all the complications @laursen describes). The State Description Pattern controls how the Item is shown.
However, the state that the Item actually carries will be defined by the unit metadata or, if that isn’t defined, the system default.
There is a PR open to add the unit to every place where an Item can be created in MainUI. Upon creating the Item through the + on the Items page, the Model page, or by “add equipment to model” or “add points to model” there will be a field showing what unit the Item will use by default and allow the user to change it prior to the Item ever being created.
This greatly eases the burden on the users and eliminates surprises.
Sorry, no!.
What do yout think a car bindig shall deliver?
km / miles or meters / feet based on LocaleProvider?
state of charge in unit one?
tire pressure in bar / psi based on LocaelProvider? It’s not Hectopascal!
What’s your determination on selecting system default as meter / feet?
You took a bad example! Of course the API can deliver miles as default but it’s the duty of the binding delevoper to transform the values.
Same quote but the UI cannot provide the right values without correct unit usage. State pattern ok but oh-charts are not showing correct values - see discussion here
It shouldn’t matter at all in which unit it delivers some state. I think it should make as few internal conversions as possible, and just deliver whatever it has available. Then openHAB will take the responsibility of converting that into a target unit. So if an API provides temperature in °F, deliver that. If an API provides distance in miles, deliver that.
A separate concern is the item unit and system defaults units. As @rlkoshak mentioned, some UI improvements seem to be under development, so that unit will be defaulted in the UI and adjustable. That seems like a good idea, since the unit defined in the item metadata has most impact, especially if missing.
Leaving the unit metadata undefined has some drawbacks:
No control over persistence. If regional settings are changed, system default unit can change, and persistence will be corrupted.
For Number:Dimensionless there is no reasonable default.
In some cases it’s not enough knowing if we are using the metric or imperial system (as example). As in my washing machine example, both litres and m³ are metric, but litres makes most sense, therefore it would make sense to set this as item unit.
When unit is configured for the item, the state pattern %unit% will automatically pick up that unit and use as display unit. That also comes with a - perhaps theoretical - drawback: We have now locked the unit, so the regional settings will no longer have any impact. I don’t know if a multi-user openHAB system can be imagined where different users would have different unit “preferences”, but I think we can park that for now and the sake of this specific discussion.
And last, of course, the unit can again override the state pattern with a unit of preference.
So, in order of priority:
System default
Item unit.
Channel state pattern.
Item state pattern
The user can control all of them, except the channel state pattern when a custom unit is used rather than %unit%.
What came to my mind in my previous post was the possibility of the binding giving a unit “hint”. The Miele@home binding could hint that “l,gal” is preferred. When creating an item, the UI could then default the unit as follows:
If no hint is given, the system default is suggested. i.e. how is currently works.
If a hint is given, find a unit on the list that corresponds to system settings, e.g. “l” for metric/SI and “gal” for imperial - and suggest that.
The same could be done by providing “km,mi” - so that “km” would be defaulted in Europe rather than “m”.
I have no idea if this would hold, but I am pretty sure it will quickly be corrected if not. But even if this is not feasible, it will still be a massive improvement to have m³, gal and l suggested for Number:Volume items and being able to simply adjust it before creating the item in the UI.
I don’t think the binding should have to use LocaleProvider for determining a unit. This task would be redundant and should be left to openHAB based on user preferences.
But I do think it would make sense if the binding could hint that “%” should be the preferred unit for a Number:Dimensionless representing relative humidity, so that this could be defaulted in the UI when creating the item - rather than one.
I don’t understand this part. Do you mean that the binding should determine unit from LocaleProvider, convert internally to that unit, and update channel state with a QuantityType of that unit? This wouldn’t make any difference compared to just delivering the “native” unit, since openHAB will convert it anyway.
But if this is your suggestion, i.e. you think it should be up to the binding to overrule item unit and be responsible for conversions, then I don’t agree. That would introduce redundancy, complexity and take away control from the users.
Do you mean the user will still be able to control the unit metadata, so that logic only has affect when unit metadata is missing?
In that case, conceptually on a high level I don’t have a problem with that. But I think it would be better to be able to suggest the preferred units to the UI by means of binding metadata (e.g. as part of channel type definitions and channel definitions) rather than overriding anything in the binding, and having to deal with this directly in all bindings.
Something needs to be the default though. And currently the default is a simple ratio (i.e. ONE), which makes some sense in a wider UoM kind of perspective. But simple ratios are almost never used in a home automation context. It’s always going to be % or, to a much lesser extent dB.
It would make sense to me to default Number:Dimensionless to % in OH since that’s the most common use of that. But I don’t know if OH has control over that or if that’s controlled by the upstream library.
This sounds good! So binding can give a unit hint which automatically is accepted if the user doesn’t change it. I think my below remarks are solved if this works.
Is this an idea or is there already some source code available?
Sorry, no, it’s only an idea generated from this discussion we are having. Unless @rlkoshak or anyone else following this can see any problems with this proposal (or another approach that would be smarter), I could create a detailed RfC issue in the core repository for being further discussed amongst core maintainers/contributors/others.
Sorry, didn’t mean it like that. But if the idea could be quickly be dismissed for any reason, I would rather save my time for working it into an issue. So nothing personal, just probing for quick feedback in order to be lazy.
I can’t predict how this one would be received. I think it’s worth filing to see what happens. Just make it clear that it is not the bindings overriding the unit, just providing an alternative default.
There might be some push back as this will probably require some significant changes to some of the APIs, particularly if the add-on’s suggestion is to be reflected in MainUI when creating the Item (which it should). But beyond that is seems reasonable to me.
).
But even if this is not feasible, it will still be a massive improvement to have m³, gal and l suggested for