I wonder if you are confusing and overcomplicating things at this stage?
The binding should of course be allowed to update state with value plus whatever (appropriate) unit to match item type, but it cannot and must not be allowed to define the default unit. Updating a value with a unit is something different from making that the default unit for that item. Note you can have the default unit be lets say W for a Number:Energy item as enforced by the recent change in 3.4 and have the binding send 1kW. That would set the value to 1kW. Only if the binding did send “1” i.e. without unit, the default unit would apply and the state would become 1W.
Where does your assumption (and not only yours) originate from that the core converts anything when there’s a value+unit update and that unit mismatches the default unit ? OH doesn’t do that and AFAIK never did. (The exception being the imperial<->metric conversion if applicable.)
If you send 1.4mm to a Number:Length item that has a default unit of m, the value will become 1.4mm, plain and simple. If you send “1.4”, it’ll set the value to 1.4m (not mm) as the default unit will be applied.
I still don’t know what “physically right” means. Some would argue that here is no such thing as a “physically right” unit in the Imperial system at all, and yet all those units are equally able to measure quantities as any other units.
I’m cautious about this. Perhaps in cases where there isn’t a defaultUnit defined it makes some sense. But it should be overridable through the defaultUnit. Ultimately, the end user knows way more about their requirements than we will. We shouldn’t cut off options unnecessarily.
Me too but there is one use case where it makes sense: options for a selection list. Often an Item can only support a certain fixed set of options that the binding knows ahead of time. It’s really nice that it can push those to the Item because setting options can be a pain.
Depends on the context. The binding knows best what units the device is reporting. If it can’t know, the end user knows best. In all cases the end user knows best how they want to see it, use it, and store it in persistence.
This is only an issue for restoreOnStartup.
If the binding is supplying a state description with units, it darn sure better be updating the Item with those same units. Since the Item update units take precedence over all, that becomes the defacto default. However, restoreOnStartup doesn’t have that info so if there isn’t a defaultUnit set, it will have to convert the state to a default unit or else restoreOnStartup will apply the wrong units.
Users who want to avoid that conversion have the option to set a defaultUnit to specify how it should be saved and restored.
If there is a binding that is supplying the state description with units but not using a QuantityType to update the Item, that binding needs to be fixed.
Come on don’t start splitting hairs philosophically. In return, I won’t start discussing if this all is of any relevance at all as we all are just living in the matrix (I took the red pill).
Physics (or the conventions used in physics) define base units. That’s W, C, m and s and so on in the metric system or in imperial system W, F, ft and s. For multidimensional units the “physically right” base is the product of single-dimensioned ones i.e. Ws is “right”, Wh is not.
Now all I said is we should treat these base units as the prime candidates for our default units, but we don’t have to use them at all cost. It’s all conventions. So as Wh is more commonly used than J=Ws, we should use that as our default unit for Number:Energy.
Now for a break.
I think it’s overdue for an important note on scale prefixes as a lot of confusion is originating from that: Scale prefixes aren’t “units” !
To handle decimal prefixes(*) correctly, you should view them to be a part of the value and not of the unit. The unit of an item state 2kW is W, the value is 2000 !
(*) other scale factors, too: h and mi are “just” non-decimal prefixes for the (base) units s and ft.
Again: the internally stored value doesn’t matter. There is simply no need to convert anything because this can be done in rules and/or state descriptions for display purposes. Persistence is always done in a fixed unit, as long as you don’t change the configuration.
Well, let’s revisit the “original” problem as scaling/normalization is that golden lamb that we all still seem to be dancing around.
Before 3.4 default units added, I had an Number:Energy item set to “1kW” and persisted.
Now on updating OH, restoreOnStartup is called to retrieve the value from persistence and the now running 3.4 applies a new default W.
If the internal representation is unconvertedly copied back i.e. same what was persisted, the state now will be 1W as only the unit (or scale part of the unit, actually) is replaced, or ?
As we want the state after restore to be 1kW or 1000W like before, the has to be multiplied by 1000 at some stage, correct ? Does that currently happen ? when exactly ?
Or has OH already been treating scale prefixes as part of the value, not the unit ?
If so, if I have a default unit W and send 1kW there, on assignment, it’ll have converted the 1 to 1000 and stored that as the internal value. It knew and still knows W is the implicit base unit to apply to the internally stored value. But in addition, it keeps the information that the value is to be displayed as kW by default (if no conversion is requested). So that info is merely a default display pattern than “the” unit.
BUT - and that’s what makes the difference when you consider the scale prefix to be a part of the value, not the unit - it does not mean the state is 1000kW.
Does anyone know ? Is anybody of you able to try it out and confirm or falsify ?
That’s why storage unit and display unit should be separated from each other. You might be storing values using kWh, but once reaching high enough value you might want to swap display to MWh with more decimal points. Doing it so with OH < 4.0 makes all persisted states a waste or manual migration.
I’m not splitting hairs. I’m trying to understand. Because in Imperial units, which usually don’t have nice scalar modifiers like SI units do (there’s no such thing as a kiloyard for example), the “physically correct” unit depends on what you are measuring. If you are measuring the area of a house, you’d use square feet. The size of a parcel of land is measured in acres, or if it’s a big farm/ranch hectares, or for really big areas like the sizes of cities in square miles.
If measuing the rate of rainfall you’d use inches per hour but the speed of a car would be miles per hour. You’d never measure the speed of a car in inches per second nor would you ever measure rainfall in miles per hour.
There is no base unit that is the always correct one. It’s always situational dependent. And conversions between them are not always even and straight forward. For example, a hectare is 2.47105381 acres or 107639.1 square feet.
This is where I’m coming from. A world where the correct default depends on the context of what you are measuring. For power and other types of units where the average person is less likely to encounter them maybe it’s a different story. But for the common ones like length, speed, volume, etc. there isn’t a correct default. It depends on what it is you are measuring.
Obviously we have to pick something but that’s Something has to be selected as a default when no other information is available, but the user needs to be able to override it both in how it’s displayed and how it’s persisted.
It doesn’t.
I think what we are dancing around is an edge case.
If the binding provides the unit in the update to the Item we are good.
If the binding doesn’t provide the unit in the update to the Item, here we are good too.
However, what if the binding provides the unit in the update but that unit is different from the system default for that Item (let’s make it easy and say it’s km where the default is m) and there is no defaultUnit configured nor a state description configured?
What is going to be saved to persistence? What’s going to be restored on startup?
We want to avoid, if we can, that we save km but restore it with m as the unit (unless it’s properly scaled).
I think all the other cases are well handled and to me at least it is clear where the units come from and where they go. But this one I think remains unclear what would happen and what should happen.
Only in the case where you don’t have a State Description defined. Otherwise, right now in 3.4, the unit in the State Description is used.
But your example is the same as what I am concerned about too.
Not shown here is the impact of defaultUnit, right? With defaultUnit set OH 4 would persist and restore assuming that unit, not the system default, right?
Correct. It works the same as the unit in the state description now. The difference is that currently persistence, representation and added unit (added when a decimal type is passed to the item) are coupled. With the defaultUnit the representation (state-description) is de-coupled from persistence/added unit (which both use defaultUnit).
Which is more or less what I meant. We have a set of defaults now to match what’s used in the majority of use cases in physics, but we can still selectively replace them if we identify unpopular choices (such as J → should be turned into Wh).
I think it does. It’s not as intuitive as in SI as the naming scheme doesn’t include the base unit’s name and scale factors are not 10^n, but it’s essentially the same. A yard is 3 feet so you could write “3ft” instead of “yd” with “3” being the scale prefix and ft being the base unit.
Thanks for that overview. Is that theory or did you really validate that in some way?
If I’m getting it right, we will not have a problem with persisted values in rows 1&2, only in 3
(btw someone already found that: After Update to OH3.4 one issue in ruleDSL - #14 by J-N-K).
4 is when sticking with <= 3.3 so we need not worry about and the user can fix it himself.
Good you highlighted the missing unit in 3.3 but 3.3 is nothing we need to take action on.
We should document that table in some central docs location and advise users to a) set state description today (preferrably before an upgrade to 3.4 so it should be part of breaking change docs) for any items they care about and b) (just proposal) to replace %unit% by the unit they want to use/see displayed.
For clarity, OH4 means after merging Jan’s outstanding PR#3248 to implement defaultUnit metadata, correct ?
Please note that this is only relevant for a small subset of items: Only Number items that are linked to channels that send a value WITH unit, WITHOUT a default unit in pre-3.4 openHAB and WITHOUT a state description that defines a unit.
A default unit was already present for
temperature (°C / °F)
pressure (Pa / inHg)
speed (km/h / mph)
length (m / in)
intensity (W/m2, in both system)
dimensionless (ONE, in both systems)
angle (°, in both systems)
I believe that greatly reduces the number of items you need to check. You can type Number: in the search field of the item’s page and see which need adjustment.
Edit: If you export all your items using curl -X 'GET' 'http://localhost:8080/rest/items?metadata=statedescription&recursive=false' -H 'accept: application/json' > items.json you can use
const fs = require('fs');
fs.readFile('/Users/jan/Documents/items.json', 'utf8', (err, data) => {
if (err) {
console.error(err);
return;
}
const items = JSON.parse(data);
let itemCount = 0;
items.forEach(item => {
if (item.type.startsWith('Number:')) {
if (item.stateDescription == undefined) {
console.log(item.type + ' ' + item.name + ' has no state description at all');
itemCount++;
} else if (item.stateDescription.pattern.includes('%unit%')) {
console.log(item.type + ' ' + item.name + ' has state description with unit placeholder');
itemCount++;
} else if (item.metadata == undefined || item.metadata.stateDescription == undefined) {
console.log(item.type + ' ' + item.name + ' has a state description not set in metadata');
itemCount++;
}
}
});
console.log('Total number of items that need your attention: ' + itemCount);
});
and it prints a list of all items that MIGHT need attention. It does not filter the ones that had a default unit before and also prints those that have a state description set by a binding (which is perfectly fine). You should pay attention to those missing a state description or have a state description with %unit% in it.
Regarding the “normalization”, I see the point of @Spaceman_Spiff. I have no strong opinion on that, as I already said: it doesn’t make a difference in what unit values are stored inside the item, representation is done by the state description. However, this will not solve the issue unless Add ItemStateUpdatedEvent and enable group channel-links by J-N-K · Pull Request #3141 · openhab/openhab-core · GitHub is merged, because currently the ItemStateEvent carries the unit that is sent by the binding/rule, not the one that is stored in the item.
Yes - and that is the correct unit which it should carry. This event should not be normalized.
The ItemStateUpdateEvent would then carry the state with the normalized unit and scale.
If you interact with external systems you have to normalize at some point.
So why not make it consistent and use the normalized value throughout openHAB.
That way persistence, API, Logs etc. is all in sync and it’s clear what is going on.
And as you said:
The internal storage of the value doesn’t matter because the datatype is precise enough
Graphical representation in the GUI is detached from the internal item state unit and scale
I’ve created the issue because I though it’s an elegant way to make UoM a really powerful and useful feature. It was more an “it would be great if it is that way in the long term” issue than an “this is an idea for a quick fix” issue.
You mean the graphical representation e.g. in the GUI (just to be sure - I think there was enough confusion in the last couple of days )
In short, is use the API explorer. I’ve you full out the door for one item, you’ll just need to change the Item name and the metadata to apply and hit submit again.
But like @J-N-K says, only a fraction of your 400 items will need to be adjusted.
)
)