Persisting percentages

I (again) scratched my head for way too long… I have a reading for relative humidity indoors and like all other items it’s being persisted. All good and dandy until…
I wanted to graph the change in humidity. So I naturally created a rule to “deltaSince”. But I don’t get the correct value. The humidity is currently at 26% and has been since the 5 minutes and looking in the past. So I would expect delta to be 0. But it’s 25.74!
So I dug deeper. If I “query” the current state of the item, it’s “26 %” But if I want the historic sate, it’s “0.26”. The delta between these is 25.74.
Running OpenHAB 3.0.1 and the measurement is coming from the ValloxMV binding. It’s of type: Number:Dimensionless

That is correct, in a way. 0.26 with no units is a ratio or factor for a Number:Dimensionless type. It’s mathematically the same as 26% or -11.7dB

However I do not think this is how it’s supposed to work - persistence should be storing your Item’s numerical value without units. It can’t store units. When you later read back records, it will assume the data is in the same units as the linked Item is now.
Or …is the assumed unit the Item’s default unit? That is different. Number:Dimensionless default unit is “blank” i.e. ratio.
Could you look at your Item in API explorer and see if you have “pattern” set, for an Item default?

While you’re in that, see what values are actually persisted too?

I think we ought to see the way you are using deltaSince() in your rule as well, although I doubt there is an error there. If persistence made the correct assumptions about storing and retrieving units,it should work.

I appreciate the very fast reply!
Here’s first what’s persisted, which is in line with what historicState reports:

{
  "name": "ValloxVentilationUnit_Humidity",
  "datapoints": "79",
  "data": [
    {
      "time": 1616504400000,
      "state": "0.26"
    },
    {
      "time": 1616504460000,
      "state": "0.26"
    },
    {
      "time": 1616504520000,
      "state": "0.26"
    },
    {
      "time": 1616504580000,
      "state": "0.26"
    },
    {
      "time": 1616504640000,
      "state": "0.26"
    },
    {
      "time": 1616504700000,
      "state": "0.26"
    },
    {
      "time": 1616504760000,
      "state": "0.26"
    },
    {
      "time": 1616504820000,
      "state": "0.26"
    },
    {
      "time": 1616504880000,
      "state": "0.26"
    },
    {
      "time": 1616504940000,
      "state": "0.26"
    },
    {
      "time": 1616505000000,
      "state": "0.26"
    },
    {
      "time": 1616505060000,
      "state": "0.26"
    },
    {
      "time": 1616505120000,
      "state": "0.26"
    },
    {
      "time": 1616505180000,
      "state": "0.26"
    },
    {
      "time": 1616505240000,
      "state": "0.26"
    },
    {
      "time": 1616505300000,
      "state": "0.26"
    },
    {
      "time": 1616505360000,
      "state": "0.26"
    },
    {
      "time": 1616505420000,
      "state": "0.26"
    },
    {
      "time": 1616505480000,
      "state": "0.26"
    },
    {
      "time": 1616505540000,
      "state": "0.26"
    },
    {
      "time": 1616505600000,
      "state": "0.26"
    },
    {
      "time": 1616505660000,
      "state": "0.26"
    },
    {
      "time": 1616505720000,
      "state": "0.26"
    },
    {
      "time": 1616505780000,
      "state": "0.26"
    },
    {
      "time": 1616505840000,
      "state": "0.26"
    },
    {
      "time": 1616505900000,
      "state": "0.26"
    },
    {
      "time": 1616505960000,
      "state": "0.26"
    },
    {
      "time": 1616506020000,
      "state": "0.26"
    },
    {
      "time": 1616506080000,
      "state": "0.26"
    },
    {
      "time": 1616506140000,
      "state": "0.26"
    },
    {
      "time": 1616506200000,
      "state": "0.26"
    },
    {
      "time": 1616506260000,
      "state": "0.26"
    },
    {
      "time": 1616506320000,
      "state": "0.26"
    },
    {
      "time": 1616506380000,
      "state": "0.26"
    },
    {
      "time": 1616506440000,
      "state": "0.26"
    },
    {
      "time": 1616506500000,
      "state": "0.26"
    },
    {
      "time": 1616506560000,
      "state": "0.26"
    },
    {
      "time": 1616506620000,
      "state": "0.26"
    },
    {
      "time": 1616506680000,
      "state": "0.26"
    },
    {
      "time": 1616506740000,
      "state": "0.26"
    },
    {
      "time": 1616506800000,
      "state": "0.26"
    },
    {
      "time": 1616506860000,
      "state": "0.25"
    },
    {
      "time": 1616506920000,
      "state": "0.25"
    },
    {
      "time": 1616506980000,
      "state": "0.25"
    },
    {
      "time": 1616507040000,
      "state": "0.25"
    },
    {
      "time": 1616507100000,
      "state": "0.25"
    },
    {
      "time": 1616507160000,
      "state": "0.25"
    },
    {
      "time": 1616507220000,
      "state": "0.25"
    },
    {
      "time": 1616507280000,
      "state": "0.25"
    },
    {
      "time": 1616507340000,
      "state": "0.25"
    },
    {
      "time": 1616507400000,
      "state": "0.25"
    },
    {
      "time": 1616507460000,
      "state": "0.25"
    },
    {
      "time": 1616507520000,
      "state": "0.25"
    },
    {
      "time": 1616507580000,
      "state": "0.25"
    },
    {
      "time": 1616507640000,
      "state": "0.25"
    },
    {
      "time": 1616507700000,
      "state": "0.25"
    },
    {
      "time": 1616507760000,
      "state": "0.25"
    },
    {
      "time": 1616507820000,
      "state": "0.25"
    },
    {
      "time": 1616507880000,
      "state": "0.25"
    },
    {
      "time": 1616507940000,
      "state": "0.25"
    },
    {
      "time": 1616508000000,
      "state": "0.25"
    },
    {
      "time": 1616508060000,
      "state": "0.25"
    },
    {
      "time": 1616508120000,
      "state": "0.25"
    },
    {
      "time": 1616508180000,
      "state": "0.25"
    },
    {
      "time": 1616508240000,
      "state": "0.25"
    },
    {
      "time": 1616508300000,
      "state": "0.25"
    },
    {
      "time": 1616508360000,
      "state": "0.25"
    },
    {
      "time": 1616508420000,
      "state": "0.25"
    },
    {
      "time": 1616508480000,
      "state": "0.25"
    },
    {
      "time": 1616508540000,
      "state": "0.25"
    },
    {
      "time": 1616508600000,
      "state": "0.25"
    },
    {
      "time": 1616508660000,
      "state": "0.25"
    },
    {
      "time": 1616508720000,
      "state": "0.25"
    },
    {
      "time": 1616508780000,
      "state": "0.25"
    },
    {
      "time": 1616508840000,
      "state": "0.25"
    },
    {
      "time": 1616508900000,
      "state": "0.25"
    },
    {
      "time": 1616508960000,
      "state": "0.25"
    },
    {
      "time": 1616509020000,
      "state": "0.25"
    },
    {
      "time": 1616509080000,
      "state": "0.25"
    }
  ]
}

I haven’t set any metadata for the item, but the binding might? This is what the API explorer says about the item:

{
  "link": "http://192.168.86.64:8080/rest/items/ValloxVentilationUnit_Humidity",
  "state": "25 %",
  "stateDescription": {
    "minimum": 0,
    "maximum": 100,
    "pattern": "%d %unit%",
    "readOnly": true,
    "options": []
  },
  "editable": true,
  "type": "Number:Dimensionless",
  "name": "ValloxVentilationUnit_Humidity",
  "label": "Sisäilman kosteus",
  "category": "humidity",
  "tags": [
    "Measurement",
    "Humidity"
  ],
  "groupNames": [
    "gHouse",
    "ValloxVentilationUnit"
  ]
}

In the rule I have the the following:

var DtRH = ValloxVentilationUnit_Humidity.deltaSince(now.minusMinutes(5))

Im doing the same for CO2 (from the same binding) and there the values make sence. But again I do have a stete description for that (it’s also Number:Dimensionless.
I’ll experiment with adding a state description for humidity too and see what happens.

This is a workaround, not a proper fix, but yes try setting that to
"pattern": "%d %%",
I’m not sure what impact this will have on pre-existing data, probably get 0.26% for old records.

This would indeed appear to have solved the issue. I now get 0.00 as delta as expected.
Hopefully this is 1) useful for someone else and 2) fixed at some stage :slight_smile:

It won’t unless we do something.

Would you confirm that your binding has been updating this Item with a % value all along? i.e. you have changes in events.log to “26 %” etc.

Would you also confirm your older data remains as 0.26?

I suspect the real issue here is Dimensionless quantities oddball behaviour in accepting no-unit as valid ‘ratio’ unit.
Not all Quantity types have a system default unit, that’s really for metric/imperial selection and doesn’t apply here.
But with persisting Dimensionless, no-default is getting used as ratio-default.

That is very true! The only thing my limited abilites can do, is provide answers for your questions.

Yes. Here’s an snipped from before I changed the presentation:

16:57:37.556 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ValloxVentilationUnit_Humidity' changed from 31 % to 30 %

This is after the change:

19:02:45.413 [INFO ] [openhab.event.ItemStateChangedEvent  ] - Item 'ValloxVentilationUnit_Humidity' changed from 41 % to 40 %

API Explorer also still displays it as 39 %

Querying the persistence data gives (and this must be right when the change in pattern was made):

    {
      "time": 1616511600000,
      "state": "0.3"
    },
    {
      "time": 1616511660000,
      "state": "29.505"
    },

Once again, thanks for all the help and if there’s any more info that I can help with, I’ll gladly do so!

1 Like

Thankyou for the extra detail, in particular that confirms persistence is applying an (unwanted) conversion on write.

Hi,

May I ask which persistence service you are using?

I remember I have seen this too when implementing support of QuantityType for JDBC persistence. Because we cannot store the unit in the database (it is not supported by the framework) most of the implementations use a conversation to the base unit given by the Item when storing the state. For Dimensionless this is unit ONE which leads to unwanted side effects in conversion. Not only for percentage, for ppm and others too. The solution in my PR was to exclude all values from conversion if the unit equals to ONE. We maybe can fix it in the persistence service you are using - if it is not JDBC.

I’m using the default OH3 persistence. Indeed valuable information I neglected to include in the first post.

This would be fine if it also converted “on the way back” when reading. Dimensionless has this weird confusion between units ONE and ‘none’, and we get this convert on write but not on read for the same Item/system environment.

That is not necessary. The persistence service assumes the value in the database has the same unit as the Item.

Trouble is, “same unit as the Item” is ambiguous.
In this case the Item current state has a % unit, the Item ‘pattern’ has no unit, as so far as I can make out the Item type of Dimensionless has either no base unit or the base unit ONE.

For those who can read code – the item’s unit reduced with the following logic:

State description is the primary source for this information. If I read the code correctly: if that is not available, openHAB falls back to default unit of the system for that given quantity type (Dimensionless).

That would be in line with my observations.

yep indeed. I was recently introducing the unit support for AWS DynamoDB Persistence and researched how this is handled by other persistence adding. Those that do support QuantityTypes, have this same weakness – changes in “item’s unit” (per the definition quoted above) mess up historical data queries. It’s an compromise…