JSR223 Python: Working with QuantityTypes, how do I get an int/float?

Setup, Configuration and Use Scripts & Rules
, ,
1

The Helper Library docs show just about every way one would need to work with QuantityTypes except for one. There will be cases where a user needs to convert a QuantityType to a plain old Python primitive int/float. In Rules DSL I’d use: (SomeItem.state as QuantityType<Number>).floatValue. I can’t figure out how to do something equivalent in Jython.

In my particular case, I’m trying to create a hysteresis library function.

def hysteresis(target, value, low=0, high=0):
    rval = 0
    if   value < (target - low):  rval = (-1)
    elif value > (target + high): rval = 1
    return rval

If target and value are a QuantityType, there is no way I can know what QuantityType it is without iterating through all the possible combinations. Without knowing the QuantityType, I can’t initialize low and high to 0 and without something in those values I don’t know that I can even perform a hysteresis check at all.

Obviously, a work around will be to force the caller to pass in both a low and a high and all four values will have to be of the same type. But it seems like I ought to be able to get a plain old primitive out of a QuantityType.

That won’t work because QuantityTypes don’t work with the regular +, - operators. I’d have to have two implementations, one that calls target.sub(low) and taget.add(high) and the other that calls -, and + depending on the type of the arguments.

Ideas?

2

In case you haven’t found these, the builtin min and max functions may be helpful for this.

Here’s an example (the previous and current values are QuantityTypes)…

min(lux_previous.intValue(), lux_current.intValue()) <= low_lux_trigger <= max(lux_previous.intValue(), lux_current.intValue())
3

I tried to call intValue() on the QuantityTypes and I got a non-such symbol (or whatever) error. Hmmmm. I’m not sure what I did before but now it works. :expressionless: Maybe I skipped the parens? Who knows. I didn’t check in the failures.

I’m not sure the min and max does exactly what I want to do here, but it’s useful to know they exist. I want to know if the current is lower, between, or above two values. The two values are defined by the current value and an offset. I don’t really care about the actual state.

For example, I want to know if my humidity is below 30%, above 32% or between 32%. If it’s below I turn on the humidifier, above I turn off the humidifier, between. I want the function to tell me whether to turn on, off, or do nothing.

My current version which appears to work is:

def hysteresis(target, value, low=0, high=0):
    rval = 0
    if   value < (target - low):  rval = (-1)
    elif value > (target + high): rval = 1
    return rval

Example usage:

if hysteresis(100, 89, low=10) < 0: log.info("Test1: success!")
else: log.info("Test1: failure!")

hyst = hysteresis(items["vMBR_Target_Humidity"].floatValue(), items["vMBR_Humidity"].floatValue(), low=2)
if not hyst : sendCommandCheckFirst("aMBR_Humidifier", "ON" if hyst > 0 else "OFF")
4

Looks good… min and max wouldn’t save anything. Maybe allow QuantityType args?

def hysteresis(target, value, low=0, high=0):
    if type(value) == QuantityType:
        value = value.floatValue()
    if type(target) == QuantityType:
        target = target.floatValue()
    if value < target - low:
        return -1
    elif value > target + high:
        return 1
    else:
        return 0
5

That’s a good way to do it. I was debating the best way to handle and had something like:

tgt_low = target.sub(low) if isinstance(target, QuantityType) else target - low

but wasn’t really happy with it because I could foresee a bunch of up front checking I didn’t want to sign up for right now. But doing it like that I can support a quantity type for any of the arguments (should probably do the same for low and high, I could see someone trying to pass those as a quantity type for some reason) without requiring them to all be QuantityType. I added in DecimalType for good measure.

def hysteresis(target, value, low=0, high=0):
    if type(target) == QuantityType or type(target) == DecimalType: target = target.floatValue()
    if type(value)  == QuantityType or type(value)  == DecimalType: value = value.floatValue()
    if type(low)    == QuantityType or type(low)    == DecimalType: low = low.floatValue()
    if type(high)   == QuantityType or type(high)   == DecimalType: high = high.floatValue()
    rval = 0
    if   value < (target - low):  rval = (-1)
    elif value > (target + high): rval = 1
    return rval

NOTE, only minimally tested at this point.

Edit: Is this worth submitting to the Community libs? It’s kind of small but when you tell a user “hysteresis” you can almost hear the ??? As part of my learning process I’m experimenting with creating generic building blocks that users can reuse. I’ve a list of DPs and tutorials I want to work on, I already did Time of Day and Gatekeeper, this was the next one on the list.

6

Could shorten those checks with

isinstance(target, (QuantityType, DecimalType))
7

Hmmm… isinstance can take a tuple…

def hysteresis(target, value, low=0, high=0):
    if isinstance(target, (QuantityType, DecimalType)):
        target = target.floatValue()
    if isinstance(value, (QuantityType, DecimalType)):
        value = value.floatValue()
    if isinstance(low, (QuantityType, DecimalType)):
        low = low.floatValue()
    if isinstance(high, (QuantityType, DecimalType)):
        high = high.floatValue()
    rval = 0
    if   value < (target - low):
        rval = (-1)
    elif value > (target + high):
        rval = 1
    return rval

For readability, I wouldn’t put your conditionals on one line like that.

8

I went to check the temperature control script for my fermenter and I think it would be better to not include the low and high values in the allowed range. For example, you might want to allow for lower values, but not allow any value higher than the target. Using <= and >= resolves this.

def hysteresis(target, value, low=0, high=0):
    if isinstance(target, (QuantityType, DecimalType)):
        target = target.floatValue()
    if isinstance(value, (QuantityType, DecimalType)):
        value = value.floatValue()
    if isinstance(low, (QuantityType, DecimalType)):
        low = low.floatValue()
    if isinstance(high, (QuantityType, DecimalType)):
        high = high.floatValue()
    rval = 0
    if   value <= (target - low):
        rval = (-1)
    elif value >= (target + high):
        rval = 1
    return rval

try:
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"5 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == 0
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"4 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == -1
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"6 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == 1

    assert hysteresis(5, 5, 1, 1) == 0
    assert hysteresis(5, 4, 1, 1) == -1
    assert hysteresis(5, 6, 1, 1) == 1
except AssertionError:
    import traceback
    log.error("Exception: {}".format(traceback.format_exc()))
else:
    log.warn("Tests passed!")
9

I wasn’t aware that isinstance takes a tuple.

I find it harder to read when there are a series of one line if conditions like that. But I guess that’s personal preference. Though this did have me look up the Python style guide (I’ve been meaning to look into that) and I guess it does violate the “no lone longer than 79 characters” recommendation. I’ve always hated that restriction in a style guide, but if that’s the way python says I need to do it…

Would it be possible to put all the arguments into a tuple and loop through them akin to:

for arg in (target, value, low, high):
    if isinstance(arg, (QuantityType, DecimalType, PercentType)): arg = arg.floatValue()

and make this work? (Note I added PercentType). I hate duplicated code and it feels like I should be able to handle this a little more cleanly. All I can think of is to create an empty list and append each value to that, but I’m not sure that really does much for me.

def hysteresis(target, value, low=0, high=0):

    args = []
    for arg in (target, value, low, high):
        if isinstance(arg, (QuantityType, DecimalType, PercentType)): 
            args.append(arg.floatValue())
        else:
            args.append(arg)

    rval = 0
    if   args[1] < (args[0] - args[2]):  rval = (-1)
    elif args[1] > (args[0] + args[3]):  rval = 1
    return rval

I lose that block of repeated code but I end up obfuscating the part of the function that does the actual work.

I initially implemented it that way but it ended up with an ambiguity in the edge case where low == 0 and high == 0. What is correct in that case? Both -1, 1, and 0 are correct.

Maybe the better way would be to return None or throw an except in that case because if both are zero, one is probably calling the function erroneously in the first place. Is there a Python or Helper Library preference for which is better, None or except?

Or maybe I return 0 in that case… That would probably be the lest disruptive and not necessarily incorrect.

10

I’m on my phone so I can’t write the code, but what about a get_float function that does the type checking and conversion. Then just run all 4 values through it before the comparison. Easier to read than the arg index, but that works too.

I think returning 0 would make the most sense. Wrap the other checks in : if high != 0 and low != 0:

11

I agree, it is easier to read.

Current version looks like this:

def get_float(value):
    if isinstance(value, (QuantityType, DecimalType, PercentType)):
        value = value.floatValue()
    return value

@log_traceback
def hysteresis(target, value, low=0, high=0):

    target = get_float(target)
    value  = get_float(value)
    low    = get_float(low)
    high   = get_float(high)

    rval = 0
    if low == 0 and high == 0:     rval = 0
    elif value <= (target - low):  rval = (-1)
    elif value >= (target + high): rval = 1
    
    return rval
1 Like
12

And just because it popped into my head. Say that get_float function isn’t used anywhere else, you can actually put it inside the hysterisis function. This keeps the module namespace cleaner.

13

Start with PEP20 before going to PEP8. In Python, you’ll find a lot of references to readability. IMO, cramming the conditionals on one line is much less readable than putting them on separate lines. IIRC, there’s something in PEP8 about this. Definitely personal preference… until you share your code :wink:. In general, it’s best to spell things out in Python rather than to make it as tight and concise as possible.

How about…

    target, value, low, high = map(lambda arg: arg.floatValue() if isinstance(arg, (QuantityType, DecimalType, PercentType)) else arg, [target, value, low, high])

But is that more readable?

You’re right… I didn’t think of that case. How about…

def hysteresis(target, value, low=0, high=0):
    target, value, low, high = map(lambda arg: arg.floatValue() if isinstance(arg, (QuantityType, DecimalType, PercentType)) else arg, [target, value, low, high])
    if value == target or target - low < value < target + high:
        rval = 0
    elif value >= target + high:
        rval = 1
    else:
        rval = -1
    return rval

try:
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"5 °F"), QuantityType(u"0 °F"), QuantityType(u"0 °F")) == 0
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"6 °F"), QuantityType(u"0 °F"), QuantityType(u"0 °F")) == 1
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"5 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == 0
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"4 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == -1
    assert hysteresis(QuantityType(u"5 °F"), QuantityType(u"6 °F"), QuantityType(u"1 °F"), QuantityType(u"1 °F")) == 1

    assert hysteresis(5, 5, 1, 1) == 0
    assert hysteresis(5, 4, 1, 1) == -1
    assert hysteresis(5, 6, 1, 1) == 1
except AssertionError:
    import traceback
    log.error("Exception: {}".format(traceback.format_exc()))
else:
    log.warn("Tests passed!")

I don’t see how this could be generalized and would need to be decided case by case. IMO for this, an exception would be better than None, but returning 0 is best. Without specifying low or high, the function is checking for equality.

14

I thought of that, but this feels like a function that will have other uses. Any time one has, for example, Items with two different numerical types that they need to compare or do math with they will need to convert it to a primitive. On the other hand, doing so in those cases will also be one liners so I’m not sure how much the function would save users in the long run.

That would be along the lines I was thinking, but no, it’s not more readable. Honestly, I find the way that those sorts of collection operations are done in Python to be challenging to read, especially when you want to string a bunch of stream operators like filter, map, reduce together. It’s like spending your life doing math using infix notation and then given a reverse polish notation calculator to take the test. I need to learn the alternative approach Michael pointed out in another post, though that too, at first look, has an arcane feel akin to regular expressions.

Anyway, here’s the latest along with the tests which are all passing.

@log_traceback
def hysteresis(target, value, low=0, high=0):
    def get_float(value):
        if isinstance(value, (QuantityType, DecimalType, PercentType)):
            value = value.floatValue()
        return value

    target = get_float(target)
    value  = get_float(value)
    low    = get_float(low)
    high   = get_float(high)

    if value == target or target - low < value < target + high: rval = 0
    elif value <= (target - low): rval = (-1)
    else: rval = 1
    
    return rval

try:
    assert hysteresis(30, 30, 1, 1) == 0
    assert hysteresis(30, 29, 1, 1) == -1
    assert hysteresis(30, 31, 1, 1) == 1
    assert hysteresis(30, 30) == 0
    assert hysteresis(30, 31) == 1
    assert hysteresis(30, 29) == -1
    assert hysteresis(QuantityType(u"30 %"), QuantityType(u"29 %"), low=QuantityType(u"1 %")) == -1
    assert hysteresis(QuantityType(u"30 %"), QuantityType(u"29 %"), 1, 1) == -1
    assert hysteresis(QuantityType(u"30 %"), QuantityType(u"31 %"), high=QuantityType(u"1 %")) == 1
    assert hysteresis(DecimalType(30), DecimalType(29), low=DecimalType(1)) == -1
    assert hysteresis(DecimalType(30), DecimalType(29), 1, 1) == -1
    assert hysteresis(DecimalType(30), DecimalType(31), high=DecimalType(1)) == 1
    assert hysteresis(PercentType(30), PercentType(29), low=PercentType(1)) == -1
    assert hysteresis(PercentType(30), PercentType(29), 1, 1) == -1
    assert hysteresis(PercentType(30), PercentType(31), high=PercentType(1)) == 1
    assert hysteresis(QuantityType(u"30 %"), DecimalType(29), PercentType(1), 1) == -1

except AssertionError:
    import traceback
    log.error("Exception: {}".format(traceback.format_exc()))
else:
    log.info("hysteresis tests passed!")

Is this something worth submitting to the Helper Library? Maybe in community? I don’t think it necessarily belongs in core. I’m cool either way. I have multiple places where I do this in my code so it was well worth implementing just in my setup.

Thanks for all the help!

15

Oh my … the ghost of the punched card lingers on.

16

I agree! It just occurred to me that it might not be obvious that it was possible to nest functions like that.

Which approach was that?

Perhaps an addition to core.util?

17

There are legitimate reasons why you would want to do that. The justification given in the style guide is so you can display and edit two files at the same time side by side. It also forces you to think about what you are doing. I think the Python folks are happier to more but simpler lines of code rather than fewer lines which definitely gets reinforced by the 79 characters per line (I just can’t do it, I’m using 80, don’t tell anyone :wink: ). I even set up the line in VSCode to tell me where the end is.

I don’t have to like it, but I see the benefit.

I wish I could remember the name you used. Something like “list comprehension.” Instead of map and filter calls, there were brackets and array like operations. I didn’t stop to study it and now I can’t find it. :frowning:

18

Yes exactly. You can also do dictionary comprehension.

list_comp = [item for item in other_list if condition]
dict_comp = {key:other_dict[key] for key in other_dict} # this is much faster than itering over k,v pairs

Is the same as:

list_iter = []
for item in other_list:
    if condition:
        list_iter.append(item)

The conditions are optional and are used to filter only for desired elements.

1 Like
19

The map() and filter() functions used with lambdas, and even more so list comprehensions, are inherently complex, but they shouldn’t be avoided because of their complexity. They do get easier to comprehend with familiarity. Here are three ways to do the variable unpacking…

list comprehension…

target, value, low, high = [x.floatValue() if isinstance(x, (QuantityType, DecimalType, PercentType)) else x for x in [target, value, low, high]]

map() with a lambda…

target, value, low, high = map(lambda arg: arg.floatValue() if isinstance(arg, (QuantityType, DecimalType, PercentType)) else arg, [target, value, low, high])

map with a function (but this needs to be read along with the get_float function)…

    target, value, low, high = map(get_float, [target, value, low, high])

The hysteresis function is something that I would approve in core.utils. There are a couple things to cleanup, but we can address them in the PR. I’m hesitant to put anything into core that couldn’t be replicated in a scripting API. I can’t think of any utility functions for automation in OH, but I hope to change that.

In the Jython helper libraries, I’ve used 79 chars as a recommendation only and have only enforced the limitation in docstrings. Also, the first char is at column 1, so the 79th char is column 80. :slightly_smiling_face:

20

I can move it. I’ve currently submitted it as a community library because it’s something that is more useful in general and doesn’t really have anything to do with supporting the interface between Python and openHAB itself. As I looked at the functions, it looked like hysteresis was an apple and core.utils was all oranges.

But I do see it as kind of the next layer up in building blocks for rules developers. The Helper Libraries are about making Python work with openHAB’s API. Hysteresis and the DPs would build on that to provide reusable bits of functionality like common calculations, management of one timer per Item, gatekeeper, etc. which seem best to go in Community. Or maybe best to start in Community and then migrate to the main libraries, but in some new module path. I don’t think they would belong in core.

What I’ve thought of so far which can all be implemented generically (some of these are more than just functions, one timer per Item and gatekeeper below are classes to allow users the option of having more than one active in a given set of rules at a time):

  • hysteresis (obviously)
  • management of one timer per Item (you’ve seen the first version of this code already, I’ve moved out from anti-flapping to just generic management)
  • gatekeeper DP
  • Time of Day DP (I want to try to generalize this to a generic state machine at some point)
  • Generic Presence Detection
  • Cascading Timers (e.g. lawn irrigation)
  • Motion sensor timer
  • Generic Is Alive?
  • Manual trigger detection
  • Countdown Timer (put how much time is left on a Timer on the sitemap)
  • Event Limit (kind of the flip side of gatekeeper, instead of delaying a command, ignore the command until a certain amount of time has passed)

As I progress I’m sure I’ll think of more. Once I cover my own examples and DPs I hope to move on to DPs and tutorials posted by others.

I’m certain I can implement all of the above as either a module or a Script with all configuration taking place in configuration.py and in Items and Groups.

I have four implemented thus far (I’ve only submitted hysteresis as a PR so far, I’ll submit the others once I get feedback from the review of that one).

Oh, I’m not looking at avoiding them. I’m just thinking about rewriting Design Pattern: Working with Groups in Rules for the Helper Library docs and trying to figure out which approach to favor. It seems like most blogs and SO posts and the like indicate that List Comprehension is more pythonic, but so far I haven’t decided which is better for the user migrating from Rules DSL.

Rules DSL List stream interface (which comes from Java) has a more intuitive left to right layout. But the map/filter functions definitely have a inner to outer layout and list comprehension has a right to left layout and the syntax is more arcane. I’m not sure which would be easier for migrators to follow and wonder if we should recommend splitting them into separate lines for clarity.

It is clear that with the map/filter functions it does not stream. First all the filter stuff must complete and then the map starts working on the result. List comprehension appears to perhaps work more like the Java stream API where the results from the first operation are streamed to the next one as they become available (e.g. given, Blah.members.filter[ i | i.state == ON].map[state], the map will start running on the first matching result from the filter so you could have both the filter and the map running at the same time rather than the map waiting for the filter to finish).

For the library submission PR though, I’ll switch it to use list comprehension.

1 Like