Unsubscribe from events in HABApp?

1

Hi,

I started using HABApp (Version 24.11.1) icw OH (4.1.3).

First off: a big thank you to Sebastian [Spaceman_Spiff] for making this plugin available. Being able to use a modern python to write scripts + use a debugger while doing so makes a lot of difference. I really appreciate this, so: Thank You!

I have a question though: I am writing a rule that powers an (OpenHASP) display.
In this rule, I have several event handlers (started through event_listen()) that monitor MQTT topics and in response they update other MQTT topics in the background.

I notice that it happens, especially during development, that when the Rule restarts/reloads, the old event handlers that were installed during previous instance of that Rule are still active. The new instance re-creates these event listeners, and before I know it, I have a lot of duplicate event listeners in the background, and they can step on each other as they all try to alter the same mqtt topicsā€¦

I have added some hacks and I can programmatically detect when this happened. But I seem to find no way of fixing the situation. I already tried just exiting the whole python program, using sys.exit() or plain exit() but that has no effect. (HABApp just keeps running) (*)

I did see in the docs that there is a HABApp.Rule.on_rule_removed(), but I cannot seem to find how to remove the event listeners that were installed by the Rule.

Does anyone know how to do this?

thanks & br, Johan.

(*) just to clarify : the intention of exiting the program would be to let the OS detect the process exit and start a new HABApp process. A brute-force way to get rid of the orphaned event handlersā€¦

2

i do not have the issue with not cancelling listeners when reloading a rule (also not if it crashes). earlier i had a rule with a threading part, there i also had to restart habapp when my rule crashed to end the threading rule.

in my shutter rule i have a listener when to open in the morning. when i change the desired time i cancel the listener and create a new like:

self.listener_morning = self.run.on_every_day(time(hour=hour, minute=minute), self.morning_open)
...
self.listener_morning.cancel()

i am not sure if this will still work when you put it in def on_rule_removed(self): (if then self.listener_morning is still accessible)

3

Hi Stefan,

thanks for your reply. In my case the event handler is not a scheduled task, instead it is started with listen_event()

here is a small code snippet:


    [...]
    self.load_widget(hasp_id.page, hasp_id.id-3, design, cfg, jsonl_cmd_topic)
    
    if None != item:
      def update_in(event):
        value = event.value
        value = conv.get(value, -1) if conv else value
        jsonl_cmd_topic.publish(f"""{{{hasp_id.json(-2)},"text":"{icon(value)}"}}""")
        jsonl_cmd_topic.publish(f"""{{{hasp_id.json()},"val":"{value}"}}""")
      
      item.listen_event(update_in, ValueUpdateEventFilter())

      def update_out(event):
        ev = event.value
        if ev['event'] == 'up':
            item.oh_send_command(r_conv[ev['val']] if conv else ev['val'])
      
      self.listen_event(hasp_id.topic(), update_out, ValueUpdateEventFilter())
      [...]

when instantiating a widget and if an [openhab] item was passed in, after drawing the widget on the screen, there is an event handler (update_in) that monitors the [openhab] item, and updates the UI (by sending a small json command to openHASP) when the item was changed (either through this widget or through some other action)

secondly, there is another event handler (update_out) that monitors (by listening to the appropriate MQTT item) if someone presses the button on the openHASP screen. In such case, the event handler will alter the state of the [openhab] item.

Iā€™ve noticed a few times that, after the Rule reloads and re-creates all widgets, there are orphaned event handlers still running and altering the itemā€™s states.

That said, I now notice that listen_event returns an EventBusListener which also has a cancel() methodā€¦ that may very well be the key to the solution.

Iā€™ll work on this, and Iā€™ll report back in this thread if I was able to solve it this way.

Thanks again for your message, you gave me a lead :slight_smile:

1 Like
4

This was indeed how I solved it. My class now holds on to a list of listeners that it launched, and it cancels them one by one when reloading (for whatever reason).

1 Like
5

Hm - normally this is not necessary.

When a rule is unloaded all active listeners that are created with .listen_event should be automatically unloaded. If you set the log to debug you should see the appropriate messages.
Your either hitting a bug or something strange is happening.
Do you have a minimal example with which reproduces the issue?

6

hi Sebastian,

Thanks for this additional info. That is good to know.

When I look at my code, I see some of my listeners were started using self.listen_event(item, eventhandler, eventfilter) while some others were started like this: item.listen_event(eventhandler, eventfilter)

Could that difference have something to do with the fact that some were not being unloaded automatically?

I donā€™t have a minimal example to illustrate the problem, but I will try to create one that can reproduce it. Iā€™ll let you know once I have it.

thanks & br, Johan.

7

hi @Spaceman_Spiff,

Please find a minimum configuration that reproduces the state where there are orphan message handlers. It happens when one rule refers to another; and then only one of the 2 gets reloaded:

create the following 2 rules:

dummy_HASP_plate.py :

import HABApp
from HABApp.core.events import ValueUpdateEventFilter
from HABApp.core.items import Item

from datetime import timedelta
import logging

import typing

if typing.TYPE_CHECKING:                            # This is only here to allow
  from .dummy_HASP_utils import dummy_HASP_loader   # type hints for the IDE


log = logging.getLogger('dummy_HASP_plate')


class dummy_HASP_plate(HABApp.Rule):

    def __init__(self, plate_name) -> None:
        super().__init__()

        # dummy item, that simulates the online/offline behavior of HASP plates
        self.HASP_LWT_item = Item.get_create_item("dummy HASP LWT topic", "offline")

        # another dummy item, this one simulates a periodic update (such as temperature)
        self.some_periodic_item = Item.get_create_item("dummy periodic item", 0)
        # in 30 seconds from now, we'll increment this every 5 seconds, wrapping around at 97
        self.run.at(self.run.trigger.interval(start = timedelta(seconds=30), interval=timedelta(seconds=5)), lambda : self.some_periodic_item.post_value( (self.some_periodic_item.get_value() + 1) % 97))

        # listen to mqtt topics o/t plate
        self.listen_event(f'dummy HASP LWT topic', self.lwt_updated, ValueUpdateEventFilter())

        # simulate that the plate comes online 10 seconds from now:
        self.run.once(10, lambda : self.HASP_LWT_item.post_value("online"))

        # also listen to the dummy recurrent update (for debug purposes only)
        self.listen_event(f'dummy periodic item', self.periodic_updated, ValueUpdateEventFilter())


    def lwt_updated(self, event) -> None:
        self.is_online = event.value == 'online'
        log.info( f'mqtt topic "{event.name}" updated to {event.value}  (lwt update)')
        if self.is_online:
            self.run.soon(self.__install_hasp_plate)


    def periodic_updated(self, event) -> None:
        log.info( f'mqtt topic "{event.name}" updated to {event.value}')


    def __install_hasp_plate(self):
        self.loader = self.get_rule('dummy_HASP_loader')
        self.loader.load_design()           


dummy_HASP_plate("dummy_plate")

dummy_HASP_utils.py :

import HABApp
from HABApp.core.events import ValueUpdateEventFilter


import logging


log = logging.getLogger('dummy_HASP_loader')


class dummy_HASP_loader(HABApp.Rule):

    def __init__(self) -> None:
        super().__init__()


    def load_design(self) -> None:

        def update_solar(event):
          log.info( f'mqtt topic "{event.name}" updated to {event.value}')

        self.listen_event(f'dummy periodic item', update_solar, ValueUpdateEventFilter())


dummy_HASP_loader()

The 2nd file is a utility that gets re-used between various plates. It installs the UI and attaches events to the UI for various openHASP plates.

The first file is an instantiation for a specific plate. It calls the helper load_design.

If you have the editor open on file #1 and save it, it will reload the first file, but not the 2nd.
Looking at the log you can see that each time, event handlers for dummy_HASP_loader are being duplicated:

from the log:

[2024-12-26 12:16:00,753] [        dummy_HASP_loader]     INFO | mqtt topic "dummy periodic item" updated to 84
[2024-12-26 12:16:05,753] [        dummy_HASP_loader]     INFO | mqtt topic "dummy periodic item" updated to 85
[2024-12-26 12:16:05,754] [         dummy_HASP_plate]     INFO | mqtt topic "dummy periodic item" updated to 85
[2024-12-26 12:16:05,754] [        dummy_HASP_loader]     INFO | mqtt topic "dummy periodic item" updated to 85
[2024-12-26 12:16:05,754] [        dummy_HASP_loader]     INFO | mqtt topic "dummy periodic item" updated to 85
[2024-12-26 12:16:10,753] [        dummy_HASP_loader]     INFO | mqtt topic "dummy periodic item" updated to 86

=> [ dummy_HASP_plate] INFO | mqtt topic "dummy periodic item" updated to 85 appears three times

8

Note also that a similar scenario happens when the plate goes ā€˜offlineā€™ and then ā€˜onlineā€™ again.

the following section will then be executed another time, adding more event handlers:

    def __install_hasp_plate(self):
        self.loader = self.get_rule('dummy_HASP_loader')
        self.loader.load_design()

This is because the dummy_HASP_loader is instantiated once (at the bottom of dummy_HASP_utils.py), and other rules all (re-)import that same instance over and over.

If the other rules could instantiate their own (local) copy of dummy_HASP_loader rule then it could be destroyed (and perform its cleanup) when the client ruleā€™s local instance of dummy_HASP_loader goes out of scopeā€¦

9

You are creating the event listener in dummy_HASP_loader and not in dummy_HASP_plate.

From Rule A you access Rule B and create the event listener in Rule B every time something happens in Rule A.
To unload the event listeners you would have to unload Rule B and not Rule A.
You are unintentionally mixing rule scopes here.

Why donā€™t you put all your HASP Plate drivers in one file, define a base class and make the other drivers inherit from the base class? That way you donā€™t have these problems at all.

Reusing rule files is always tricky and I always recommend against it.
Either the code is so static that it can be used as a rule library (lib folder). However changes will then only picked up on HABApp restart.
Or the code is dynamic then it should be one file and e.g. (HABApp) items should be used to interact with parts of the code.

10

hi @Spaceman_Spiff,

yes, I do understand the root issue is that : Rule B creates event listeners on behalf of Rule A, but then the lifetime of Rule B exceeds the lifetime of Rule A.

That said, I could not find a better way to organize this code - I am new to the HABApp framework though, so I am still learning.

Note that what I shared is a simplified version of the code. The real implementation of dummy_HASP_loader is currently >400 lines long, and I expect it will double its size a few times still, as I add more widgets.

Also, the real dummy_HASP_plate is currently ~200 lines of code, and I expect to create 10s of these. Think of each plate as a smart lightswitch, and each switch will have its own design / layout of widgets.

I like the suggestion of subclassing where dummy_HASP_loader would be an (abstract) base class, but I think there will be too much code put it all in one python file.

Some additional info about the design I have now (and that I did like, apart from the lifetime issues) : there is a plate_loader that centralizes all of the logic and the handling of the messaging. The (10s of) plate objects just ask plate_loader to draw widgets (= typically a button with some indicator of the itemā€™s state, tied to an item either in OH or MQTT) on location ā€˜xā€™,ā€˜yā€™ of page ā€˜pā€™ of the plate, and then the plate object is done with it. ā€œSet and forget itā€, really. And the same is true for the plate_loader: it attaches message handlers to the widget & item(s) and also the loader is done with each widget instance as well. Itā€™s a clean and elegant solution and it allows re-use of widgetsā€™ implementations, on a different location or even on a different plate, with a different label or icon, etc. All of this without duplicating any of the logic. Really elegant.

To address the lifetime issues, I just tried this little hack:

in dummy_HASP_utils.py I added a factory for dummy_HASP_loader:


class loader_factory(HABApp.Rule):

    def __init__(self) -> None:
        super().__init__()

    def create_loader(self):
      return dummy_HASP_loader()


loader_factory()

and then in dummy_HASP_plate make this change:

    def __install_hasp_plate(self):
        #self.loader = self.get_rule('dummy_HASP_loader')
        self.loader = self.get_rule('loader_factory').create_loader()
        self.loader.load_design()

hoping this would have a new instance of class B for each class A, so that if class A goes out of scope, then itā€™s instance of class B would too. Unfortunately, it does not work as expected. The instance of class B keeps on going after class A gets reloaded. The log even shows this warning:

HABApp.Rule]  WARNING | Added another rule of type dummy_HASP_loader but file load has already been completed!

About:

Maybe that is something I could look into. Is there a description somewhere how to create a rule library? Is it just a matter of placing .py files in a certain location?

thanks & br, Johan.

11

hi @Spaceman_Spiff,

I tinkered a bit more with the example and moved the plate_loader to the library (\lib\ dir) as follows:

from HABApp.core.items import Item
from HABApp.core.events import ValueUpdateEventFilter


import logging


log = logging.getLogger('dummy_HASP_utils2')


class dummy_HASP_loader():

    def __init__(self, plate_name) -> None:
      self.plate_name = plate_name
      log.info(f"dummy_HASP_utils2 : dummy_HASP_loader '{self.plate_name}' is being instantiated")


    def __del__(self):
      log.info(f"dummy_HASP_utils2 : dummy_HASP_loader '{self.plate_name}' is being destroyed")
      self.handler.cancel()   # this is needed!


    def load_design(self) -> None:
        
        self.item = Item.get_create_item("dummy periodic item")

        def update_solar(event):
          log.info( f'mqtt topic "{event.name}" updated to {event.value}')

        self.handler = self.item.listen_event(update_solar, ValueUpdateEventFilter())

It works pretty well, but as you can see I still need to manually keep track of which event listeners are running and make sure to cancel these when the object goes out of scope. (That was a bit of a surprise tbh. because the item it listens to was created in this class)

On the client side, I can do this now:

    def lwt_updated(self, event) -> None:
        self.is_online = event.value == 'online'
        log.info( f'mqtt topic "{event.name}" updated to {event.value}  (lwt update)')
        if self.is_online:
            self.run.soon(self.__install_hasp_plate)
        else:
            log.warning("will be destroying!")
            self.loader = None

    def __install_hasp_plate(self):
        self.loader = dummy_HASP_loader("dummy_HASP_plate")
        self.loader.load_design()

The loader now no longer is an HABApp rule. And, when the (physical) plate goes offline, the messaging stops when the loader looses scope.

This works similarly when the plate (=the HAPApp.rule) is being reloaded, in the background the loader instance gets destroyed.

Thanks & BR, Johan.

12

Hm - there is still something flaky going on.
I would have expected the scope to be the calling rule, too.
If I have time Iā€™ll investigate a little bit further.

Additional ideas (from the limited information I have):
Your hasp loader doesnā€™t need to be a class. It could be a function that returns both and item name and a new function (or a list of those). That way you can listen to the event in the rule.
Iā€™m also not sure why you would want to unsubscribe when your hasp plate goes offline, imho thereā€™s no harm in keeping the subscription alive.
However I think the most elegant solution would still be to define a base class (you can do that in the lib folder, too) and then inherit in the rule file from the base class and instantiate the rule there.

13

hi @Spaceman_Spiff,

I thought about this a bit more and I think the underlying reason is: in that last example the dummy_HASP_loader is no longer a subclass of HABApp.Rule. Instead it is a plain Python class, so it does not have a Context bound to it.

The item itself was created inside that class, but I assume itā€™s bound to the scope of the calling rule (dummy_HASP_plate) which does have a context. So therefore even if dummy_HASP_loader is destroyed, that ā€˜self.itemā€™ (and its handler) that was in it, is still around.

Note that the real def load_design(self) -> None: does more than just install event listeners: it draws the widgets on the screen. Whenever a plate goes offline, that typically means it was powered down or restarted. When it comes back online, its screen will be blank, so load_design() needs to be called again to draw the widgets. It makes sense to install the listeners at the same time (while drawing the widgets) because that is also when the loader finds out which widgets will be drawn.

To keep the messaging running would require 2 calls essentially: one to draw the widgets and one to install the listeners, where the 2nd one is only called once in the lifetime of the application. That makes the design a bit more complicated imho.

Furthermore, I do also envision future cases where I will dynamically add & remove widgets while the plate is online, and then I will need to remove listeners (the openHASP MQTT topics are bound to the idā€™s of the buttons on the screen. there is only a limited range of id values available, so you want to reuse id numbers when drawing something new. removing a button and then drawing a different one with the same id will recycle an MQTT topic for a different target, so that old listener should be stopped)

Thanks & BR, Johan.