HABApp - Easy automation with openHAB

Hi Sebastian, Following the clue, I removed HABApp and did a clean install, and fresh configuration. When starting as a service the log file HABApp.log seems to show correct operation (albeit with a connection to openhab issue which I will look at seperately).
When I stop the service, and start manually via /opt/habapp/bin/python3 /opt/habapp/bin/habapp -c /etc/openhab/habapp, on the console I get:

                                        1.0.8
Error loading logging config: Unable to configure handler 'HABApp_default'
Error loading logging config: Unable to configure handler 'HABApp_default'
/opt/habapp/lib/python3.9/site-packages/ruamel/yaml/reader.py:188: ResourceWarning:unclosed file <_io.FileIO name='/var/log/openhab/HABApp_events.log' mode='ab' closefd=True>
Error loading logging config: Unable to configure handler 'HABApp_default'
Error loading logging config: Unable to configure handler 'HABApp_default'
Executing <Task finished name='Task-1' coro=<Runtime.start() done, defined at /opt/habapp/lib/python3.9/site-packages/HABApp/runtime/runtime.py:29> result=None created at /opt/habapp/lib/python3.9/site-packages/HABApp/__main__.py:38> took 0.184 seconds

Should I be concerned?

It think this is a simple permission error. The user which you are running HABApp does not have sufficient permissions to the log folder and is a different one than the one which is used by systemctl.
I think systemctl runs as a openhab user but you log in with openhabian user.
So you should prefix your command with

sudo -u openhab opt/habapp/bin/python3 /opt/habapp/bin/habapp -c /etc/openhab/habapp

However Iā€™d just start it through systemctl.

i am on openhabian with oh 4.0.0.m1 and yesterday i tried to update to m2

but when finished my log got flooded with errors like this:

==> /var/log/openhab/HABApp.log <==

2023-04-18 08:15:41.094 [ERROR] [HABApp                              ] - Error Unknown Event: ItemStateUpdatedEvent for {'topic': 'openhab/items/iCPU_Temp_str/stateupdated', 'payload': '{"type":"String","value":"62"}', 'type': 'ItemStateUpdatedEvent'} in on_sse_event:

2023-04-18 08:15:41.095 [ERROR] [HABApp                              ] - --------------------------------------------------------------------------------

2023-04-18 08:15:41.096 [ERROR] [HABApp                              ] - Traceback (most recent call last):

2023-04-18 08:15:41.097 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/map_events.py", line 45, in get_event

2023-04-18 08:15:41.098 [ERROR] [HABApp                              ] -     return _events[event_type].from_dict(topic, payload)

2023-04-18 08:15:41.098 [ERROR] [HABApp                              ] - KeyError: 'ItemStateUpdatedEvent'

2023-04-18 08:15:41.099 [ERROR] [HABApp                              ] - 

2023-04-18 08:15:41.100 [ERROR] [HABApp                              ] - During handling of the above exception, another exception occurred:

2023-04-18 08:15:41.100 [ERROR] [HABApp                              ] - 

2023-04-18 08:15:41.101 [ERROR] [HABApp                              ] - Traceback (most recent call last):

2023-04-18 08:15:41.102 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/connection_handler/sse_handler.py", line 31, in on_sse_event

2023-04-18 08:15:41.102 [ERROR] [HABApp                              ] -     event = get_event(event_dict)

2023-04-18 08:15:41.103 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/map_events.py", line 47, in get_event

2023-04-18 08:15:41.104 [ERROR] [HABApp                              ] -     raise ValueError(f'Unknown Event: {event_type:s} for {_in_dict}')

2023-04-18 08:15:41.104 [ERROR] [HABApp                              ] - ValueError: Unknown Event: ItemStateUpdatedEvent for {'topic': 'openhab/items/iCPU_Temp_str/stateupdated', 'payload': '{"type":"String","value":"62"}', 'type': 'ItemStateUpdatedEvent'}

2023-04-18 08:15:41.141 [ERROR] [HABApp                              ] - Error Unknown Event: ItemStateUpdatedEvent for {'topic': 'openhab/items/iOwTemp_KWL_Fortluft/stateupdated', 'payload': '{"type":"Quantity","value":"9.5 Ā°C"}', 'type': 'ItemStateUpdatedEvent'} in on_sse_event:

2023-04-18 08:15:41.142 [ERROR] [HABApp                              ] - --------------------------------------------------------------------------------

2023-04-18 08:15:41.142 [ERROR] [HABApp                              ] - Traceback (most recent call last):

2023-04-18 08:15:41.143 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/map_events.py", line 45, in get_event

2023-04-18 08:15:41.144 [ERROR] [HABApp                              ] -     return _events[event_type].from_dict(topic, payload)

2023-04-18 08:15:41.144 [ERROR] [HABApp                              ] - KeyError: 'ItemStateUpdatedEvent'

2023-04-18 08:15:41.145 [ERROR] [HABApp                              ] - 

2023-04-18 08:15:41.146 [ERROR] [HABApp                              ] - During handling of the above exception, another exception occurred:

2023-04-18 08:15:41.146 [ERROR] [HABApp                              ] - 

2023-04-18 08:15:41.147 [ERROR] [HABApp                              ] - Traceback (most recent call last):

2023-04-18 08:15:41.148 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/connection_handler/sse_handler.py", line 31, in on_sse_event

2023-04-18 08:15:41.148 [ERROR] [HABApp                              ] -     event = get_event(event_dict)

2023-04-18 08:15:41.149 [ERROR] [HABApp                              ] -   File "/opt/habapp/lib/python3.9/site-packages/HABApp/openhab/map_events.py", line 47, in get_event

2023-04-18 08:15:41.149 [ERROR] [HABApp                              ] -     raise ValueError(f'Unknown Event: {event_type:s} for {_in_dict}')

2023-04-18 08:15:41.150 [ERROR] [HABApp                              ] - ValueError: Unknown Event: ItemStateUpdatedEvent for {'topic': 'openhab/items/iOwTemp_KWL_Fortluft/stateupdated', 'payload': '{"type":"Quantity","value":"9.5 Ā°C"}', 'type': 'ItemStateUpdatedEvent'}

as i didnt know how to procceed i restored a backup and now i am again on m1

i saw here somthing i think could be the reason:

should i wait with update to m2 or did i do something wrong?

Yes - OH4 is not yet supported

i would like to use the DictParameter to load parameters and use them in a rule. amazing that if i change a parameter in the yaml file it automatically gets loaded in my rule.

but what can i do if i want to change one of the parameters through an item. how can i write the new value back to the yaml file?

i tried to do this manually in my rule by import yaml but then i recieve an error

2023-04-23 21:16:47,486 [ERROR] [My_HABApp] - ### faulty.py detected error in HabAPP: Exception in load: No module named 'yaml'

but when i try to

pip install pyyaml

or

pip3 install pyyaml

i recieve

Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (5.3.1)

is there a possibllity to update a yaml file through a new value from an item?

There is currently no mechanism in place to achieve that.
You could load the param file with an yaml reader, change the value in place and write it back to disk.
But that seems like a lot of (unnecessary) work.
However if you have a valid use case you can open an issue and I might add it in one of the upcoming versions.

In the mean time Iā€™d suggest you use a normal openHAB item together with mapdb to achieve the same behavior.

1 Like

my problem was i could not import yaml because habapp always complained

Exception in load: No module named 'yaml'

but when i wanted to install pyyaml on command line i received:

Requirement already satisfied: pyyaml in /usr/lib/python3/dist-packages (5.3.1)

now i found a way (hopefully correct) how to install pyyaml for habapp from commandline:

cd /opt/habapp
source bin/activate
pip install pyyaml

now i can import yaml and created a rule to update values that where changed through an item, with the benefit from habapp that if i edit the yaml file direct the values are passed to the rule without need to reload:

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# HABApp:
#   reloads on:
#    - params/heating_demo.yml
"""
example how to have a parameter file that automatically passes values to the rule
without need to reload the rule and additionally values can be changed from an item
and the new value will be updated in the yaml file
(the dictionary at the end of the file is only needed for the very first load to
create the yaml file)
"""
import logging
import yaml

from HABApp import Rule, DictParameter, Parameter
from HABApp.openhab.items import OpenhabItem
from HABApp.openhab.events import ItemStateChangedEventFilter

class Heating(Rule):
    def __init__(self, key, values):
        super().__init__()
        self.log = logging.getLogger('My_HABApp')
        self.log.debug('### Heating: rule (re)loaded ###')

        para_file = 'heating_demo'
        self.zone_parameter = DictParameter(para_file, key, default_value=values)
        self.yaml_file = f'/etc/openhab/habapp/params/{para_file:s}.yml'
        self.zone_name = key

        # working this way the variable will NOT be updated when yaml changed:
        # self.setpoint_item_name = self.zone_parameter['setpoint_item_name']
        # working this way the variable will be updaten when yaml changed:
        self.setpoint_item_name = Parameter(para_file, key, "setpoint_item_name").value
        self.setpoint_temp = Parameter(para_file, key, "setpoint_temp").value

        # set item with saved value from yaml file
        self.oh.post_update(self.setpoint_item_name,
                            self.setpoint_temp)

        # init trigger for regularly query of the temp sensors
        self.run.every(0, 10, self.parameter_info)

        # init trigger if item for setpoint temp was adjusted
        OpenhabItem.get_item(self.setpoint_item_name).\
        listen_event(self.setpoint_temp_changed, ItemStateChangedEventFilter())

    def setpoint_temp_changed(self, event):
        '''
        setpoint temp changed, update value in yaml file
        '''
        self.log.debug('### Heating: setpoint_temp_changed from '\
                      f'{self.setpoint_temp} to {event.value} degrees ###')

        config = yaml.safe_load(open(self.yaml_file, encoding="utf-8"))
        config[self.zone_name]['setpoint_temp'] = float(event.value)
        yaml.dump(config, open(self.yaml_file, 'w', encoding="utf-8"))

    def parameter_info(self):
        '''
        see all values that are stored in the yaml file
        '''
        # self.log.debug(f'### this are the parameters to work with: {self.zone_parameter}')
        self.log.debug(f'### this is the parameter self.setpoint_item_name: {self.setpoint_item_name}')

heating_zones_dict = {
    'EK_Fbh_FlEWoK' : {
        'sensor_item_names' : ['iEK_Fbh_FlEWoK_Sensor'],    # Temperatur Sensor Item(s)
        'sensor_owns' : ['*#4*1*0##'],                      # OWN(s) fĆ¼r Abfrage Temperatur(en)
        'setpoint_item_name' : 'iEK_Fbh_FlEWoK_Set',        # Item zur Einstellung Soll-Temperatur
        'setpoint_temp' : 21.0,                             # (Grad) Soll-Temp (nur Erstinit.)
        'actor_item_names' : ['iEK_Fbh_FlEWoK_Actor'],      # Aktor fĆ¼r Fbh Stellantrieb
        'state_item_name' : 'iEK_Fbh_FlEWoK_State'
    },
    'EK_Fbh_Schlafen' : {
        'sensor_item_names' : ['iEK_Fbh_Schlafen_Sensor'],
        'sensor_owns' : ['*#4*2*0##'],
        'setpoint_item_name' : 'iEK_Fbh_Schlafen_Set',
        'setpoint_temp' : 11.0,
        'actor_item_names' : ['iEK_Fbh_Schlafen_Actor'],
        'state_item_name' : 'iEK_Fbh_Schlafen_State'
    },
    'EK_Fbh_Bad' : {
        'sensor_item_names' : ['iEK_Fbh_Bad_Sensor'],
        'sensor_owns' : ['*#4*3*0##'],
        'setpoint_item_name' : 'iEK_Fbh_Bad_Set',
        'setpoint_temp' : 22.0,
        'actor_item_names' : ['iEK_Fbh_Bad_Actor'],
        'state_item_name' : 'iEK_Fbh_Bad_State'
    }
}

for k, v in heating_zones_dict.items():
    Heating(k, v)

of course, this could also be achieved with a persisted item, my goal was to have all parameters for heating together :slight_smile:

1 Like

If the keys in your parameter dict are constant and everywhere the same you can also get the values directly instead of doing the lookup every time

self.setpoint_item_name = Parameter(para_file, key, "setpoint_item_name")
self.setpoint_temp = Parameter(para_file, key, "setpoint_temp")

# use like this if you want to access the value
self.log.debug(f'{self.setpoint_item:s}' changed to xxx')
# or like this but make sure to never cache the .value
self.my_func(self.setpoint_item.value)

Your watchdog can be written more readable:

    def watchdog(self):
        item = OpenhabItem.get_item(self.zone_parameter['setpoint_item_name'])
        temp = self.zone_parameter['setpoint_temp']
        item.oh.post_update_if(temp, not_equal=temp)

thank you for the suggestions, i edited my post.

now i realized that if i change the name of an item that has a listen_event this will no longer trigger. so i either could cancel the listener or (i think better) i use your file properties and let the rule reload. i already implemented that in my rule.

important: first i could not get it work, although the yaml changed the rule did not reload. that was when i worte the code at this position:

#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
example how ....bla bla
"""
# HABApp:
#   reloads on:
#    - params/heating_demo.yml

but since i moved it to this position it works great:

#!/usr/bin/python3
# -*- coding: utf-8 -*-
# HABApp:
#   reloads on:
#    - params/heating_demo.yml
"""
example how ....bla bla
"""

then i even dont need a watchdog to update the item although i like your suggestion to write it more readable - i will remember that :slight_smile:

1 Like

Subscribing to mqtt topic hirarchy?

I would need to listen to mqtt events of a complete hirarchy (e.g. ā€œshellies/#ā€). I know I can specify in the configfile to subscribe to a topic hirarchy but I donā€™t know how to listen to it without having to specify a listen statement for every concrete topic.
Is there a way to listen to all events on the bus?

My workaround would be to create a dedicated mqtt client instance for that.

No - currently not.

I typically solve this by creating a rule per shelly device which then subscribes to all the corresponding channels.

class ShellyShutter(Rule):
    def __init__(self, device_id: str):
        super().__init__()
        self.device_id = device_id

        item_mqtt_pos = MqttItem.get_create_item(f'shellies/shellyswitch25-{self.device_id}/roller/0/pos', 0)
        ...

for _id in DictParameter('shelly', 'shutter').keys():
    ShellyShutter(_id)

I follow a similar approach. I additionally wanted to listen to the announce topics to add some kind of discovery functionality for tasmotas and shellies. Iā€™ll have a look into adding a generic catch-all listener.

I usually take a little bit of different approach now since I prefer the ā€œfail fastā€ concept.
I have a parameter file where I configure my devices. That way I will have an error e.g. if an item is not there. Obviously this does not apply here since the mqtt topics are only generate when generating the item. Also I use the parameter file as a kind of scratchpad to add additional information as a comment.

Watching all shellies is easy because they announce through a special topic (or at least they used to):

class ShellyAnnounceWatcher(Rule):
    def __init__(self):
        super().__init__()
        self.temperature = MqttItem('shellies/announce')
        self.temperature.listen_event(self.announce, MqttValueUpdateEventFilter())

        self.params = Parameter('shelly')

    def announce(self, event: MqttValueUpdateEvent):
        # Try to resolve name
        _id = event.value['id'].split('-')[1]
        name = _id
        for _d in self.params.value.values():
            if name in _d:
                name = _d[name]
                break
        
        if isinstance(name, list):
            name = name[0]
        logging.getLogger(f'Shelly.{_id[-6:]}').debug(f'{name} announce: {event.value}')

How do I work around long script executions on habapp scripts? :thinking: thereā€™s some functions my scripts do (sending data over a slow mobile broadband connection) that take some time to finish. Habapp throws warnings ā€œExecution of Publish2Hyperion.publish_cam_image took too long: 5.06sā€

By the way (suggestion box): HabApp should have its own category now on the forums :slightly_smiling_face:

I run those tasks in threads, but I know @Spaceman_Spiff disapproves!

Yeah my first thoughts was threads/asyncio but thought Iā€™d hear whats recommended to make as little ā€œmessā€ as possible in regards to HabApp. Have you had any problems ?
These functions would mostly be a fire-and-forget kind of function, like upload image etc.

Iā€™ve had zero problems. I have many HABApp rules which look like this:

def doorbell_rings(self, event: ValueUpdateEvent):

    threading.Thread(target=self.doorbell_thread, args=[event.name, event.value]).start()

Although Iā€™m betting it would all go horribly wrong if the thread took a very long to complete or, worse, didnā€™t ever stop.

I already asked Kai half a year ago but he disapproved so itā€™ll stay under 3rd party.


Either do it properly with asyncio or if itā€™s fire and forget you can easily run a python script.
Itā€™s not really an issue because HABApp uses a threadpool internally for sync functions so the warning can be ignored.
Maybe I should add a config switch or some other way to prevent it from being logged.
Because the solution should not be just to circumvent the warning :wink:

The internal thread pool is way better because you get beautiful trace backs, error wrappers and error processing etc.

2 Likes

yes, internal thread pool would be much better - perhaps a config switch? Or, even better, and now Iā€™m being really demanding, a special item we could turn on and off without restarting habapp, to assist debugging?