HABApp 1.0 Beta Test

Spaceman_Spiff · June 28, 2022, 9:25am

Openhab 3.3 is now released and unfortunately there was a breaking change which will generate error messages in the logs (the “ALIVE” event).
So now I have to hurry up with the next HABApp release even though time is short.
To speed things up and because there were so many changes since the last release I decided it would be best to do a public beta test.

Many things changed so expect things to break! Use only for testing or on your own risk.

The main goal of this test is to fix the remaining bugs, get feedback on the new things and also create a comprehensive migration manual.

Changes so far

Startup issues are gone with a new and improved connection mechanism. Big thanks to @j-n-k who did all the work on the openhab side. HABApp now only works with OH3.3!
New configuration library: More settings can be configured in the configuration file. Config values are also described in the docs. Also better error messages (hopefully)
Improved event log performance (BufferEventFile no longer needed and should be removed)
Improved openhab performance (added some buffers)
Improved mqtt performance
Better tracebacks in case of error (imho )
EventFilters can be logically combined (“and”, “or”) so rules trigger only once
Label, Groups and Metadata is part of the OpenhabItem and can easily be accessed
Added possibility to run arbitrary user code before the HABApp configuration is loaded
Fixed setup issues
Fixed some known bugs and introduced new ones
Docker file changed to a multi stage build. Mount points changed to /habapp.

Migration to new version

self.listen_event now requires an instance of EventFilter.

Old:

from HABApp.core.events import ValueUpdateEvent
...
self.my_sensor = Item.get_item('my_sensor')
self.my_sensor.listen_event(self.movement, ValueUpdateEvent)

New:

from HABApp.core.events import ValueUpdateEvent, ValueUpdateEventFilter
...
self.my_sensor = Item.get_item('my_sensor')
self.my_sensor.listen_event(self.movement, ValueUpdateEventFilter())   # <-- Instance of EventFilter

HABApp:
  ValueUpdateEvent -> ValueUpdateEventFilter()
  ValueChangeEvent -> ValueChangeEventFilter()

Openhab:
  ItemStateEvent        -> ItemStateEventFilter()
  ItemStateChangedEvent -> ItemStateChangedEventFilter()
  ItemCommandEvent      -> ItemCommandEventFilter()

MQTT:
  MqttValueUpdateEvent -> MqttValueUpdateEventFilter()
  MqttValueChangeEvent -> MqttValueChangeEventFilter()

Migration to new docker image

change the mount point of the config from /config to /habapp/config
The new image doesn’t run as root. You can set USER_ID and GROUP_ID to the user you want habapp to run with. It’s necessary to modify the permissions of the mounted folder accordingly.

Installation

The dev docs can be found here.
If something is missing please let me know.

The current dev branch can be installed in the venv with the following command:

pip install git+https://github.com/spacemanspiff2007/HABApp.git@Develop

Please let HABApp generate the both the logging and the app config and work from there.

To update to the most recent version you have to uninstall HABApp first and then reinstall.

pip uninstall habapp
pip install git+https://github.com/spacemanspiff2007/HABApp.git@Develop

I’ll ping a couple of users which were interested in a beta test but of course every one is invited!
Thank you for your feedback and tests in advance!

@Dominik_Bernhardt @yfaway @dan12345 @alfista2600

dan12345 · June 28, 2022, 10:24am

brilliant - thank you!

alfista2600 · June 28, 2022, 11:39am

sweet! I’d be happy to test, and probably will. I just got through a migration from independent rules to a parameterized script that auto-creates rules, which was somewhat painful for the wife to live through. I have to review and see if I have anything critical (to her) that might fall victim to beta code…

Spaceman_Spiff · June 28, 2022, 11:51am

That sounds like you should rather use RuleClasses and pass Parameters to the instances or create the Rule Instances from a Parameter file.

Be aware this is a beta and there are issues to expected!!
Safe you and your wife some headache and try things out locally first!
Just create an openhab instance with habapp on your main machine and play around a little bit.

alfista2600 · June 28, 2022, 12:43pm

Thanks, I’ll try to spin up another pi to do some testing… time always limited.
I think I’m using the process you describe above and it is working well. I do have some questions, but would rather you use your limited time on stabilizing this release before helping lousy coders like me.

Dominik_Bernhardt · June 28, 2022, 7:38pm

Thanks a lot. Will give it a try the next days

Dominik_Bernhardt · June 28, 2022, 8:38pm

[2022-06-28T22:36:23+0200] [HABApp                                            ]     INFO | HABApp Version 0.31.3 Dev
[2022-06-28T22:36:23+0200] [HABApp.mqtt.connection                            ]     INFO | Connecting to mqtt:8883
[2022-06-28T22:36:23+0200] [HABApp.openhab.connection                         ]     INFO | Connected to OpenHAB version 3.3.0 (Release Build)
[2022-06-28T22:36:23+0200] [HABApp.openhab.connection                         ]     INFO | Waiting for openHAB startup to be complete

Result of systeminfo endpoint:

{
  "systemInfo": {
    "configFolder": "/openhab/conf",
    "userdataFolder": "/openhab/userdata",
    "logFolder": "/openhab/userdata/logs",
    "javaVersion": "11.0.15",
    "javaVendor": "Eclipse Adoptium",
    "javaVendorVersion": "Temurin-11.0.15+10",
    "osName": "Linux",
    "osVersion": "5.10.92-v8+",
    "osArchitecture": "aarch64",
    "availableProcessors": 4,
    "freeMemory": 80119376,
    "totalMemory": 196083712,
    "startLevel": 70
  }
}

Spaceman_Spiff · June 29, 2022, 3:47am

Does it never get past this message or does it start eventually?

Dominik_Bernhardt · June 29, 2022, 5:57am

No it does not get passed this point. However it rather seems to be an OH issue: openHAB 3.3 Release discussion - #45 by Dominik_Bernhardt

Spaceman_Spiff · June 29, 2022, 8:03am

Do you think it makes sense to make this configurable?

Dominik_Bernhardt · June 29, 2022, 5:48pm

While I have fixed my issue (It was a thing that could not be initialized) I think waiting only until startlevel 70 would for me be a good idea. Otherwise a thing causing issues would block the whole startup of my smart home and I could not even send a notification.
At the end I actually think it is a bug in OH. One failed thing should not stop the complete startup sequence.

Dominik_Bernhardt · June 29, 2022, 5:50pm

I just came across another breaking change:
I my code I use something like:

for item in HABApp.core.Items.get_all_items():
            if isinstance(item, OpenhabItem):
               ...

However, get_all_items seems to not exist anymore. What would be the recommended replacement?

Update:
Ok, found the solution. It’s as simple as this:

from HABApp.core import Items
for item in Items.get_items ():

Spaceman_Spiff · June 30, 2022, 3:54am

The idea was to wait until the restore from the persistence service is done and only then connect to openhab so the items have already a defined state. I guess I’ll add a configuration option for which a default is not created in the config file. So you can manually add min_start_level in the config file and it’s also described in the docs but it’s not created by default.

I only try to keep things inside the rule consistent. If other things change I’ll gladly help you out but I don’t consider it a breaking change. It’s impossible to keep all the modules stable and do development.
But since it’s only advanced coders who use this any way (like you) they’ll figure it out anyway (like you)

It probably makes sense so make the method from the rule self.get_items directly available on the item registry. Wdyt?

Can you reproduce the issue? Then we could create a bug in the oh repo.

Dominik_Bernhardt · June 30, 2022, 5:33am

At lesat someone else reported the same issue in the 3.3 release thread. I’ll try to reproduce and file a bug.

Dominik_Bernhardt · June 30, 2022, 5:38am

And one more thing…
I got habapp starting without and error messages and updated my first rules to use the new EventFilter syntax but I don’t see any events
Habapp is successfully querrying all Items and things from OH but the eventlog is empty beside ThingStatusEvents.

Spaceman_Spiff · June 30, 2022, 11:17am

I though you’d announce same great PR that fixes all my bugs.

That’s really strange. I can not reproduce the issue.
Is it just with logging e.g. do your rules process the events correctly?
Or does openHAB emit events but HABApp somehow doesn’t subscribe to them?

Spaceman_Spiff · July 1, 2022, 4:25am

Dockerfile also changed to a multi stage build and it now can run without root privileges.
However the volume mounts changed so it would be nice if someone could test this and write a short text how to migrate to the new docker file.

Dominik_Bernhardt · July 1, 2022, 8:07pm

At least I setup an environment to debug HABApp to better analyse the issues. Not yet there to provide any meanigfull PR

It actually seems to be on the OH side. Also when having breakpoints in the eventhandle I can only see ALIVE, and ThingEvents. No single item event is comming through. On the OH UI though I can see the stream of events in the developer tools.
I also tested with the latest stable HABApp release and I have the same issue there. At least it is not related to the Beta.

Spaceman_Spiff · July 2, 2022, 5:58am

It was just a small joke because apple used to announce the new and exiting stuff this way.

Does it work with OH3.2?
Maybe the event topic filter is incorrect and there were some changes on the OH side how the subscription was evaluated.
I’ll try to investigate further, too.

yfaway · July 2, 2022, 2:39pm

Seb, is it this version still compatible with OH 3.2? I still haven’t migrated to 3.3 yet.