HABApp potential thread / event contention issue

Okay, so with fresh OH and HABApp containers, I was not able to reproduce the problem with MQTT replay and a few test items in OH. Everything flowed smoothly and was lightning fast.

So, I’ve now spent the rest all day trying other things.. like faster SSDs in the RPi. Nothing made any appreciable difference.

Then, I discovered the docker stats command.

CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O     PIDS
b0fe37a38f44   openhab_openhab_1   133.28%   0B / 0B             0.00%     8.21MB / 23.2MB   448MB / 0B    175
86d103476284   samba_samba_1       0.01%     0B / 0B             0.00%     121kB / 130kB     40.9MB / 0B   5
f4a36d129c6f   mosquitto_mqtt_1    1.65%     0B / 0B             0.00%     8.67MB / 17.3MB   6.09MB / 0B   1
f3b07ba3f563   unifi               29.63%    0B / 0B             0.00%     3.67MB / 596kB    281MB / 0B    132
38c1a9f43a85   habapp_habapp_1     48.89%    0B / 0B             0.00%     3.13MB / 692kB    0B / 0B       21

Hoooooooly crap. I was not expecting that. When HABApp is running, openhab and habapp use 30-40% each constantly, with occasional huge peaks.
If I shut HABApp down, OH drops to 2-3% idle.

So, that’s some crazy CPU usage. But, I figured, maybe it’s one of my scripts?
So, I moved ALL of my scripts out.
Usage dropped way down to single digit %, for both HA and OH.
Then I moved the scripts back, one by one.
Then suddenly I had moved ALL the scripts back.
Usage was still in the single digits, and performance was perfect!

So then, I shut the habapp container down… and started it again.
As you can probably guess, CPU is pegged again!

And then, I moved the scripts out again, folder by folder.. and this time CPU usage STAYED high. With no scripts at all remaining.

Then I moved them all back in, and all back out. CPU usage dropped again!
..and then I moved the back in. CPU usage stayed low.

Restarting the container without scripts was totally fine too.

So, what do you think @Spaceman_Spiff?

Actually, I have an idea what you might say:

Do you think this might be the same issue that is solved in the beta? If so I’d sure love to try it.

I myself have my doubts as to whether it’s the same issue.
Because, I noticed now when modifying a particular script, that not only was CPU usage high, and would not go down when I moved the script out… the script kept running and HABApp would not reload it no matter what I did, until I restarted the HABApp container.

So, could it be that something in the auto file refresh part of HABApp got stuck in a tight loop?

By the way, this is the performance on my system when CPU is not pegged.

CONTAINER ID   NAME                CPU %     MEM USAGE / LIMIT   MEM %     NET I/O           BLOCK I/O     PIDS
b0fe37a38f44   openhab_openhab_1   1.93%     0B / 0B             0.00%     22.9MB / 54.5MB   448MB / 0B    168
86d103476284   samba_samba_1       0.02%     0B / 0B             0.00%     2.01MB / 2.48MB   40.9MB / 0B   5
f4a36d129c6f   mosquitto_mqtt_1    1.03%     0B / 0B             0.00%     26.9MB / 54.8MB   6.09MB / 0B   1
f3b07ba3f563   unifi               6.71%     0B / 0B             0.00%     10.7MB / 1.59MB   283MB / 0B    132
df5b4d6d6dc3   habapp_habapp_1     2.40%     0B / 0B             0.00%     2.52MB / 340kB    0B / 0B       19

Edit:
Turns out I still had one script that was using an MqttItem.
Once I changed that to use my own directmqtt object, HABApp seems to starts up fine, without getting into the pegged CPU state.