Performance and response time: Best practices?

Setup, Configuration and Use Runtime
1

I was wondering if there are any known best practices or places to look to find general performance issues + to be able to fix this? I am running a relatively big Openhab configuration (175 things, 951 items) and am facing issues that appear random, like:

  • Alexa commands have a massive delay one they, the next day they nearly work in real time
  • General lag or spontaneous outage of some add ons (like Telegram)
  • General delay in parsing rules and scripts

I am also running a gulp build script to “combine” a lot of rules of items in single files. Is/was this a good idea at all or is it recommended to serve as small (rule) files as possible?

Looking forward to hear your ideas/feedback.

2

That’s pretty average I think.

You might need to put the binding into debug mode but try to see where in the sequence of events the delay occurs. Is there delay between commanding the Item and the event appearing in the logs? Delay between when the event appears in the logs and Alexa picks it up. And so on.

Same as above.

What’s the hardware? OS? 32-or 64-bit? Rules language?

How are resources on the machine? CPU pegged? RAM used up? Swap being used?

This looks like your RAM is used up and you are using Swap.

1 Like
3

Thanks for you reply! Also good to know that my setup is not “that big”. I am currently running/using a dedicated Raspberry Pi 4 with 8GB of Ram (64 Bit). Right now the usage is the following:

###############################################################################
###############  openhabian  ##################################################
###############################################################################
##   Release = Debian GNU/Linux 11 (bullseye)
##    Kernel = Linux 6.1.21-v8+
##  Platform = Finished Check for Raspberry Pi EEPROM updates.
##    Uptime = 7 day(s). 16:48:46
## CPU Usage = 1% avg over 4 cpu(s) (4 core(s) x 1 socket(s))
##  CPU Load = 1m: 0.38, 5m: 0.46, 15m: 0.34
##    Memory = Free: 4.55GB (59%), Used: 3.12GB (41%), Total: 7.67GB
##      Swap = Free: 2.99GB (100%), Used: 0.00GB (0%), Total: 2.99GB
##      Root = Free: 104.17GB (92%), Used: 7.97GB (8%), Total: 116.94GB
##   Updates = 10 apt updates available.
##  Sessions = 1 session(s)
## Processes = 136 running processes of 4194304 maximum processes
###############################################################################

I also plan to upgrade to a Raspberry Pi 5 to speed up things. In general I don’t use the GUI and have everything written in Java. Should I switch to JS or why are you asking for the language?

4

Monitor the resources using top or htop if you can to see if there is a spike in anything when it takes Alexa a long time to reply.

Overall your resources look like they are unlikely to be the problem. But random delays are often a symptom of a machine under heavy load or low amounts of RAM.

Some rules languages have a known “first run” penalty which is present under certain circumstances but not others. For example, on an ARM processor running 32-bit Java the first run of a rule in JS Scripting, Blockly, or Rules DSL prior to OH 4.2 could take up to 30 seconds. But on the same hardware running 64-bit hardware that first run drops by 10X. And for rules that don’t run very often they can become garbage collected making such that the next run is like the first run, injecting random 30 second delays periodically.

If you are using 64-bit Java that’s unlikely to be the source of the delays regardless of the language used.

But to be certain, when you say “Java” you mean JRule from the marketplace?

5

I’m not familiar with openhabian, but I’m curious how you can have so much free memory. What’s the -Xmx (Maximum memory) setting for your java runtime?

I have 128GB RAM on my system, it runs openhab and many other things, and my top shows:

MiB Mem : 128760.6 total,    782.2 free,  46396.3 used,  81582.1 buff/cache

My java options: EXTRA_JAVA_OPTS: "-Xms16g -Xmx16g ....." - Note, this is extremely excessive. openHAB can work fine with 1-2GB of RAM available.
items: 1719, things: 141, rules: 218 (some of these are test items/things/rules)

OpenHAB will slow down when Java runs out of memory, so maybe look into increasing the -Xmx setting, especially since you have some free memory.

1 Like
6

Thank you for your thoughts, this sounds like something I should try. Will do this afap and come back with the results! :slight_smile:

@rlkoshak To be honst I don’t know how the way my rules are written is called. I started this many years ago when it was the only way i knew and ported it since then over all upcoming versions of OH. Here is an example. In general I tried to split rules in scripts. So nearly all rules are calling one ore many scripts interally. This was my try to keep things more abstract.

I also try to save results of calculations in items to react on their changed event in another rules. So there may be some chains with 3 or 4 iterations of those chains. Can this be also be a problem?

check_for_idle_rooms.rules:

rule "Run check_for_idle_rooms"
when
    Time cron "0 0/1 * * * ?" // Every minute
then
    callScript("check_for_idle_rooms")
end

check_for_idle_rooms.script:

logInfo("script", "check_for_idle_rooms");

val currentHour             = now.getHour();
val shortThresholdInMinutes = 5;
val thresholdInMinutes      = 30;
val nowInMilliseconds       = new DateTimeType().getZonedDateTime().toInstant.toEpochMilli;
val nowInSeconds            = nowInMilliseconds / 1000;
val nowInMinutes            = nowInSeconds / 60;

//
// Bathroom
//

val lastMotionBathroomInMilliseconds      = (Virtual_Bathroom_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionBathroomInSeconds           = lastMotionBathroomInMilliseconds / 1000;
val lastMotionBathroomInMinutes           = lastMotionBathroomInSeconds / 60;
var lastMotionBathroomDifferenceInMinutes = nowInMinutes - lastMotionBathroomInMinutes;

sendCommand(Virtual_Bathroom_LastMotion_WholeRoom_IdleTime, lastMotionBathroomDifferenceInMinutes);

logInfo("check_for_idle_rooms", "bathroom idle minutes: " + lastMotionBathroomDifferenceInMinutes);

if (
    lastMotionBathroomDifferenceInMinutes != NULL &&
    lastMotionBathroomDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_Bathroom_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Bathroom_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Bathroom_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Bathroom_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Bedroom
//

val lastMotionBedroomInMilliseconds      = (Virtual_Bedroom_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionBedroomInSeconds           = lastMotionBedroomInMilliseconds / 1000;
val lastMotionBedroomInMinutes           = lastMotionBedroomInSeconds / 60;
var lastMotionBedroomDifferenceInMinutes = nowInMinutes - lastMotionBedroomInMinutes;
var bedroomThresholdInMinutes            = thresholdInMinutes;

if (currentHour >= 0 && currentHour <= 6) {
    bedroomThresholdInMinutes = shortThresholdInMinutes;
}

sendCommand(Virtual_Bedroom_LastMotion_WholeRoom_IdleTime, lastMotionBedroomDifferenceInMinutes);

logInfo("check_for_idle_rooms", "bedroom idle minutes: " + lastMotionBedroomDifferenceInMinutes);

if (
    lastMotionBedroomDifferenceInMinutes != NULL &&
    lastMotionBedroomDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_Bedroom_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Bedroom_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Bedroom_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Bedroom_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Dining room
//

val lastMotionDiningRoomInMilliseconds      = (Virtual_DiningRoom_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionDiningRoomInSeconds           = lastMotionDiningRoomInMilliseconds / 1000;
val lastMotionDiningRoomInMinutes           = lastMotionDiningRoomInSeconds / 60;
var lastMotionDiningRoomDifferenceInMinutes = nowInMinutes - lastMotionDiningRoomInMinutes;

sendCommand(Virtual_DiningRoom_LastMotion_WholeRoom_IdleTime, lastMotionDiningRoomDifferenceInMinutes);

logInfo("check_for_idle_rooms", "dining room idle minutes: " + lastMotionDiningRoomDifferenceInMinutes);

if (
    lastMotionDiningRoomDifferenceInMinutes != NULL &&
    lastMotionDiningRoomDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_DiningRoom_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_DiningRoom_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_DiningRoom_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_DiningRoom_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// GuestBathroom
//

val lastMotionGuestBathroomInMilliseconds      = (Virtual_GuestBathroom_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionGuestBathroomInSeconds           = lastMotionGuestBathroomInMilliseconds / 1000;
val lastMotionGuestBathroomInMinutes           = lastMotionGuestBathroomInSeconds / 60;
var lastMotionGuestBathroomDifferenceInMinutes = nowInMinutes - lastMotionGuestBathroomInMinutes;

sendCommand(Virtual_GuestBathroom_LastMotion_WholeRoom_IdleTime, lastMotionGuestBathroomDifferenceInMinutes);

logInfo("check_for_idle_rooms", "guest bathroom idle minutes: " + lastMotionGuestBathroomDifferenceInMinutes);

if (
    lastMotionGuestBathroomDifferenceInMinutes != NULL &&
    lastMotionGuestBathroomDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_GuestBathroom_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_GuestBathroom_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_GuestBathroom_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_GuestBathroom_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Hallway
//

val lastMotionHallwayInMilliseconds      = (Virtual_Hallway_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionHallwayInSeconds           = lastMotionHallwayInMilliseconds / 1000;
val lastMotionHallwayInMinutes           = lastMotionHallwayInSeconds / 60;
var lastMotionHallwayDifferenceInMinutes = nowInMinutes - lastMotionHallwayInMinutes;

sendCommand(Virtual_Hallway_LastMotion_WholeRoom_IdleTime, lastMotionHallwayDifferenceInMinutes);

logInfo("check_for_idle_rooms", "hallway idle minutes: " + lastMotionHallwayDifferenceInMinutes);

if (
    lastMotionHallwayDifferenceInMinutes != NULL &&
    lastMotionHallwayDifferenceInMinutes > shortThresholdInMinutes
) {
    if (Virtual_Hallway_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Hallway_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Hallway_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Hallway_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Kitchen
//

val lastMotionKitchenInMilliseconds      = (Virtual_Kitchen_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionKitchenInSeconds           = lastMotionKitchenInMilliseconds / 1000;
val lastMotionKitchenInMinutes           = lastMotionKitchenInSeconds / 60;
var lastMotionKitchenDifferenceInMinutes = nowInMinutes - lastMotionKitchenInMinutes;

sendCommand(Virtual_Kitchen_LastMotion_WholeRoom_IdleTime, lastMotionKitchenDifferenceInMinutes);

logInfo("check_for_idle_rooms", "kitchen idle minutes: " + lastMotionKitchenDifferenceInMinutes);

if (
    lastMotionKitchenDifferenceInMinutes != NULL &&
    lastMotionKitchenDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_Kitchen_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Kitchen_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Kitchen_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Kitchen_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Living room
//

val lastMotionLivingRoomInMilliseconds      = (Virtual_LivingRoom_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionLivingRoomInSeconds           = lastMotionLivingRoomInMilliseconds / 1000;
val lastMotionLivingRoomInMinutes           = lastMotionLivingRoomInSeconds / 60;
var lastMotionLivingRoomDifferenceInMinutes = nowInMinutes - lastMotionLivingRoomInMinutes;

sendCommand(Virtual_LivingRoom_LastMotion_WholeRoom_IdleTime, lastMotionLivingRoomDifferenceInMinutes);

logInfo("check_for_idle_rooms", "living room idle minutes: " + lastMotionLivingRoomDifferenceInMinutes);

if (
    lastMotionLivingRoomDifferenceInMinutes != NULL &&
    lastMotionLivingRoomDifferenceInMinutes > thresholdInMinutes
) {
    if (
        Philips_LivingRoom_TV_OnWall_Power.state == OFF &&
        Virtual_LivingRoom_LastMotion_WholeRoom_IsIdle.state != ON
    ) {
        sendCommand(Virtual_LivingRoom_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_LivingRoom_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_LivingRoom_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Office
//

val lastMotionOfficeInMilliseconds      = (Virtual_Office_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionOfficeInSeconds           = lastMotionOfficeInMilliseconds / 1000;
val lastMotionOfficeInMinutes           = lastMotionOfficeInSeconds / 60;
var lastMotionOfficeDifferenceInMinutes = nowInMinutes - lastMotionOfficeInMinutes;

sendCommand(Virtual_Office_LastMotion_WholeRoom_IdleTime, lastMotionOfficeDifferenceInMinutes);

logInfo("check_for_idle_rooms", "office idle minutes: " + lastMotionOfficeDifferenceInMinutes);

if (
    lastMotionOfficeDifferenceInMinutes != NULL &&
    lastMotionOfficeDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_Office_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Office_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Office_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Office_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

//
// Pantry
//

val lastMotionPantryInMilliseconds      = (Virtual_Pantry_LastMotion_WholeRoom_DateTime.state as DateTimeType).getZonedDateTime().toInstant.toEpochMilli;
val lastMotionPantryInSeconds           = lastMotionPantryInMilliseconds / 1000;
val lastMotionPantryInMinutes           = lastMotionPantryInSeconds / 60;
var lastMotionPantryDifferenceInMinutes = nowInMinutes - lastMotionPantryInMinutes;

sendCommand(Virtual_Pantry_LastMotion_WholeRoom_IdleTime, lastMotionPantryDifferenceInMinutes);

logInfo("check_for_idle_rooms", "pantry idle minutes: " + lastMotionPantryDifferenceInMinutes);

if (
    lastMotionPantryDifferenceInMinutes != NULL &&
    lastMotionPantryDifferenceInMinutes > thresholdInMinutes
) {
    if (Virtual_Pantry_LastMotion_WholeRoom_IsIdle.state != ON) {
        sendCommand(Virtual_Pantry_LastMotion_WholeRoom_IsIdle, ON)
    }
} else {
    if (Virtual_Pantry_LastMotion_WholeRoom_IsIdle.state != OFF) {
        sendCommand(Virtual_Pantry_LastMotion_WholeRoom_IsIdle, OFF)
    }
}

For example somewhere else I listen on Virtual_Bathroom_LastMotion_WholeRoom_IsIdle again to turn off lights:

rule "Virtual_Bathroom_LastMotion_WholeRoom_IsIdle"
when
    Item Virtual_Bathroom_LastMotion_WholeRoom_IsIdle received command ON
then
    if (
        Hue_Virtual_LightZone_Bathroom_Power.state == ON &&
        Virtual_EveryWhere_Switch_Automation_Light.state == ON &&
        Virtual_Everywhere_LastActiveRoom_Room_Name.state != "bathroom"
    ) {
        logInfo("Virtual_Bathroom_LastMotion_WholeRoom_IsIdle", "bathroom: turning off lights since idle threshold reached");

        sendCommand(Hue_Virtual_LightZone_Bathroom_Power, OFF)
    }
end
7

You’re using RulesDSL. It is still a supported scripting engine for openHAB, however, nowadays there are better options that would make scripting easier and/or more powerful:

These three have the most developed support and helper library that makes scripting for openhab easier, compared to RulesDSL.

Take a look at conversions between rulesdsl, js, jython, and jruby

Your technique of “calling another script” can be done by creating a method / function, either within the same script, or by using a common include file.

This however, probably isn’t the cause of your performance issues. My bet is on the memory limit on java as I mentioned above.

8

That is probably hugely overkill to dedicate 16 GB of RAM to OH right out of the gate.

I’ve found The Doctor Binding helps find issues with your system [4.0.0;4.9.9] to be an easy way to track and understand how OH is using the heap. At a minimum I’d set the -Xms property much lower (maybe 4G) and let Java grab more ram as it needes it. That -Xms property causes Java to reserve that much RAM when it starts up whether or not it’s ever going to use it.

Unless jRuby is horribly ineffecient with RAM, which I highly doubt, you don’t have so many Items and Thing to require 16 GB of RAM to run in. But The Doctor addon can show you for sure.

For comparison I have 380 Items, 80 Things, and 74 rules and even with the default settings (which I think is 1g) I’m using less than 10% of the heap after garbage collection.

I mainly don’t want future readers of the forum to think that 16g is a normal amount of RAM to dedicate to OH.

That’s Rules DSL, not Java. It’s a custom programming language created by and for OH and, frankly, it has fallen behind in a number of ways. I do not recommend new development of rules using Rules DSL because of that. But that’s not the topic of this thread.

Knowing that you are using Rules DSL though does help analyze the performance though.

I’m less convinced but The Doctor add-on I linked to above should tell us if that’s the case. There is one Channel you can link to an Item and generate a chart of the heap usage over time. Perhaps there is an issue when garbage collection kicks in? We can correlate when that happens to when the delay to Alexa et al occurs. GC happened when the chart drops.

9

Good point! I have added a note in my post above.
I am aware that it’s far bigger than necessary/normal, and I deliberately set my -Xms to the same as -Xmx.

For @thomaskekeisen I’d suggest setting your -Xmx to 2g or 3g, or if you aren’t running anything else, go for 4g. What I don’t know is how openhabian sets it up, whether it has a hard coded memory limit or sets it dynamically based on the system memory. It’s good to know what the current setting is on your system, as a baseline before changing it.