Cleaning up the startup process / renaming rules (windows possible)

startup
openhab2
openhabian
Tags: #<Tag:0x00007fe060044618> #<Tag:0x00007fe060045130> #<Tag:0x00007fe06004bdf0>

(Joachim Boeddeker) #1

I’ve run in some issues with the current startup process. Everything is starting at once which leads to several exceptions, errors and other problems. In my case the most annoying problem is the Homematic-Channel not found-problem.

According to my opinion, a reason is the triggering of rules during startup which leads to timing problems and other race conditions which then lead to further problems after startup.

I often read about moving or renaming the rules and copying back after startup, so the idea is not from me. But i never saw a solution which does all the stuff in the background, without manual interference, once it has been installed.

I know, this is rather a workaround than a solution and it also just works for a linux installation, but it may help some people possibly. If not, it helps me. :wink:

In the discussion some alternatives emerged. All alternatives work similar and share the same bash-script.
I hope, this helps somebody. If you have any further questions, feel free to ask.

My prefered variant

This is the most simple alternative, but just works with linux and systemd (e.g. openhabian). Thanks to @ptweety for this variant.

Simply create the file `/etc/systemd/system/openhab2.service.d/override.conf. This overrides the standard startup script where applicable.

If you are on stretch-based openhabian, you may use sudo systemctl edit openhab2.service, otherwise you have to use sudo nano /etc/systemd/system/openhab2.service.d/override.conf

[Service]
ExecStartPre=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.rules" -exec /usr/bin/rename.ul .rules .rules_away {} \\;'
ExecStartPre=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.script" -exec /usr/bin/rename.ul .script .script_away {} \\;'
ExecStartPost=/bin/sleep 240
ExecStartPost=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.script_away" -exec /usr/bin/rename.ul .script_away .script {} \\;'
ExecStartPost=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.rules_away" -exec /usr/bin/rename.ul .rules_away .rules {} \\;'
TimeoutStartSec=360

That’s it.

Common for all other alternatives

The bash script (all my bash scripts are located in /etc/openhab2/exec-scripts, if you want a different storage make sure you change all references as well):

“/etc/openhab2/exec-scripts/moverules.sh”

#!/bin/bash

ORG=$1
NEW=$2
IGNORE=moverules.rules

for f in /etc/openhab2/rules/*.${ORG};
do
    CURRENT=$(basename $f)
    if [ "$CURRENT" == "$IGNORE" ]    
    then
    	echo "ignoring $IGNORE"
    else
        OLDFILE=$f
        NEWFILE=${f%$ORG}$NEW
        mv "$OLDFILE" "$NEWFILE"
    fi
done

I also translated the bash script to windows batch.

“C:\openHAB2\conf\exec-scripts\moverules.cmd”

@echo off

SET ORG=%1
SET NEW=%2
SET IGNORE="moverules.rules"

cd C:\openHAB2\conf\rules
for %%f in (*.%ORG%) do ( 
    if %%f NEQ %IGNORE% (
       echo  %%f %%~nf.%NEW%
    )
)

Alternative 1: Two rules, no servicefile

One rule renames the rules to *.rules at startup + 240 seconds, one rule renames the rules to *.rules_away at shutdown. This variant is able to run under Windows.
Please be aware, this is untested, i do not run openHAB on windows.

Pros: no changes to service file, no need to reapply the patch after update, able to run on Windows
Cons: rules are not named rules while the service is not running, system needs one restart after a power loss or installation

Changes:

  • a rules file which calls the bash script (/etc/openhab2/rules/moverules.rules)

The single new rules file (moverules.rules):

val Integer secondsDelay = 240
val String executeCmdBack = "/etc/openhab2/exec-scripts/moverules.sh rules_away rules"
val String executeCmdAway = "/etc/openhab2/exec-scripts/moverules.sh rules rules_away"
// exchange above for that on windows
//val String executeCmdBack = "C:\openHAB2\conf\exec-scripts\moverules.cmd@@rules_away@@rules"
//val String executeCmdAway = "C:\openHAB2\conf\exec-scripts\moverules.cmd@@rules@@rules_away"


var Timer renameTimer = null

rule "rename rules back in"
when 
    System started
then    
    logInfo("renameRules", "system started triggered")

    if (renameTimer === null) 
    {
        renameTimer = createTimer(now.plusSeconds(secondsDelay)) 
                            [|
                                logInfo("renameRules", "rename rules back in")
                                executeCommandLine(executeCmdBack)
                            ]
        logInfo("renameRules", "timer created")
    }
    else 
    {
        logInfo("renameRules", "timer already existing")
    } 
end 	

rule "rename rules away"
when 
    System shuts down
then 
    logInfo("renameRules", "rename rules away")
    executeCommandLine(executeCmdAway)
end    

Alternative 2: Servicefile and one rule

Pros: rules are correctly named at all times, except for startup
Cons: The patch to the service file has to be done after each update to openHAB, just works on systemd enabled linux variants

Changes:

  • a rules file which calls the bash script (/etc/openhab2/rules/moverules.rules)
  • an addition to the service definition (/usr/lib/systemd/system/openhab2.service)

The single new rules file (moverules.rules):

val Integer secondsDelay = 240

var Timer renameTimer = null

rule "rename rules back in"
when 
    System started
then    
    logInfo("renameRules", "system started triggered")

    if (renameTimer === null) 
    {
        renameTimer = createTimer(now.plusSeconds(secondsDelay)) 
                            [|
                                logInfo("renameRules", "rename rules back in")
                                val String cmd = "/etc/openhab2/exec-scripts/moverules.sh rules_away rules"
                                executeCommandLine(cmd)
                            ]
        logInfo("renameRules", "timer created")
    }
    else 
    {
        logInfo("renameRules", "timer already existing")
    } 
end 	

The addition to the service file (in openhabian this is /usr/lib/systemd/system/openhab2.service )
Just add the line:

...
ExecStartPre=/etc/openhab2/exec-scripts/moverules.sh rules rules_away
...

The complete service file

[Unit]
Description=openHAB 2 - empowering the smart home
Documentation=http://docs.openhab.org
Documentation=https://community.openhab.org
Wants=network-online.target
After=network-online.target

[Service]
Environment=OPENHAB_HOME=/usr/share/openhab2
Environment=OPENHAB_CONF=/etc/openhab2
Environment=OPENHAB_RUNTIME=/usr/share/openhab2/runtime
Environment=OPENHAB_USERDATA=/var/lib/openhab2
Environment=OPENHAB_LOGDIR=/var/log/openhab2
Environment=OPENHAB_STARTMODE=daemon
EnvironmentFile=-/etc/default/openhab2

User=openhab
Group=openhab

WorkingDirectory=/usr/share/openhab2
ExecStartPre=/etc/openhab2/exec-scripts/moverules.sh rules rules_away
ExecStart=/usr/share/openhab2/runtime/bin/karaf $OPENHAB_STARTMODE
ExecStop=/usr/share/openhab2/runtime/bin/karaf stop

SuccessExitStatus=0 143
RestartSec=5
Restart=on-failure
TimeoutStopSec=120

LimitNOFILE=102642

[Install]
WantedBy=multi-user.target

Alternative 3: Servicefile, no rules

Pros: rules are correctly named at all times, except for startup, no rules involved
Cons: The patch to the service file has to be done after each update to openHAB, just works on systemd enabled linux variants

The addition to the service file (in openhabian this is /usr/lib/systemd/system/openhab2.service )
Just add the lines:

...
ExecStartPre=/etc/openhab2/exec-scripts/moverules.sh rules rules_away
...
ExecStartPost=/bin/sleep 240
ExecStartPost=/etc/openhab2/exec-scripts/moverules.sh rules_away rules
...

The complete service file

[Unit]
Description=openHAB 2 - empowering the smart home
Documentation=http://docs.openhab.org
Documentation=https://community.openhab.org
Wants=network-online.target
After=network-online.target

[Service]
Environment=OPENHAB_HOME=/usr/share/openhab2
Environment=OPENHAB_CONF=/etc/openhab2
Environment=OPENHAB_RUNTIME=/usr/share/openhab2/runtime
Environment=OPENHAB_USERDATA=/var/lib/openhab2
Environment=OPENHAB_LOGDIR=/var/log/openhab2
Environment=OPENHAB_STARTMODE=daemon
EnvironmentFile=-/etc/default/openhab2

User=openhab
Group=openhab

WorkingDirectory=/usr/share/openhab2
ExecStartPre=/etc/openhab2/exec-scripts/moverules.sh rules rules_away
ExecStart=/usr/share/openhab2/runtime/bin/karaf $OPENHAB_STARTMODE
ExecStop=/usr/share/openhab2/runtime/bin/karaf stop
ExecStartPost=/bin/sleep 240
ExecStartPost=/etc/openhab2/exec-scripts/moverules.sh rules_away rules

SuccessExitStatus=0 143
RestartSec=5
Restart=on-failure
TimeoutStopSec=120

LimitNOFILE=102642

[Install]
WantedBy=multi-user.target


Disappointed with Openhab 2.2
RPI 3+ turn slow since openhab 2.1
Startup Issue
Homematic Binding: "Channel not found for datapoint" errors for definitely existing channels
Massive Problems after a reboot
[SOLVED] Rule problems after powerON
[SOLVED] Issues in rules after upgrading to #1518
[SOLVED] Very strange error during startup of openhab2
Odd behavior using ReentrantLock
Homematic Binding: "Channel not found for datapoint" errors for definitely existing channels
MQTT publish action not available at startup
Openhab works slow, CPU of the RPI is high
(Matthias) #2

Hi @job,

I wonder whether it’s an improvement to your solution if you just use something like this (not tested):

ExecStartPre=/usr/bin/rename.ul .rules .x $OPENHAB_CONF/rules/*.rules
ExecStart=/usr/share/openhab2/runtime/bin/karaf $OPENHAB_STARTMODE
ExecStartPost=/bin/sleep 1m
ExecStartPost=/usr/bin/rename.ul .x .rules $OPENHAB_CONF/rules/*.x

(Joachim Boeddeker) #3

Hi @ptweety,

If it works, it is of course simpler (=better). Just one file needs to be changed. :wink:

But i am unsure, if the StartPosts are started sequentially or in parallel, and if the sequence is defined strictly. I have no idea.

Additionally, the delay could be a configuration parameter.

This should be tested.

Thanks!


(HomeAutomation) #4

Is there a shutdown rule tirgger available to rename it back at shutdown?

rule "rename rules at shut down"
when 
    something like
    System stopped
then

(Rossko57) #5

I don’t think renaming at shutdown would help. The underlying issues seem to be around threads and queuing at startup; the renaming trick makes system read the files again after the startup chaos has calmed done a bit?


(Markus Storm) #6

Works fine for me. Thanks! I, too, have been wanting to implement that for ages, have just been to lazy. Sorry :slight_smile:


(Markus Storm) #7

Just upgraded to latest snapshot and noticed this line went missing, apparently openhab2.service gets overwritten.
So we should think of a different method of ‘anchoring’ the move script.


(Joachim Boeddeker) #8

Added 2 variants of the process. If you used the prior version, please have in mind, that i changed the suffix for unused rules from “rulesx” to “rules_away”.


(Matthias) #9

Hi @job,

I like to come back an this topic with my current solution. Now it’s also tested and at least working for me :wink:

But first let me recommend to not alter the upstream version of the openhab2.service file. Instead I would go with an override of the same. One can do this with

sudo systemctl edit openhab2.service

That will create a new file /etc/systemd/system/openhab2.service.d/override.conf with all the content you like. This file will survive also an upgrade of openhab to the next version.

To undo such a change you can also activate the original state with

sudo systemctl revert openhab2.service

The content of the file may e.g. be (according Alternative 3 from the first post):

[Service]
ExecStartPre=/etc/openhab2/exec-scripts/moverules.sh rules rules_away
ExecStartPost=/bin/sleep 240
ExecStartPost=/etc/openhab2/exec-scripts/moverules.sh rules_away rules

Next we want to

sudo systemctl daemon-reload

Now, another alternative is to go with this override to the openhab2.service

[Service]
ExecStartPre=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.rules" -exec /usr/bin/rename.ul .rules .x {} \\;'
ExecStartPost=-/bin/sleep 90
ExecStartPost=-/bin/bash -c '/usr/bin/find ${OPENHAB_CONF} -name "*.x" -exec /usr/bin/rename.ul .x .rules {} \\;'
TimeoutStartSec=180

Well, TimeoutStartSec is optional, but I found it useful to prevent the openhab2.service to go into failed-state due to long startup times. I’ve set this value to the duration of the sleep (90 seconds) plus another 90 seconds.


Sometimes rules don't get executed?
System Started - What state is OH in when this fires?
Start up ver slow, can I adjust startup progress,
Startup Rule is triggered/fires multiple times not only on Start
(Joachim Boeddeker) #10

:+1:

Nice, @ptweety.

That’s a great approach. Your approach will replace Alternative 2+3, if you don’t mind.

No rules, no shell schript, just a small override to the service configuration. Great!

Thanks.


(Markus Storm) #11

Nice approach but seems it’s not applicable everywhere. I’m running a Jessie based openHABian which I believe is the most common Linux variant to find among OH users.

[14:25:55] root@openHABianPi:/home/pi# systemctl list-unit-files openhab2.service
UNIT FILE        STATE
openhab2.service enabled

1 unit files listed.
[14:25:58] root@openHABianPi:/home/pi# systemctl edit  openhab2.service
Unknown operation 'edit'.
[14:26:08] root@openHABianPi:/home/pi#

(Matthias) #12

Hi @mstormi,

with debian jessie you have an outdated version of systemd that does not support systemctl edit. But you can create the folders for your services and your own override.conf file by yourself.

sudo mkdir /etc/systemd/system/openhab2.service.d
sudo nano /etc/systemd/system/openhab2.service.d/override.conf

Rough start (lots of errors) on OpenHab 2.3 Release
(Ray) #13

Thanks @job this certainly makes startup much faster and more stable.

As a side it also meant I could use a new naming convention on my rule files so that I could control loading order as well. Just suffixed everything with a numeric like:

nnn-myrule.rules

Also added a little sleep (1 second for now) in between each ‘mv’ (or in my case ‘ln’) so OH has a little breather as it picks up each file. Still playing with whether it is worth it and how long to sleep but just felt it was better than dumping the whole ‘rules’ loading on it at once.


(Markus Storm) #14

There of course is the big point how to process items files before processing the rules which is what all of this thread is about, but what’s the point in introducing a processing order inside the set of rules files?


(Ray) #15

I don’t think it’s absolutely essential but I do have certain rule files that I consider more low level than others. Maybe generic helpers like item synchronisation, MQTT messaging for group state changes (can’t get the binding to do that for me), alert and debug rules etc so I’d rather load those early on.

Also the likes of Astro rules trigger a lot events in other rules ‘on change’ so why not just ensure it is loaded early updated and then the other startup rules will initialise themselves from the latest value rather than initialise and then change seconds later.

So maybe more preference and optimisation than a requirement :slight_smile:


(marbon87) #16

If anyone is looking for a solution in combination with the official docker-image, you can use a ramdisk (option in docker) and create a custom entrypoint, that copies all files to that ramdisk before the rules above renames them.


(E. Gerland) #17

I am considering to implement your workaround (alternative 1).
Two question though.

  1. The 240 seconds timer (waiting time) is pretty system depended, I guess.
    So how to know when the system is ready for activating the rules?
  2. I have a “system started” rule to initialize some items during startup in place already.
    WIll this still work?

(Joachim Boeddeker) #18
  1. yes, it is system dependent. Watch your log. You will see. At some time no bindings, things or services are being initialized, it’s just simple item-changes. (Be aware, all changes from NULL to anything are persistence initializing)
  2. you need to put this rule in the moverules.rule file, then it will work. Otherwise not.

Alternative 1 needs one shutdown to become functional.


(E. Gerland) #19

Alright - thanks a lot.


(Michael Cumming) #20

This is no longer on problem using the current release of OpenHABian