Execution of python scripts from rules

Setup, Configuration and Use Scripts & Rules
1
  • Platform information:
    • Hardware: Raspberry Pi 3 Model B Plus Rev 1.3
    • OS: Raspbian GNU/Linux 10 (buster)
    • Java Runtime Environment: OpenJDK Runtime Environment Zulu11.50+19-CA (build 11.0.12+7-LTS)
    • openHAB version: 3.1.0 Stable

In rules i have:

var testmotion = executeCommandLine(Duration.ofSeconds(30), "python3", "/etc/openhab/scripts/motionsensor.py")
executeCommandLine(testmotion)

if I execute this motionsensor.py as openhab user:

sudo -u openhab python3 /etc/openhab/scripts/motionsensor.py

everything works. But when it gets executed from the rules, i get error that modules (picamera…) are not installed.

2021-10-29 11:21:19.067 [WARN ] [rg.openhab.core.io.net.exec.ExecUtil] - Error occurred when executing commandLine '[Traceback (most recent call last):

  File "/etc/openhab/scripts/motionsensor.py", line 10, in <module>
    from picamera import PiCamera
ModuleNotFoundError: No module named 'picamera'
]'

java.io.IOException: Cannot run program "Traceback (most recent call last):
  File "/etc/openhab/scripts/motionsensor.py", line 10, in <module>
    from picamera import PiCamera

ModuleNotFoundError: No module named 'picamera'
": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) ~[?:?]
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) ~[?:?]
	at org.openhab.core.io.net.exec.ExecUtil.executeCommandLine(ExecUtil.java:59) ~[?:?]
	at org.openhab.core.model.script.actions.Exec.executeCommandLine(Exec.java:40) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]

…

I guess it uses different python environments. I don’t know how to solve this since in script I have lines that try to install modules if they are not installed:

try:
    from picamera import PiCamera
except ImportError:
    print('Module picamera does not exist... Installing...')
    subprocess.call('pip3 install picamera', shell=True)
    print('Module picamera installed.')
    exit(1)

This is the same script that i had in OH 2.5 and it worked there.

Thanks for the help.

2

What are you expecting this to do? It will (attempt) to execute a script, and then use the results of that script as a pointer to some other script to execute.

Problems are usually to do with permissions, when running from console you are not the same user as the openHAB service.

3

Script just takes photos every second with PiCamera() module.
But I’m sure that script is going to work once I solve this import problem. Is there a way to change
which user runs executeCommandLine parameter. Or any other way to tell openHab service to run as shell maybe?

4

No.
Have you looked for other threads about executeCommandLine(), mentioning sudo perhaps?

5

First, please use code fences when posting logs, scripts and code. It is way easier to read.

There are icons across the top of the editor or

```
code goes here
```

As @rossko57 points out, you have two lines that are executing scripts.

var testmotion = executeCommandLine(Duration.ofSeconds(30), “python3”, “/etc/openhab/scripts/motionsensor.py”)

This line executes python3, passing it the full path motionsensor.py as an argument and it wait upto 30 seconds for the script to complete before timing out. The text output (both stdout and stderr) is captured and put into the variable testmotion

executeCommandLine(testmotion)

This line tries to execute the output from motionsensor.py as a command. It does not wait for results and it does not capture the output from the script.

So which one of these two executeCommandLine statements is failing?

What exactly is motionsensor.py returning? What exactly is the output of motionsensor.py, when executed as a command, returning? Add logging statements to find out. It looks like it is failing on the first one but we should not assume.

You didn’t mention it but can we assume you are not running openHAB in a container like Docker?

How was the PiCamera library installed. Just a plain pip3 or globally using sudo pip3? Or are you using an env?

No. openHAB runs as the openhab user which is a limited rights user. It can’t become some other user willy-nilly. That would defeat the whole point of running as a limited rights user. You may as well run openHAB as root, in which case yes you can run it as any user you want to since it’ll have root.

No but you can pass your command to bash though or put your command into a shell script and execute that from openHAB. That will give the command executed a shell.

6

Oh yes sorry for not formatting the correct way, it was my first post.
It looks better now :slight_smile:

Ok so if i only have:

var testmotion = executeCommandLine(Duration.ofSeconds(30), “python3”, “/etc/openhab/scripts/motionsensor.py”)

then I don’t see any output in logs.

And output of the

executeCommandLine(testmotion)

returns error from the first post.

I think that there is something wrong with the way that python3 is executed or maybe with permissions.
Because if i try:

val testmotion = executeCommandLine(Duration.ofSeconds(19), "python3", "/etc/openhab/scripts/motionsensor.py")
logInfo("test", testmotion)

I get this in the logs everytime script is executed.

==> /var/log/openhab/openhab.log <==

2021-10-29 18:43:16.306 [INFO ] [org.openhab.core.model.script.test  ] - Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple

Requirement already satisfied: picamera in ./.local/lib/python3.7/site-packages (1.13)

and if i change try and except for loading module to just

from picamera import PiCamera

then everytime script is executed i get:

==> /var/log/openhab/openhab.log <==

2021-10-29 18:45:26.063 [INFO ] [org.openhab.core.model.script.test  ] - Traceback (most recent call last):

  File "/etc/openhab/scripts/motionsensor.py", line 10, in <module>

    from picamera import PiCamera

ModuleNotFoundError: No module named 'picamera'

picamera module is installed with pip3

root@openhab:/etc/openhab/scripts# sudo -u openhab pip3 install picamera
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Requirement already satisfied: picamera in /var/lib/openhab/.local/lib/python3.7/site-packages (1.13)

This is a very strange thing, because it seems like that picamera module is already installed for openhab user (which it is since it works ok if i run it as openhab user from shell). But on the other hand it cannot get loaded because there is no picamera module installed ?!?!?!’

So ok, I made a shell script that executes:

#!/bin/bash

whoami
/usr/bin/python3 /etc/openhab/scripts/motionsensor.py

and every time a get the same error in logs:

==> /var/log/openhab/openhab.log <==

2021-10-29 19:14:45.102 [WARN ] [rg.openhab.core.io.net.exec.ExecUtil] - Error occurred when executing commandLine '[+ whoami

openhab

+ /usr/bin/python3 /etc/openhab/scripts/motionsensor.py

* failed to open vchiq instance

]'

java.io.IOException: Cannot run program "+ whoami

openhab

+ /usr/bin/python3 /etc/openhab/scripts/motionsensor.py

* failed to open vchiq instance

": error=2, No such file or directory

	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1128) ~[?:?]

	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1071) ~[?:?]

	at org.openhab.core.io.net.exec.ExecUtil.executeCommandLine(ExecUtil.java:59) ~[?:?]

	at org.openhab.core.model.script.actions.Exec.executeCommandLine(Exec.java:40) ~[?:?]

	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]

	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]

	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1192) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeOperation(XbaseInterpreter.java:1167) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._invokeFeature(XbaseInterpreter.java:1153) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.invokeFeature(XbaseInterpreter.java:1098) ~[?:?]

	at org.openhab.core.model.script.interpreter.ScriptInterpreter.invokeFeature(ScriptInterpreter.java:151) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:1008) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:971) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:247) ~[?:?]

	at org.openhab.core.model.script.interpreter.ScriptInterpreter.doEvaluate(ScriptInterpreter.java:226) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:227) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter._doEvaluate(XbaseInterpreter.java:475) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.doEvaluate(XbaseInterpreter.java:251) ~[?:?]

	at org.openhab.core.model.script.interpreter.ScriptInterpreter.doEvaluate(ScriptInterpreter.java:226) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.internalEvaluate(XbaseInterpreter.java:227) ~[?:?]

	at org.eclipse.xtext.xbase.interpreter.impl.XbaseInterpreter.evaluate(XbaseInterpreter.java:213) ~[?:?]

	at org.openhab.core.model.script.runtime.internal.engine.ScriptImpl.execute(ScriptImpl.java:80) ~[?:?]

	at org.openhab.core.model.script.runtime.internal.engine.DSLScriptEngine.eval(DSLScriptEngine.java:131) ~[?:?]

	at org.openhab.core.automation.module.script.internal.handler.ScriptActionHandler.lambda$0(ScriptActionHandler.java:62) ~[?:?]

	at java.util.Optional.ifPresent(Optional.java:183) [?:?]

	at org.openhab.core.automation.module.script.internal.handler.ScriptActionHandler.execute(ScriptActionHandler.java:59) [bundleFile:?]

	at org.openhab.core.automation.internal.RuleEngineImpl.executeActions(RuleEngineImpl.java:1183) [bundleFile:?]

	at org.openhab.core.automation.internal.RuleEngineImpl.runRule(RuleEngineImpl.java:991) [bundleFile:?]

	at org.openhab.core.automation.internal.TriggerHandlerCallbackImpl$TriggerData.run(TriggerHandlerCallbackImpl.java:90) [bundleFile:?]

	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]

	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]

	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]

	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]

	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]

	at java.lang.Thread.run(Thread.java:829) [?:?]

Caused by: java.io.IOException: error=2, No such file or directory

	at java.lang.ProcessImpl.forkAndExec(Native Method) ~[?:?]

	at java.lang.ProcessImpl.<init>(ProcessImpl.java:340) ~[?:?]

	at java.lang.ProcessImpl.start(ProcessImpl.java:271) ~[?:?]

	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1107) ~[?:?]

	... 36 more

First i get * failed to open vchiq instance error which means openhab user is not in video group, but it is, because from shell it works ok and i have checked.

video:x:44:openhabian,openhab

And this error is also strange because it tells me that “rg.openhab.core.io.net.exec.ExecUtil” and java cannot run program “whoami” but then returns openhab, which is correct how it should be.

java.io.IOException: Cannot run program "+ whoami

I bet there is something wring with permissions or with my installation of openhab.
Openhab is installed on RPI 3 with image file openhabian-pi-raspios32-v1.6.6.img.xz.
No docker container or anything.

7

Oh and this is my whole motionstatus.py python script.

#!/usr/bin/python3

import datetime
import time
import os
import sys
import subprocess
from threading import Thread
#from picamera import PiCamera

#Config = ConfigParser.ConfigParser()
try:
    from picamera import PiCamera
except ImportError:
    print('Module picamera does not exist... Installing...')
    subprocess.call('pip3 install picamera', shell=True)
    print('Module picamera installed.')
    exit(1)

try:
    import paho.mqtt.client as mqtt
except ImportError:
    print('Module paho-mqtt does not exist... Installing...')
    subprocess.call('pip3 install paho-mqtt', shell=True)
    print('Module paho-mqtt installed.')
    exit(1)

def on_connect(client, userdata, flags, rc):
    client.subscribe('house/motion/imagename')

client = mqtt.Client()
client.on_connect = on_connect
client.connect('192.168.0.1', 1883, 60)

try:
    from settings import *
except ImportError:
    print("ERROR : Could Not Import settings.py check if it exists.")
    exit(1)

if not os.path.exists(CaptureFolder):
    try:
        print(("INFO: Creating {0} folder").format(CaptureFolder))
        os.makedirs(CaptureFolder)
    except OSError:
        print(("ERROR : Could Not Create Folder {0}").format(CaptureFolder))
        exit(1)

# Camera Settings
camera = PiCamera()
camera.resolution = (ResolutionWidth, ResolutionHeight)
camera.exposure_mode = CameraExposure
camera.awb_mode = CameraAWBmode

# Delete files to free up some space
def keepDiskSpaceFree(bytesToReserve):
    try:
        if (getFreeSpace() < bytesToReserve):
            now = time.time()
            only_files = []
            subprocess.call('telegram-send "Deleted files to avoid filling disk"', shell=True)

            for file in sorted(os.listdir(CaptureFolder + '/')):
                file_full_path = os.path.join(CaptureFolder + '/',file)
                if os.path.isfile(file_full_path) and file.endswith(file_ends_with):
                    #Delete files older than x days
                    if os.stat(file_full_path).st_mtime < now - how_many_days_old_images_to_remove * 86400: 
                        os.remove(file_full_path)
                        print("File Removed: ", file)
                        if (getFreeSpace() > bytesToReserve):
                            return
    except:
        subprocess.call('telegram-send "Cannot delete old files, disk is filling up, motion capture still working."', shell=True)        

# Get available disk space
def getFreeSpace():
    st = os.statvfs(CaptureFolder + "/")
    du = st.f_bavail * st.f_frsize
    return du

# def syncFilesToSrv():
#     subprocess.call('telegram-send "File sync started."', shell=True)
#     subprocess.call(['./rclone-sync.sh'], shell=True)
#     subprocess.call('telegram-send "File sync complete!"', shell=True)


# def SendImageToTelegram():
#     subprocess.call('telegram-send -i {0}'.format(ImageToSend), shell=True)


def startCapture():
    camera.start_preview()
    print('\nMovement detected from PIR sensor starting Image Capture')
    try:
        for i in range(CaptureNumImages):
            fileDate = time.strftime(CaptureFileDateTimeFormat)
            camera.capture('{0}/{1}-{2}.jpg'.format(CaptureFolder, CaptureFileName, fileDate))
            #print ('Capturing image {0} - file: {1}-{2}.jpg'.format(i, CaptureFileName, fileDate))
            time.sleep(1)
            if i == 0:
                #global ImageToSend
                #ImageToSend = '{0}/{1}-{2}.jpg'.format(CaptureFolder, CaptureFileName, fileDate)
                mqttMsg = '{0}-{1}.jpg'.format(CaptureFileName, fileDate)
                ret = client.publish(topic, mqttMsg)
    finally:
        camera.stop_preview()
        #SendImageToTelegram()

if __name__ == '__main__':
    NowDateTime = time.strftime("%d-%m-%Y %H:%M:%S")
    print(("INFO: Image capture start time: {0}".format(NowDateTime)))
    try:
        startCapture()
        thrDeleteFiles = Thread(target=keepDiskSpaceFree(diskSpaceToReserve))
        thrDeleteFiles.start()
        thrDeleteFiles.join()
    except KeyboardInterrupt:
        print("\nINFO: CTRL + C detected from user. Quitting...")

But it is the same if i only use test.py with only this code:

import subprocess
subprocess.call('/usr/bin/pip3 install picamera', shell=True)
from picamera import PiCamera
8

You wouldn’t. You’re not logging anything in your rule and the command is completing successfully.

No, that’s not the output of the command.

To get the actual full output of the command you need to run it just like the first executeCommandLine. Add a Duration as the first argument and set the result to a variable. Then log it out.

While you are at it, also log out testmotion too so you know what command that second executeCommandLine is actually trying to run.

The error log is only going to show you stderr. There is likely lots of useful information in stdout too and you are completely throwing that away.

Honestly, how do you know? You have no idea what testmotion is so you have no idea what command that second executeCommandLine is trying to run.

You need to work through this deliberately and step-by-step. Don’t assume anything. Verify everything. Start at the beginning and verify each and every step is working as expected. That means log out testmotion so you know exactly what command the second executeCommandLine is running. Capture the full output from the second executeCommandLine and log that out.

At this point you don’t even know what command is being executed. It’s far too early to start jumping at what the actual problem is.

9

Ok so I did some logging from python. And imports failed because modules have to be installed with root and not with openhab user. This is different than in OH2.5. Also the “* failed to open vchiq instance error” failed again because I had to reboot after adding user to the group.
I’m going to try to figure out why modules have to be installed as root. Python 2 and 3 was already installed with the image.

But now at least everything is working.

Thanks for the help.

10

I guess what happened is that you created the script with a different user as user “openhab”.

Did you set the permissions correctly?
e.g.

sudo chmod 777 /etc/openhab/scripts/motionsensor.py
11

Permissions are:

-rwxr-xr-x  1 openhabian openhab    4175 Nov  1 09:06 motionsensor.py

But it is the same with 777. Also if is set chown openhab.openhab …

12

if you want to execute a bash script from openhab, you have to add this script to sudoers first.

Anyway, I would suggest to simplify everything and get a working structure.
Change your rule to this:

val testmotion = executeCommandLine(Duration.ofSeconds(19), "python3", "/etc/openhab/scripts/motionsensor.py")
logInfo("test", testmotion)

And change motionsensor.py to this:

Print(„Message from py-script“)

if you see the message in openhab log, add one step after the other

2 Likes