Help Migrating External Python script to inside OH3

Setup, Configuration and Use Scripts & Rules
1

Hello,

I’m running a python script which needs pandas as a service on the same RasPi 4 that my OH3 installation is on.
I’d to bring this to inside the OH3 scripting environment for easier management.

The script may be interesting to many users, as I’m getting the departure times for public transit in Germany. So I’d like to also make a thread once I got it running.

I do not really know where to start, as I do not know if these libraries I need (requests,datetime,time,json,pandas) are possible to have in jython?

Of course I would not need the MQTT if I already have it inside OH3!

Is this correct?? I am a bit confused.

So any hints are very welcome

If anyone is interested, here is my python script:

#!/usr/bin/python3
import requests
import datetime
import time
import json
#from pandas import json_normalize 
from pandas import json_normalize
import pandas as pd
import paho.mqtt.client as mqtt

# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    #client.subscribe("$SYS/#")

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    print(msg.topic+" "+str(msg.payload))

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.username_pw_set("openhabian", password="YOURPW")

client.connect("openhab.local", 1883, 60)

def departure_times():
    # example
    #r=requests.get("http://gtfsr.vbn.de/api/routers/connect/plan?arriveBy=false&date=04-16-2020&fromPlace=53.062870,8.809780&toPlace=53.062870,8.78617&time=13:00:00&mode=WALK,TRANSIT&maxWalkDistance=300", headers={"Authorization":"YOURAUTHKEY","Host":"gtfsr.vbn.de"})
    #print(r.content)

    # OTP documentation
    # http://dev.opentripplanner.org/apidoc/1.4.0/resource_IndexAPI.html#resource_IndexAPI_getStoptimesForStop_GET
    # http://docs.opentripplanner.org/en/latest/Basic-Tutorial/

    # VBN
    # https://www.vbn.de/service/entwicklerinfos/

    #Bremen","stopId":"1:0000000001"
    #/otp/routers/{routerId}/index/stops/{stopId}/stoptimes
    #headers={"Authorization":"YOURAUTHKEY","Host":"gtfsr.vbn.de"})
    r=requests.get("http://gtfsr.vbn.de/api/routers/connect/index/stops/1:00000000001/stoptimes?detail=true&timeRange=7200&numberOfDepartures=20", headers={"Authorization":"YOURAUTHKEY"})

    # convert to json object
    json_obj = json.loads(r.content.decode('utf-8'))

    # normalize json
    df = json_normalize(json_obj,['times']) 
    
    # Filter out the unwanted departures
    df = df.query('headsign not in ["BSAG"]').reset_index()

    # get current time
    now = datetime.datetime.now()
    # get time difference to UTC
    diff = datetime.datetime.now() - datetime.datetime.utcnow()

    # create column for departure time as readable string
    df['departureTimeDT'] = pd.to_datetime(df['serviceDay']+df['realtimeDeparture'],unit='s')+diff

    # sort entries by realtime arrival time (provided in epoch format)
    df = df.sort_values(by=['departureTimeDT']).reset_index()

    # only keep next X departures
    # df = df.iloc[:3]

    print(df)

    # create column for departure time in HH:MM format
    df['depTimeHHMM'] = df['departureTimeDT'].dt.strftime('%H:%M')

    # create column for departure time in minutes from now
    df['depInMin'] = ((df['departureTimeDT'] - now).dt.seconds/60).astype(int)

    # drop unnecessary columns
    df = df[['headsign','departureTimeDT','depTimeHHMM','depInMin']]

    print(df)

    next3in1string = (str(df['depInMin'][0]) + "-" + str(df['depInMin'][1]) + "-" + str(df['depInMin'][2]))
    #print(next3in1string)

    dfout = df.to_json(orient='index',date_format='iso')

    #print(dfout)

    return dfout,next3in1string


client.loop_start()

while True:
    data,next3in1string = departure_times()
    client.publish("BSAG/next_tram_data",data)
    client.publish("BSAG/next_3_trams",next3in1string)
    time.sleep(1*60)
2

Jython is stuck at Python 2.7 only, so any Python3 specific stuff will have to be modified (if at all possible).

If you need to stick with Python 3, maybe try HabApp?

Or if you want to keep it all internal to OH, perhaps use the HTTP and MQTT bindings, together with a more simple Cron triggered rule?

1 Like
3

You may be able to use @Spaceman_Spiff 's HABApp which is Python 3 based.

4

It may be possible also with Python 2.7.

I can’t just use HTTP and MQTT Bindings, because I have to deserialize, then filter and sort the results.

I don’t know if there is a way with the existing rules/scripting to get this really bis JSON organized. If there are any examples, I’m happy to try! I’m not totally fixated on python, it just seemed the easiest at the time.

HABapp seems very interesting. I’ll give it a shot.

5

Actually ir is not straight CPython 2.7. OH uses Java based Jython.

6

Which you could do in a rule.

Both Python and JavaScript rules environments have robust JSON processing capabilities built into the languages by default.

7

First, you can possibly do almost all of this using the http binding in OH3, fetch the data, extract from json etc. I do this myself for my local public transport, where their API gives me exactly the next departure as a clock time.

Your main problem is the pandas dependency, which you need to drop if you want this inside OH3. Not having analyzed your script thoroughly, I would guess you could be well of with the http binding giving you a temporary result, and then take the rest through a simple (?) Jython snippet, redoing any necessary Pandas stuff manually.

If Pandas is critical for you, go for HABApp.

8

I need to look into that. I am not an experienced programmer, more the trial-and-error type and found the pandas library to be quite welcoming.

Any pointers what functions I could try?

9

Which language?

For JavaScript, remember that JSON stands for JavaScript Object Notation. The format was invented for use in JavaScript and it’s built into the language. There are tons and tons of examples of working with JSON in JavaScript. JavaScript JSON is one reference.

For Python you have the json library. 18.2. json — JSON encoder and decoder — Python 2.7.2 documentation