[JSR223] [JYTHON] Jython with requests successful?

Do you have a helper class to deal with curl / executeCommandLine? If none exists, I might write one. It would be super simple, but it would avoid code duplications. I’m thinking something like

curl = Curl("https://www.something.com")
curl.add_header(....)
curl.set_params(....)
curl.add_data(...) to be called multiple times - for POST), or curl.set_data(pass the data in a list)
curl.get() or curl.post()

In the background it will deal with executeCommandLine’s weird “@@” thing :smiley:

Is the requests error only within jython? On my system i get an error using requests within jython. If i use pur python it works like a charm.

To day i opend an issue at jythons github page

Wow, I’ve never heard of HABApp. Took a quick look at it. It seems like a viable solution. Must investigate more.

1 Like

Correct. See Post #9

I doubt if they can do much, but I’d be curious to know. It would take a while to trickle down to OpenHAB too, so it’s a very very long shot. For the time being curl works perfectly for me. Just need to write a helper class for it to make it nicer to use.

No… I’m familiar enough with curl and OH’s requirements that I haven’t needed one.

If there were to be a fix in Jython, it would be immediately available in OH. You can manually install whatever version of Jython you’d like. I’ve experienced errors causing rules not to run in 2.7.1 and 2.7.2b2. Oddly, the 2.7.2a1+ build that I made never had an issue.

1 Like

I can confirm, the Java HTTP libraries support them. You can see examples (Rules DSL but the Java will still be the same) at:

You could also use executeCommandLine or subprocess (subprocess works a bit better in my experience) to call curl as a work around. Here is an example using subprocess and wget (I’m just grabbing an icon).

from core.rules import rule
from core.triggers import when
from configuration import weather_icon_path
import subprocess
from javax.imageio import ImageIO
from java.io import File

@rule("Weather Icon",
      description="Copy the current weather conditions icon",
      tags=["weather"])
@when("Item vWeather_Conditions_Icon changed")
@when("System started")
def cond_icon(event):
    """
    Download the weather conditions icon from DarkSky and convert it from gif to
    png.
    """
    # OpenWeatherMap
#    cond_icon.log.info("Fetching weather conditions icon to {} from {}"
#                  .format(weather_icon_path, items["vWeather_Conditions_Icon"]))
#
#    results = subprocess.check_output(['/usr/bin/wget', '-q', '-O',
#                                       weather_icon_path,
#                                       str(items["vWeather_Conditions_Icon"])])
#
#    input_file  = File(weather_icon_path)
#    if not input_file.exists():
#        cond_icon.log.warn("Failed to fetch the weather icon!")
#        return
#
#    output_file = File(weather_icon_path.replace('gif', 'png'))
#    ImageIO.write( ImageIO.read(input_file), 'png', output_file)
#
#    results = subprocess.check_output(['rm', weather_icon_path])

    # DarkSky
    cond_icon.log.info("Fetching the weather conditions icon... {}".format(ir.getItem("vWeather_Conditions_Icon").state))
    dl = subprocess.Popen(['/usr/bin/wget', '-qO-',
                  'http://argus:8080/rest/items/vWeather_Conditions_Icon/state'],
                  stdout=subprocess.PIPE)
    dd = subprocess.Popen(['/bin/dd', 'bs=22', 'skip=1'], stdin=dl.stdout, stdout=subprocess.PIPE)
    dl.wait()
    f = open(weather_icon_path, "w")
    subprocess.call(['/usr/bin/base64', '-d'], stdout=f, stdin=dd.stdout)
    dd.wait()

Note, the ImageIO class I use to convert the gif to png is a Java class.

If you make a helper I strongly recommend using subprocess instead of executeCommandLine. It can be quite challenging if not impossible to get executeCommandLine to work sometimes. The way the arguments are pass as an array or tuple in subprocess helps avoid the whole @@ nonsense. In fact, if you look at the simple command I try to run in the commented out OpenWeatherMap example above, I never was able to get that simple wget to run using executeCommandLine. I used subprocess out of desperation and was very happy with how easy it was to use.

The trouble with subprocess is that it does not have a built-in timeout. Early on when I was still getting an understanding of the HLs, I submitted a PR that used subprocess, but cancelled the PR since executeCommandLine worked just fine.

Rather than wrap executeCommandLine or subprocess, we should just make executeCommandLine easier to use or add the functionality to the HTTTP Action. It is really easy to test things out in scripted automation, but we’ll have a mess if we don’t put the functionality back into OH.

But that’s the problem. executeCommandLine doesn’t just work fine. Sometimes you need @@, sometimes you don’t. Sometimes even with the @@ it doesn’t work. Take that simple wget I have above. I tried about 30 different combinations over about two hours trying to get it to work with executeCommandLine. None of them worked. The only way I could get it to work was to put the wget into a script or call subprocess.

Perhaps bringing in subprocess32 would be a solution since it does support a timeout.

I agree, ultimately fixing executeCommandLine would be better, but right now I assert that executeCommandLine is fundamentally broken. What ever was done in it to handle the arguments before the command get’s passed to the Java libraries that execute the command only works sometimes. I would not try to base anything on it unless and until it get’s fixed.

And it’s worth noting that I have some preliminary suspicions that the timeout argument passed to executeCommandLine is not always followed either. I don’t have enough yet to file an issue though.

As for fixing the HTTP Actions, I assumed, perhaps incorrectly, that the 2.x version of the binding would implement replacements for those Actions, similar to how MQTT replaced publish.

Hmmm… I didn’t have any problems with this, unless I left out the timeout…

from core.actions import Exec

test = Exec.executeCommandLine("/usr/bin/wget -qO- https://ci.openhab.org/job/openHAB2-Bundles/lastSuccessfulBuild/console", 10000)

That was one of the variations that I tried. I don’t remember the exact behavior I saw with that specific variation but the end result was it didn’t work. For all attempts I would verify that it runs from the command prompt first as the openHAB user exactly as typed (minus any @@ of course) and verify it worked. I’d try to run it from executeCommandLine and I’d get no output or an error message. Maybe it’s because I’m running in Docker (note, I executed the command lines from inside the container). But I’m not alone. Search the forum and you will see many threads where the solution was “¯\_(ツ)_/¯, put it in a shell script and call the script from executeCommandLine.”

I actually tried subprocess.check_output before resorting to executeCommandLine. For me, subprocess.check_output didn’t work. It caused a python error about popen argument being wrong or missing argument.

executeCommandLine is not bad if you break it down. Basically it wants the “arguments” (not space - the document is unclear and misleading) separated by @@. I should probably create a PR to edit the doc

Here’s an excerpt:

    def get_command(self, url, params):
        command = ["/usr/bin/curl", "-s", "-H", self.header, self.get_url(path, params)]
        return "@@".join(command)
...
   def something(self, blahblah)
        output = Exec.executeCommandLine(self.get_command(path, params), 10000)

Note that there can be spaces in self.header, path / params etc. Once you’ve figured this out, executeCommandLine isn’t so bad.

See, in the unix shell, you would call curl like this:

curl -s -H 'Header: blah blah' 'https://xxx.yy.zz/path?blahblah'

You would enclose the argument containing spaces with single or double quotes to tell the shell that it’s a part of a single argument. The shell will strip them off before passing them to the program (curl). With executeCommandLine, such syntax is not needed because it uses @@ as an argument separator, so spaces do not cause ambiguity as to whether it’s a part of the argument or an argument separator.

Sure, subprocess is a lot more versatile with input/output. I just wish I could make it work, and I didn’t spend too much time bothering with it. Maybe I will… and I’ll ping @rlkoshak :wink:

Hmmmm… I never got that far with subprocess. Did it fail because of the lack of timeout?

That would probably be the ideal. Anyone working on it?

1 Like

I’ve just spent some time adding this to HTTP Action. It was very easy.

Now my code looks like:

        headers = {"Authorization": "FPDAPI SubscriberToken={}".format(self.token)}
        output = HTTP.sendHttpGetRequest(url, headers, 10000)

I will submit a PR to core!

2 Likes
2 Likes

And that is how it is done! Nice! The helper libraries could get huge if we don’t spend the time to get the functionality back into OH.

1 Like

Did this make it into the 2.5.x builds? I get this error using 2.5.3:

TypeError: sendHttpGetRequest(): expected 1-2 args; got 3

where the call is:

TokenResponse = HTTP.sendHttpGetRequest('https://data.tankutility.com/api/getToken', headers, 10000)

I believe it made it into 2.5.2, so 2.5.3 should have it too. Have you tried to empty your cache and tmp directories?

Emptied the cache and tmp but it made no difference. Isn’t this in the core, not an add-on? I thought the updates are now only for add-ons, not for the core. Are you running the standard 2.5.2 or 2.5.3? Or do you have your own customized core for this fix? Thanks.

I tried this code on my installation and it worked fine, no errors about the number of arguments. I am on 2.5.2. The only things in my addons folder are a custom daikin binding jar and the jython engine from scott (5iver).

Hmm, what am I doing differently? You have a completely stock OH 2.5.2 other than the one add-on you mention?

Does this match your jython engine?

203 x Active x  80 x 2.5.0.202001221001      x openHAB Core :: Bundles :: Jython ScriptEngineFactory
-rw-r--r-- 1 openhab openhab 37403148 Mar 22 16:26 /usr/share/openhab2/addons/org.openhab.core.automation.module.script.scriptenginefactory.jython-2.5.0-SNAPSHOT.jar

What bundle is your change in? Since 2.5.0 came out before your change was checked in, it must be in an add-on patch bundle? Core bundles are not updated in the 2.5.x patches.

The script is below. Yours is just like this?

"""
TankUtility Jython Rule
"""

from core.rules import rule
from core.triggers import when
from core.actions import HTTP

@rule("TankUtility", description="Picks up propane tank values from Tank Utility servers", tags=[])
@when("Time cron 0 4 9 * * ?")
def tankUtilityDataCollection(event):
    tankUtilityDataCollection.log.info("TankUtility script is running")

    # leaving commented lines as blank so line numbers match the log below
    # 
    # 

    # 
    headers="{Authorization=Basic XXXXYYYY}"
    TokenResponse = HTTP.sendHttpGetRequest("https://data.tankutility.com/api/getToken", headers, 10000)

The log:

2020-04-01 09:04:00.312 [INFO ] [jsr223.jython.TankUtility           ] - TankUtility script is running
2020-04-01 09:04:00.318 [ERROR] [jsr223.jython.TankUtility           ] - Traceback (most recent call last):
  File "/etc/openhab2/automation/lib/python/core/log.py", line 51, in wrapper
    return fn(*args, **kwargs)
  File "<script>", line 20, in tankUtilityDataCollection
TypeError: sendHttpGetRequest(): expected 1-2 args; got 3

2020-04-01 09:04:00.320 [ERROR] [e.automation.internal.RuleEngineImpl] - Failed to execute rule '9d3c7542-2a9f-4ea8-8dc8-9ab9ac820829': Fail to execute action: 1
2020-04-01 09:04:00.320 [DEBUG] [e.automation.internal.RuleEngineImpl] -
java.lang.RuntimeException: Fail to execute action: 1
        at org.openhab.core.automation.internal.RuleEngineImpl.executeActions(RuleEngineImpl.java:1197) ~[bundleFile:?]

Thanks for your help.

org.openhab.core.model.script

This change was in OHC and came out after 2.5.0.

This is not completely correct, since there were a few OHC PRs included in 2.5.1. Try upgrading to the latest release (2.5.4).