Set up HTTP Bindings when openHAB runs behind a HTTP proxy

schnarr · June 22, 2016, 2:01pm

Hi community,

I try to get openHAB working on a linux box in a network which must use a HTTP proxy to connect to the Internet.

When I once started with “openHAB 1.8.3 demo master” for example the YahooWeatherBinding always failed with:

[ERROR] [o.o.b.w.i.scheduler.WeatherJob] - query.yahooapis.com java.net.UnknownHostException: query.yahooapis.com

When I used the web browser instead to GET sth. from this URL I received some data.
The browser uses no special HTTP proxy settings but the system’s HTTP proxy settings from the environment set by:

http_proxy=http://xxx.xxx.xxx.xxx:yyyy

Somewhere in the Internet I found (only) one discussion where one was told to define some Java properties accordingly when starting openHAB:

-Dhttp.proxySet=true -Dhttp.proxyHost=xxx.xxx.xxx.xxx -Dhttp.proxyPort=yyyy

but this didn’t change anything for me.

I tried the same properties with openHAB2 and the YahooWeatherBinding still complains about

Error accessing Yahoo weather: query.yahooapis.com

There’s nothing else pointing to the cause of that error in the log file, espacially no info from the involved classes – even if I start with start_debug.sh.

Is there a way (or a documentation?) how to configure openHAB/openHAB2 to direct it’s HTTP requests through a HTTP proxy?
How can I check if openHAB/openHAB2 recognizes properties set by JVM command line options?
Is there an environment variable which I can use to set such extra JVM command line options on openHAB startup?

Thank you in advance!

Max_G · June 23, 2016, 12:31am

Out of interest: Why not let the linux box go direct (bypassing the proxy)?

schnarr · June 23, 2016, 6:06am

Since the proxy is the one and only connect for HTTP traffic to the internet, indeed.

I can’t imagine that HTTP requests via proxy is that extraordinary that openHAB refuses to deal with this at all demanding direct internet access instead.

And as far as I can read from the logs the binding uses common classes which should be capable of using a proxy:

[TRACE] [.www.protocol.http.HttpURLConnection] - ProxySelector Request for https://query.yahooapis.com/... [TRACE] [.www.protocol.http.HttpURLConnection] - Looking for HttpClient for URL https://query.yahooapis.com/... and proxy value of DIRECT [TRACE] [.www.protocol.http.HttpURLConnection] - Creating new HttpsClient with url:https://query.yahooapis.com/... and proxy:DIRECT with connect timeout:-1 [WARN ] [oweather.handler.YahooWeatherHandler] - Error accessing Yahoo weather: query.yahooapis.com

(pls. notice the “and proxy value of DIRECT” in line 2 and the “and proxy:DIRECT” in line 3)

Therefor I guess it’s just a matter of few configuration lines.
But neither I could find a documentation for that nor did the intentional work.

benhelps · June 23, 2016, 11:56am

Just curious, if you’re using the .sh files on a command line to start openHAB, would setting the proxy as an environment variable in that command line session work?

schnarr · June 23, 2016, 1:41pm

The common shell variables
http_proxy=http://xxx.xxx.xxx.xxx:yyyy https_proxy=http://xxx.xxx.xxx.xxx:yyyy
are set and exported as I wrote in the beginning. The web browser, wget, curl, … all are working well with them.

The http.xyz named java properties themselves obviously can’t be defined as shell variables due to the dot in the names – they’re seemingly not allowed in shell variable names.

I already tried to add the property definitions mentioned in my first article to the java command in start[-debug].sh in openHAB, too – with no effect.

For openHAB2 I found the environment variable EXTRA_JAVA_OPTS used in the setenv, karaf, shell and instance scripts to extend the Java command line options but an

export EXTRA_JAVA_OPTS="-Dhttp.proxySet=true -Dhttp.proxyHost=xxx.xxx.xxx.xxx -Dhttp.proxyPort=yyyy"

before invoking openHAB2’s start.sh didn’t work as well.

I did a test with

export EXTRA_JAVA_OPTS="-XshowSettings:properties -Dhttp.proxySet=true -Dhttp.proxyHost=xxx.xxx.xxx.xxx -Dhttp.proxyPort=yyyy"

before invoking openHAB2’s start.sh and it listed all the 3 http.proxyXyz properties set as intended. Obviously openHAB/openHAB2 don’t care for these properties for whatever reason.

?-/

Zvika · July 27, 2016, 1:12pm

Any updates on this?
I also suffer from similar problem.

Zvika · July 27, 2016, 2:34pm

I think I’ve solved the problem (at least for my case…)
I’ve added both -Dhttp.proxy* and -Dhttps.proxy* to the script.
However, I’m running Windows 8 and not Linux, so maybe YMMV.

schnarr · July 28, 2016, 7:34am

At least for openHAB2 adding the https versions of the java properties did the trick, indeed.
Thank you for this hint, I’m a little bit embarrassed why I couldn’t see this from the logs 8-/

So at the end under LINUX one can do the following before starting openHAB2:

export EXTRA_JAVA_OPTS="-Dhttp.proxySet=true -Dhttp.proxyHost=xxx.xxx.xxx.xxx -Dhttp.proxyPort=yyyy -Dhttps.proxySet=true -Dhttps.proxyHost=xxx.xxx.xxx.xxx -Dhttps.proxyPort=yyyy"

with xxx.xxx.xxx.xxx being the IP address of the proxy.

Zvika · August 8, 2016, 7:48am

Well, you gave me the hint to use ‘Dhttp*’ at first place

I’m still investigating it, but it seems that Weather binding doesn’t respect the proxy setting - can you verify it at your installation?

Thanks

schnarr · August 8, 2016, 8:27am

Hi Zvika,
maybe I don’t understand your question right, but at least in openHAB2 (the successor of openHAB) on Linux the Yahoo Weather Binding works with the exported EXTRA_JAVA_OPTS environment variable as described above.

I don’t have any Windoze to try this on and at the moment I’m too busy to revert my installation here to try this out on a current openHAB.

But as far as I remember openHAB isn’t aware of that environment variable openHAB2 works with. So maybe you need to set all the java options one by one in the openHAB start.bat script where java is invoked. And with the java option

-XshowSettings:properties

right there you can at least check if the options are recognized by java at all.

Zvika · August 8, 2016, 8:49am

This is my .bat file (indeed, it’s OH1, Windows):

java %DEBUG_OPTS% -Dhttp.proxySet=true -Dhttp.proxyHost=proxy....com -Dhttp.proxyPort=911 -Dhttps.proxySet=true -Dhttps.proxyHost=proxy....com -Dhttps.proxyPort=911  -Dosgi.clean=true -Declipse.ignoreApp=true -Dosgi.noShutdown=true -Djetty.port=%HTTP_PORT% -Djetty.port.ssl=%HTTPS_PORT% -Djetty.home=. -Dlogback.configurationFile=configurations/logback_debug.xml -Dfelix.fileinstall.dir=addons -Dfelix.fileinstall.filter=.*\\.jar -Djava.library.path=lib -Dorg.quartz.properties=./etc/quartz.properties -Djava.security.auth.login.config=./etc/login.conf -Dequinox.ds.block_timeout=240000 -Dequinox.scr.waitTimeOnBlock=60000 -Dfelix.fileinstall.active.level=4 -Djava.awt.headless=true -jar %EQUINOXJAR% %* -console -XshowSettings:properties

Now I think that only https proxy is working, but not http. Does it make sense?
I know that https is working, because MyOpenHab is online and updated with every change.
However, other binding, like Weather, NTP and NetworkHealthBinding (which I think use only HTTP) fail connection:

11:42:00.630 [WARN ] [inding.ntp.internal.NtpBinding:127  ] - couldn't establish network connection [host 'ptbtime1.ptb.de'] -> returning current sytem time instead
11:42:00.631 [DEBUG] [inding.ntp.internal.NtpBinding:84   ] - Got time from ptbtime1.ptb.de: 11:42:00 IDT ꦭ 񰩠8 ᦢ沨 2016
11:42:00.633 [DEBUG] [o.o.i.m.i.MyOpenHABServiceImpl:281  ] - store(Date), state = 2016-08-08T11:42:00
11:42:05.206 [DEBUG] [o.o.b.n.i.NetworkHealthBinding:88   ] - timed out while connecting to host 'openhab.org' port '80' timeout '5000'
11:42:13.201 [ERROR] [.w.i.p.AbstractWeatherProvider:146  ] - WUNDERGROUND: Connection timed out: connect
11:42:13.222 [ERROR] [o.o.b.w.i.scheduler.WeatherJob:53   ] - Connection timed out: connect

schnarr · August 8, 2016, 9:26am

Just curious: Did you try it with the proxy’s IP, too?

Zvika · August 8, 2016, 9:44am

You saved my day
That’s indeed strange.

I had to use the IP and to add a SOCKS proxy.
Now the Weather binding is working - but NTP and NetworkHealthBinding still don’t…

12:38:03.145 [WARN ] [inding.ntp.internal.NtpBinding:127  ] - couldn't establish network connection [host 'ptbtime1.ptb.de'] -> returning current sytem time instead
12:38:03.146 [DEBUG] [inding.ntp.internal.NtpBinding:84   ] - Got time from ptbtime1.ptb.de: 12:38:03 IDT ꦭ 񰩠8 ᦢ沨 2016
12:38:03.148 [DEBUG] [o.o.i.m.i.MyOpenHABServiceImpl:281  ] - store(Date), state = 2016-08-08T12:38:03
12:38:07.820 [DEBUG] [o.o.b.n.i.NetworkHealthBinding:91   ] - couldn't establish network connection [host 'openhab.org' port '80' timeout '5000']
12:38:07.823 [DEBUG] [o.o.i.m.i.MyOpenHABServiceImpl:281  ] - store(Network_OpenhabWebsite), state = OFF
12:38:13.812 [DEBUG] [.w.i.p.AbstractWeatherProvider:75   ] - WUNDERGROUND[home]: Weather[Temperature[current=33.7,min=<null>,max=<null>,feel=38.0,dewpoint=22.0],Atmosphere[humidity=51,visibility=10.0,pressure=1009.0,pressureTrend=equal,ozone=<null>,uvIndex=12],Clouds[percent=<null>],Condition[text=򰰩�禸꭬lastUpdate=Mon Aug 08 12:38:13 IDT 2016,observationTime=Mon Aug 08 12:34:53 IDT 2016,id=<null>,icon=partlycloudy,commonId=partly-cloudy-day],Precipitation[rain=0.0,snow=0.0,probability=0,total=0.0],Wind[speed=0.0,direction=E,degree=90,gust=1.1,chill=<null>],Station[name=Kidron, Israel,id=IISRAELK2,latitude=31.811628,longitude=34.798454],<null>]

For future reference, these are the SOCKS flags:
-DsocksProxyHost=a.b.c.d -DsocksProxyPort=1080