OAuth2 using just OH Rules and myopenhab.org

On prompting from a user on another thread I’ve spend a little bit of time here and there over the past few months trying to get integration with the Dexcom API. Unfortunately, after getting it almost all the way working I discovered that the data from the sensor is delayed by 3.5 hours, making it useless in a home automation context.

But, the API does use OAuth2 for authentication and I managed to figure out how to do that so it isn’t a total loss.

OAuth2

image

OAuth2 is a standard protocol for authorization services that need to share data between each other. For example, the integration between myopenhab.org and Google Assistant is authorized by OAuth2.

From the user’s perspective the OAuth2 process looks something like this:

  1. In the service that wants to connect (we’ll call it service A) to the external service (we’ll call it service B) the user performs and action (e.g. clicks a URL) indicating they want to integrate this service with the external service.

  2. The browser opens a web page for the service B asking the user to log in to service B and approve giving service A access to the requested data.

  3. The user approves the access and the browser returns to service A and now the two services are connected.

There is a whole lot of stuff that goes on under the hood though. The Dexcom API Docs actually has a pretty good description for how it works behind the scenes so please see it for the flow.

https://developer.dexcom.com/authentication

The tl;dr of it is you need to acquire an auth_token that you use for each request to the API. This auth_token is what indicates which user you want to acquire the data proves you are authorized to access that data. These auth_tokens only last for a limited amount of time. When they expire you need to refresh the auth_token using a refresh_token.

Prerequisites

For this to work you need to have myopenhab.org and the openHAB Cloud Connector binding configured and working. Alternatively this will work if you expose OH to the Internet through a reverse proxy as long as you have a valid cert (e.g. one from LetsEncrypt).

If you have your own instance of the openHAB Cloud then you should implement the OAuth2 code there and not in Rules.

Next you need to register for a developer account with service A and obtain a Client ID, Client Secret, and register a redirect URL.

Step 1: Getting the auth_code

The first step in the process is obtaining an auth_code. The auth_code is a string of letters and number that service A will send to service B’s redirect URL. It is part of the data that gets passed to the service when the user gets redirected to service B’s web page to log in and authorize the connection. So the first problem is how do I receive this HTTP GET request if I don’t have a server?

HTML File

You are in luck as OH has a simple web server that can access static html pages through myopenhab.org. So you need to create an oauth2.html file and place it in the conf/html directory (/etc/openhab2/html for an installed OH on Linux). This lets you use https://myopenhab.org/static/oauth2.html as the redirect URL you register with service A.

oauth2.html includes some JavaScript that parses out the arguments that are included as part of the URL and posts the auth_code to an Item in openHAB using openHAB’s REST API.

<!DOCTYPE html>
<html>
	<head>
		<title>OAuth2 Catcher</title>
	<script type="text/javascript">
	function getAllUrlParams(url) {

  // get query string from url (optional) or window
  var queryString = url ? url.split('?')[1] : window.location.search.slice(1);

  // we'll store the parameters here
  var obj = {};

  // if query string exists
  if (queryString) {

    // stuff after # is not part of query string, so get rid of it
    queryString = queryString.split('#')[0];

    // split our query string into its component parts
    var arr = queryString.split('&');

    for (var i=0; i<arr.length; i++) {
      // separate the keys and the values
      var a = arr[i].split('=');

      // in case params look like: list[]=thing1&list[]=thing2
      var paramNum = undefined;
      var paramName = a[0].replace(/\[\d*\]/, function(v) {
        paramNum = v.slice(1,-1);
        return '';
      });

      // set parameter value (use 'true' if empty)
      var paramValue = typeof(a[1])==='undefined' ? true : a[1];

      // (optional) keep case consistent
      paramName = paramName.toLowerCase();
      paramValue = paramValue.toLowerCase();

      // if parameter name already exists
      if (obj[paramName]) {
        // convert value to array (if still string)
        if (typeof obj[paramName] === 'string') {
          obj[paramName] = [obj[paramName]];
        }
        // if no array index number specified...
        if (typeof paramNum === 'undefined') {
          // put the value on the end of the array
          obj[paramName].push(paramValue);
        }
        // if array index number specified...
        else {
          // put the value at that index number
          obj[paramName][paramNum] = paramValue;
        }
      }
      // if param name doesn't exist yet, set it
      else {
        obj[paramName] = paramValue;
      }
    }
  }

  return obj;
}
var code = getAllUrlParams().code;
	</script>
	</head>
<script>
var item = 'Dexcom_AuthCode'
document.write("Preparing to update " + item + " Item...")
var code = getAllUrlParams().code;
var xhr = new XMLHttpRequest();
xhr.open('POST', "https://myopenhab.org/rest/items/"+item);
xhr.setRequestHeader("Content-Type", "text/plain");
xhr.setRequestHeader("Accept", "application/json");
xhr.onreadystatechange = function() {
	if(xhr.readyState == XMLHttpRequest.DONE && xhr.status == 200) {
		document.write("Received  " + code);
	}
	else if(xhr.status != 200) {
		document.write("Error in callback: " + xhr.status + " " + xhr.responseText);
	}
}
document.write("Sent " + code + "...");
xhr.send(code)
</script>

</html>

If you adopt this HTML in your adventures, change the item variable to the name of the Item you want to receive the auth_code. In the above, the Item I’m using is Dexcom_AuthCode.

How this works: When the user chooses to authorize our “app” to access the API and data service A will redirect the browser to your redirect URL and include as a URL argument the auth_code. By using https://myopenhab.org/static/oauth2.html as the redirect URL this will cause this page on your openHAB to be loaded into the browser. The browser will run the JavaScript which parses out the auth_code and sends it as a command to the Dexcom_AuthCode Item, also through myopenhab.org.

If you are not already logged into myopenhab.org in your browser you will be prompted for your username and password.

From here on out everything in internal to openHAB Rules.

NOTE: the JS code to parse out the arguments above came from a StackExchange posting that I can’t find anymore.

openHAB

At least for the Dexcom API, the auth_code only lasts for one minute. So we need to automate the next step which is why we send it as a command to an Item in OH.

So here are the Items:

String Dexcom_AuthCode // receives the auth_code
String Dexcom_AccessToken // stores the access token
String Dexcom_RefreshToken // stores the refresh token
DateTime Dexcom_Expires // stores the time when the access token will expire
Switch Dexcom_Poll // triggers a poll of the Dexcom API
Switch Dexcom_Refresh // triggers a refresh of the auth_token and refresh_token
Switch Dexcom_EGVS_Poll // triggers a poll for EVGS data

And here is the code that runs when Dexcom_AuthCode receives a command.

import java.net.URL
import javax.net.ssl.HttpsURLConnection
import java.io.BufferedInputStream
import java.io.BufferedReader
import java.io.InputStreamReader
import org.eclipse.xtext.xbase.lib.Procedures
import org.eclipse.xtext.xbase.lib.Functions

val LOG = "dexcom"

// Constants
val BASE_URL = "https://sandbox-api.dexcom.com" // change to the URL for your service
val TOKEN        = "/v2/oauth2/token" // change this to the endpoint URL for your service

// Takes an HTTPS connection and reads the response data. If the response code is not 200 the String
// "ERROR <code>" is returned where <code> is the HTTP return code. 
// This is a lambda that gets reused for all the API calls.
val Functions$Function1<HttpsURLConnection, String> readResponse = [ connection |

  logInfo("dexcom", "checking response code")
  val responseCode = connection.responseCode
  if(responseCode != 200){
    logError("dexcom", "Received non-ok return code: " + responseCode)
    return "ERROR " + responseCode.toString;
  }

  logInfo("dexcom", "pulling response")
  val StringBuffer sb = new StringBuffer()
  val BufferedReader br = new BufferedReader(new InputStreamReader(new BufferedInputStream(connection.getInputStream())))
  var String inputLine = ""
  while ((inputLine = br.readLine()) !== null) {
    sb.append(inputLine)
  }
  sb.toString
]

// Acquires a new auth token when a new auth code is retrieved or an auth token expires
val Procedures$Procedure4<String, String, Boolean, Functions$Function1<HttpsURLConnection, String>> getAuth = [ tokenEndpoint, code, refresh, readResp | 
  // Change these to your values
  val clientId = "Your Client ID"
  val clientSecret = "Your Client Secret"
  val redirect = "https://myopenhab.org/static/oauth2.html"

  // Prepare the Request
  var request = 'client_secret='+clientSecret+'&client_id='+clientId+'&{code}&grant_type={type}&redirect_uri='+redirect // the request for your service may be different
  if(refresh) request = request.replace('{code}', '&refresh_token='+code).replace('{type}', 'refresh_token')
  else        request = request.replace('{code}', '&code='+code).replace('{type}', 'authorization_code')

  val tokenURL = new URL(tokenEndpoint)
  val HttpsURLConnection connection = tokenURL.openConnection() as HttpsURLConnection

  // Set the headers and parameters
  connection.setRequestProperty('content-type', 'application/x-www-form-urlencoded')
  connection.setRequestProperty('cache-control', 'no-cache')
  connection.requestMethod = "POST"
  connection.doOutput = true
  connection.setDoInput = true

  // Make the HTTPS call
  logInfo("dexcom", "Attempting to " + if(refresh) "refresh " else "acquire " + "auth tokens")
  connection.outputStream.write(request.getBytes("UTF-8"))

  // Get the response body
  logInfo("dexcom", "Reading response")
  val response = readResp.apply(connection)
  if(response.startsWith("ERROR")) {
    logError("dexcom", "Failed to " + if(refresh) "refresh" else "obtain" + "auth token")
    return;
  }

  // Populate the token Items
  logInfo("dexcom", "Populating auth token Items")
  Dexcom_AccessToken.postUpdate(transform("JSONPATH", "$.access_token", response))
  Dexcom_RefreshToken.postUpdate(transform("JSONPATH", "$.refresh_token", response))
  Dexcom_Expires.postUpdate(now.plusSeconds(Integer::parseInt(transform("JSONPATH", "$.expires_in", response))).toString)

  // Poll Dexcom for the latest data
  Dexcom_Poll.sendCommand(ON)

]

// Use the AuthCode to get the auth and refresh tokens
rule "Received authcode"
when
  Item Dexcom_AuthCode received command
then
  getAuth.apply(BASE_URL+TOKEN, receivedCommand.toString, false, readResponse)
end

// Use the refresh token to get a new auth and refresh token
rule "Refresh the auth token"
when
  Item Dexcom_Refresh received command
then
  getAuth.apply(BASE_URL+TOKEN, Dexcom_RefreshToken.state.toString, true, readResponse)
end

How it works: the code above is specific to Dexcom. You will need to modify it for the endpoints defined by the API you are interacting with. You also need to change parsing of the results as necessary.

When a new auth_code is received a Rule triggers that calls the getAuth lambda. The lambda creates an HttpsURLConnection to the authorization API endpoint and makes the call to that end point passing the auth_code as one of the arguments. The API will return a response body that will include the auth_token, refresh_token, and how many seconds the auth_token will live. We parse those values out of the response and send them to the proper OH Items.

At some point the auth_token will need to be replaced. At that point a command will be sent to Dexcom_Refresh which will trigger a rule to call the same lambda, only this time it will use the refresh_token instead of the auth_code and a slightly different request.

Step 2: Querying the API

For Dexcom, one passes the auth_token as a header value in gets to the API endpoints. Below is the code that queries the Dexcom API for data between a date range. When the auth_token expires, a 401 HTTP code will be returned telling us we need to refresh the auth_token.

// Queries the endpoint and returns the result. Returns "REFRESH" if a 401 code was returned by Dexcom
// Returns "ERROR" with return code if result is not 401 or 200
val Functions$Function2<String, Functions$Function1<HttpsURLConnection, String>, String> queryDexcomAPI = [ endpointURL, readResponse |
  val endpoint = new URL(endpointURL)
  val HttpsURLConnection connection = endpoint.openConnection() as HttpsURLConnection
  connection.setRequestProperty('authorization', 'Bearer ' + Dexcom_AccessToken.state.toString)
  connection.requestMethod = "GET"
  connection.setDoOutput(true)

  logInfo("dexcom", "Polling Dexcom for new data")
  val response = readResponse.apply(connection)

  if(response == "ERROR 401") {
    logInfo("dexcom", "Time to refresh the auth token")
    Dexcom_Refresh.sendCommand(ON)
    return "REFRESH";
  }
  else if(response == "ERROR 400") {
    logError("dexcom", "Time window for query must be 90 days or less")
    return response;
  }
  else if(response.startsWith("ERROR")) {
    logError("dexcom", "Error getting data from Dexcom!")
    return response;
  }

  // Process the results
  logInfo("dexcom", response)  
  response
]

rule "Poll Dexcom for data"
when 
  Item Dexcom_Poll received command
then
  Dexcom_EGVS_Poll.sendCommand(ON)
  // pause before calling other endpoints to avoid hitting the server too hard
end

rule "Poll for and process EVGS Data"
when
  Item Dexcom_EGVS_Poll received command
then
  val String url = BASE_URL + EGVS.replace('{start}', '2018-02-16T00:00:00').replace('{end}', '2018-04-15T00:00:00')
  logInfo(LOG, "Querying using URL " + url)

  val response = queryDexcomAPI.apply(url, readResponse)

  // Exit if we need to refresh the auth token. The Rule will execute again when the token is refreshed
  if(response == "REFRESH") return;

end  

The calls to the end points work pretty much the same so again we use a lambda for the common code. When the event occurs to trigger a poll we call the lambda with the endpoint properly formatted URL. We pass the auth_token as a header parameter and read out the data. If a 401 return code is returned, we trigger an auth_code refresh which will trigger another poll of the API once it is done.

Conclusion

This is not the best way to do this. But it might help you prototype a connection to a REST API that requires OAuth2 in OH to figure out how it works. Once you get it working you should then plan on converting it to a binding and/or integrated with openHAB Cloud.

Good luck!

11 Likes

Great work! I was looking for this for months.
I cannot await trying to get it running!

I am looking for such an implementation to check the battery level of my smart electric drive vehicle.
Therefore I want to use the mercedes API (via OAuth2).
Charging my vehicle should be initiated when battery level is below some certain point.
So I need to query the battery status regularly.

Dumb question:

Any ideas to handle this log-in automatically?
As stated above I do not actively query the API but need to check it regularly based on some triggers in rules.
So I am not able to log in at myopenhab.org manually with username and password.

You only interact through myopenhab.org when obtaining the authcode. You should only have to do this once. Once you receive the auth code you interact with the remote API exclusively through rules.

You cannot automate getting the authcode because it is the authentication with myopenhab.org that provides the positive approval by you the user that you want to allow the external service (myopenhab.org) to be allowed to query and interact with the remote service.

But like I said, once you get the auth code, you immediately exchange that for an auth token and refresh token. All future interactions just use the fact that you know the client I’d, client secret, and auth token to allow the http requests.

I just got access to the Skydrop (sprinkler system) API so I’m going to be trying this method with their systems. If y’all have seen my posts you’ll know I’m still learning a lot so I expect it will be a little while before I have this running. Step 1 is done (get API access) so now it’s on to Step 2: read about OAuth2.

If anyone has any tips about using OAuth2 I’d be glad to get some additional guidance on how the process works.

Rich,

I wanted to follow up and say this is awesome. I used your scripts, changed anything “dexcom” to “skydrop” - and ran it. Now I can use HTTP GET and POST commands to control my sprinklers.

I don’t know if it’s BETTER than the wifi app from skydrop… but I’m always happy to get another thing consolidated into OpenHAB.

Now to fiddle with the skydrop API to see if I can make it useful.

Thanks!

I’m glad you found it useful. If you figure out the skydrop API you will have done most of the hard part for a new binding. :wink:

That’s true. I’ll start a new thread for that discussion, don’t want to dump all of that planning into this thread, we’ll keep it for OAUTH2.

Watch for it in a minute or two…

Thanks for your tutorial. Unfortunately I got the following bug, see my logs:
2019-01-16 01:09:12.778 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule ‘Poll Dexcom for data’: The name ‘Dexcom_EGVS_Poll’ cannot be resolved to an item or type; line 125, column 3, length 16

Thanks in advance for a reply.

Best regards,
Jochen

Before spending too much time on this, what is your end goal? I am because of it is specifically to get Dexcom readings, the deejay betwen the readings and the ability to pull them through this API makes it pretty much useless in a home automation context. If that is your goal, you will be better off setting up Nights out which has a much simpler API. I’ve posted a tutorial due integrating it with OH as well.

If you are just trying to get OAurh working, then the specific error means you are missing that Item. in on my phone and about to go to bed so it will be tomorrow veggie I’ll be able to remember what that item is for.

Thanks a lot for your reply rikoshak! Can you send me a link to the tuturial you mentioned - I was not able to find it by googling myself.

My final goal is to create a solution to access a RestAPI which uses OAuth2. Currently I am figuring out if this approach can outperform the idea of developing a full fledged binding, which takes much more time in my opinion. Moreover the binding won’t be useful for third parties, as far as I can guess. What do you think?

Related to the Error-Message: I guess your tutorial has an error in it, because before I can start the call, my log messages tell me, that I need to define the Item somewhere. Can you check, if the tutorial runs on your environment? I guess, that people will try to use it and it is hard and maybe frustrating if the code won’t run. Thanks for offering the tutorial, anyway and maybe thanks in advance for your verification, if this stuff runs in your openhab-instance.

Best regards,
Jochen

The tutorial is missing the Dexcom_EGVS_Poll Item. It is just a Switch.

If you are looking to interact with a service that requires OAuth, the appropriate approach really is a new binding. This tutorial is just intended to be a “plan B” in case a binding is not feasible, or as a way to prototype the interactions with the interface before writing a binding.

Thank you, learned a lot about oauth while pulling my hair out :wink:

I used your example to issue an “open” command to the Nello API. Nello is a gadget that listens for the bell and opens front doors to apartment buildings.
https://www.nello.io/en/
https://nellopublicapi.docs.apiary.io/

Problems were that it only worked with a client credentials grant, the oauth token url needed a trailing slash, and that the API itself wants PUT requests instead of GET. I think the whole authcode callback thing was not needed, and that was the only part that worked straight away :frowning:

But when I forget my frustration and am in an optimistic mood, I might try to leverage this html callback method to get ring events into OH.

I just got the nello device and I am planning to integrate it into Oh, have @AFromD gotten any further?

Sorry, I gave up on that.

I think the reason was that it was impossible to do without my own vserver on the internet. I was hoping I would be able to do it trhough myopenhab or maybe with a LAMP webhosting package I already use. But I wasn’t able to.

Sorry for this question, but in which file do i have to put all the following code below?

Please read some of the later posts. Dexcom throttles the data. Unless you have a use for driving your home automation using blood sugar readings that are 30 minutes or more old the API isn’t going to be useful for you. You would be better off setting up Nightscout and either populating it from CLARITY or using xDrip or Spike on iOS. I’ve shown how to do most of that in NightScout openHAB Integration.

And if you are so new to openHAB that you don’t know where to put Items and Rules files, I recommending stepping back and reviewing some getting started info: How to get started (there is no step-by-step tutorial).

Items go in .items files, code goes in .rules files

1 Like

This whole process used to work perfectly but not anymore.

when working
Google assistant
select openhab
GA redirects to an Oauth2 screen
fill in credentials
all devices from openhab are visible in google assistant

i unlinked my setup (because i could not sync devices anymore)

now when i do the same i get

Google assistant
select openhab
NO OAUTH2 redirect screen anymore
instead is states that Openhab is connected
then after 5 seconds it states "something went wrong, please try again.

i t looks like google is not even trying to reach openhab anymore and does not get the oauth2 screen anymore

who is reponsible for the Openhab service from within google because i think something is messed up(probably after a recent update in june/juli)

could be that everything still works for everybody but not when you unlink and link again

i tried something else.
since openhab is not the problem if figured it must be google

If i login with another google gmail account everything works perfectly

so how can it be that one account fails and another works

can google disable things on an account level?