Groovy Script Rule to Update Temperature-Offsets of AVM Fritz!DECT 301/302 based on external temperature-sensors

I’m using AVM Fritz!DECT 301 thermostats on all my radiators. These thermostats have a built in temperature-sensor which is inherently impercise caused by the placement near the radiator. While AVM offers it’s own external temperature sensors (e.g. AVM Fritz!DECT 440) which can be easily used to control the temperature offsets of the thermostats, those come at a hefty price and are integrated with in switches/wall sockets for which i had no need for.

On the other hand, AVMs (otherwise quite good) APIs for the Fritz!Box do not offer the option to control the offsets. So I had to find a way to use the web-ui of the Fritz!Box to update the offsets.

Below is my script which seems to work quite good (at least for my setup), perhaps this will help someone to start his/her own solution for the same or an equivalent problem :wink:

Right now all configuration (e.g. connection-data) has to be done in the script itself as i could not find a way to extract the required information from the bindings. And you have to install the XPATH-Transformation and place the gson-library in openhabs classpath (i downloaded it from Download gson.jar - @com.google.code.gson and put it in /usr/share/openhab/runtime/lib).

To use this you need another rule with triggers the script (i use a time-based rule which runs every 30 minutes).

// *************************************************************************************************************************************
// Groovy Script/Rule for OpenHAB to update the temperature offset of AVM Fritz! DECT 301/302 or COMET DECT Thermostats based on 
// measurements of arbitrary external temperature-sensor connected as items/things in OpenHAB.
// This implementation is based on https://github.com/SeydX/homebridge-fritz-platform/issues/149#issuecomment-883098365 and uses the
// login-example code provided by AVM here: https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AVM_Technical_Note_-_Session_ID_english_2021-05-03.pdf
// *************************************************************************************************************************************

import org.openhab.core.model.script.actions.HTTP
import org.openhab.core.model.script.actions.Log
import org.openhab.core.transform.actions.Transformation
import java.nio.charset.StandardCharsets
import java.security.InvalidKeyException
import java.security.NoSuchAlgorithmException
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import com.google.gson.Gson
import java.util.Map

import org.slf4j.LoggerFactory

def HOST = 'fritz.box' // Fritzbox-Host
def USER = '<user>' // Fritzbox-User
def PASSWORD = '<password>' // Fritzbox-Password
// Sensor-Mapping: Thermostat-ULE-Device-Name : List of Temperature-Sensors (QuantityType)
def SENSORS = [
  '<ulename>' : ['<item_id1>', '<item_id2>'],
  // ...
]


def LOG = LoggerFactory.getLogger("update-temperature-offset")


class FritzboxLoginState {
  private String host
  private String user
  private String sid
  private String challenge
  private String blockTime
  
  FritzboxLoginState(def host, def user = null, def sid, def challenge = null, def blockTime = 0) {
    this.host = host
    this.sid = sid
    this.user = user
    this.challenge = challenge
    this.blockTime = blockTime
  }
  
  String getHost() {
    return host
  }
  
  String getUser() {
    return user
  }
  
  String getSid() {
    return sid
  }
  
  int getBlockTime() {
    return blockTime
  }
  
  String getChallenge() {
    return challenge
  }

  boolean isLoggedIn() {
    return sid != "0000000000000000"
  }
  
  void logout() {
    HTTP.sendHttpGetRequest("http://${host}/login_sid.lua?logout=1&sid=${sid}", 10000)
    sid = "0000000000000000"
  }
  
  public String toString() {
    return getClass().getSimpleName() + "[host=${host}, user=${user}, sid=${sid}, challenge=${challenge}, blockTime=${blockTime}]"
  }
}


class FritzboxLogin {
  static LOG = LoggerFactory.getLogger("update-temperature-offset-fritzbox-login")
  
  private String host
  private String user
  private String password
  private FritzboxLoginState stateCache
  
  FritzboxLogin(def user, def password, def host = "fritz.box") {
    this.host = host
    this.user = user
    this.password = password
  }
  
  public getState() {
    if (!stateCache) {
      def resp = HTTP.sendHttpGetRequest("http://${host}/login_sid.lua?version=2&user=${user}", 10000)
      def user = Transformation.transform("XPATH", "/SessionInfo/Users/User[@last=1]/text()", resp)      
      def challenge = Transformation.transform("XPATH", "/SessionInfo/Challenge/text()", resp)
      def blockTime = Integer.parseInt(Transformation.transform("XPATH", "/SessionInfo/BlockTime/text()", resp))
      stateCache = (user == this.user)
        ? new FritzboxLoginState(host, user, Transformation.transform("XPATH", "/SessionInfo/SID/text()", resp), challenge, blockTime)
        : new FritzboxLoginState(host, this.user, "0000000000000000", challenge, blockTime)
    }
    return stateCache
  }
  
  public FritzboxLoginState execute() {
    return state.loggedIn ? state : login()
  }
  
  private FritzboxLoginState login() {
    def challengeResponse = calculatePbkdf2Response(state.challenge, password)
    def resp = HTTP.sendHttpPostRequest("http://${host}/login_sid.lua?version=2&user=${user}", "application/x-www-form-urlencoded", "response=${challengeResponse}", 10000)
    def sid = Transformation.transform("XPATH", "/SessionInfo/SID/text()", resp)
    if (sid == "0000000000000000") {
      throw new IllegalStateException("invalid user/password!")
    }
    
    stateCache = new FritzboxLoginState(host, user, sid)
    return stateCache
  }
  
 /**
 * Calculate the secret key on Android.
 */
 private String calculatePbkdf2Response(String challenge, String password) {
   String[] challenge_parts = challenge.split('\\$');
   int iter1 = Integer.parseInt(challenge_parts[1]);
   byte[] salt1 = fromHex(challenge_parts[2]);
   int iter2 = Integer.parseInt(challenge_parts[3]);
   byte[] salt2 = fromHex(challenge_parts[4]);
   byte[] hash1 = pbkdf2HmacSha256(password.getBytes(StandardCharsets.UTF_8), salt1, iter1);
   byte[] hash2 = pbkdf2HmacSha256(hash1, salt2, iter2);
   return challenge_parts[4] + '$' + toHex(hash2);
 }
  
 /** Hex string to bytes */
 private byte[] fromHex(String hexString) {
   int len = hexString.length() / 2;
   byte[] ret = new byte[len];
   for (int i = 0; i < len; i++) {
     ret[i] = (byte) Short.parseShort(hexString.substring(i * 2, i * 2 + 2), 16);
   }
   return ret;
 }
  
 /** byte array to hex string */
 private String toHex(byte[] bytes) {
   StringBuilder s = new StringBuilder(bytes.length * 2);
   for (byte b : bytes) { s.append(String.format("%02x", b)); }
   return s.toString();
 }
  
 /**
 * Create a pbkdf2 HMAC by appling the Hmac iter times as specified.
 * We can't use the Android-internal PBKDF2 here, as it only accepts char[] arrays, not bytes (for multi-stage hashing)
 */
 private byte[] pbkdf2HmacSha256(final byte[] password, final byte[] salt, int iters) {
   try {
     String alg = "HmacSHA256";
     Mac sha256mac = Mac.getInstance(alg);
     sha256mac.init(new SecretKeySpec(password, alg));
     byte[] ret = new byte[sha256mac.getMacLength()];
     byte[] tmp = new byte[salt.length + 4];
     System.arraycopy(salt, 0, tmp, 0, salt.length);
     tmp[salt.length + 3] = 1;
     for (int i = 0; i < iters; i++) {
       tmp = sha256mac.doFinal(tmp);
       for (int k = 0; k < ret.length; k++) { ret[k] ^= tmp[k]; }
     }
     return ret;
   } catch (NoSuchAlgorithmException | InvalidKeyException e) {
     return null; // TODO: Handle this properly
   }
 }
}


class FritzboxThermostat {
  def id
  def displayName
  def offset
  def correctedCelsius
  def doNotHeatOffsetInMinutes
  
  def FritzboxThermostat(def id, def displayName, def offset, def correctedCelsius, doNotHeatOffsetInMinutes) {
    this.id = id
    this.displayName = displayName
    this.offset = offset
    this.correctedCelsius = correctedCelsius
    this.doNotHeatOffsetInMinutes = doNotHeatOffsetInMinutes
  }
  
  def getId() {
    return id
  }
  
  def getUleDeviceName() {
    return displayName
  }
  
  def getMeasuredTemp() {
    return correctedCelsius - offset
  }
  
  def getOffset() {
    return offset
  }
  
  def getRoomTemp() {
    return correctedCelsius
  }
  
  def getWindowOpenTimer() {
    return doNotHeatOffsetInMinutes
  }
  
  def updateOffset(def newOffset) {
    return new FritzboxThermostat(id, displayName, newOffset, correctedCelsius - (offset - newOffset), doNotHeatOffsetInMinutes)
  }
  
  public String toString() {
    return getClass().getSimpleName() + "[uleDeviceName=${displayName}, id=${id}, offset=${offset}, measuredTemp=${measuredTemp}, roomTemp=${roomTemp}, windowOpenTimer=${doNotHeatOffsetInMinutes}]"
  }
}


def getThermostats(def loginState) {
  def resp = HTTP.sendHttpPostRequest("http://${loginState.host}/data.lua?sid=${loginState.sid}", "application/x-www-form-urlencoded", "?xhr=1&lang=de&page=sh_dev&xhrId=all&no_sidrenew=", 10000)    
  def devices = new Gson().fromJson(resp, Map.class)
  return devices['data']['devices']
        .findAll { it.category == "THERMOSTAT" && it.units.find { unit -> unit.type == "TEMPERATURE_SENSOR" && unit.skills.find { skill -> skill['offset'] != null } } }
        .collect { 
          def temperatureSensor = it.units.find { unit -> unit.type == "TEMPERATURE_SENSOR" }.skills.find { skill -> skill['offset'] != null }
          def windowOpenSensor = it.units.find { unit -> unit.type == "THERMOSTAT" }.skills.find { skill -> skill['temperatureDropDetection'] != null }.temperatureDropDetection
          return new FritzboxThermostat((int) it.id, it.displayName, temperatureSensor.offset, temperatureSensor.currentInCelsius, (int) windowOpenSensor.doNotHeatOffsetInMinutes)
        }
  .collectEntries { [it.displayName, it] }
}


def temperatures = SENSORS.collectEntries { name, sensors ->
  [name, (sensors.collect { sensor -> items.get(sensor).floatValue() }.inject(0) { acc, curr -> acc += curr } / sensors.size).round(1)]
}

def fritzboxLoginState = new FritzboxLogin(USER, PASSWORD, HOST).execute()
def thermostats = getThermostats(fritzboxLoginState)

def offsets = temperatures.collectEntries { name, temperature ->
  return [name, (temperature - thermostats[name].measuredTemp).round(2)]
}

def roundedOffsets = offsets.collectEntries { name, offset ->
  return [name, (offset * 2).round() / 2f ]
}

LOG.info("measured average temperatures: " + temperatures)
LOG.info("thermostat settings: " + thermostats)
LOG.debug(offsets as String)
LOG.debug(roundedOffsets as String)


def thermostatsToUpdate = thermostats
  .findAll { roundedOffsets[it.key] != it.value.offset }
  .collectEntries { name, thermostat -> [name, thermostat.updateOffset(roundedOffsets[name])] }

LOG.info("updating thermostats: " + thermostatsToUpdate)


thermostatsToUpdate.forEach { name, t ->
  HTTP.sendHttpPostRequest(
    "http://${fritzboxLoginState.host}/net/home_auto_hkr_edit.lua", 
    "application/x-www-form-urlencoded", 
    "sid=${fritzboxLoginState.sid}&device=${t.id}&Offset=${t.offset}&Roomtemp=${t.roomTemp}&ule_device_name=${t.uleDeviceName}&WindowOpenTimer=${t.windowOpenTimer}&view=&back_to_page=sh_dev&validate=apply&xhr=1&useajax=1",
    10000
  )    
  HTTP.sendHttpPostRequest(
    "http://${fritzboxLoginState.host}/data.lua", 
    "application/x-www-form-urlencoded", 
    "sid=${fritzboxLoginState.sid}&device=${t.id}&Offset=${t.offset}&Roomtemp=${t.roomTemp}&ule_device_name=${t.uleDeviceName}&WindowOpenTimer=${t.windowOpenTimer}&WindowOpenTrigger=&tempsensor=own&ExtTempsensorID=tochoose&view=&back_to_page=sh_dev&lang=de&xhr=1&no_sidrenew=&apply=&oldpage=%2Fnet%2Fhome_auto_hkr_edit.lua",
    10000
  )    
}

LOG.info("new thermostat settings: " + getThermostats(fritzboxLoginState))

fritzboxLoginState.logout()
1 Like

Hi Karsten!
Thank you for this script I’m using Home Assistant and developed a python module for the same purpose:

Currently I’m using selenium, since I didn’t find:
xhr=1&sid=${SID}&lang=de&page=sh_dev&xhrId=all&useajax=1&no_sidrenew=
I’ll definitely use your approche in the future!

Against which version of Fritz!OS did you develop / test this script? With my script I’m getting some really weird behaviour since 7.57: Video

It’s like the current temperature increases / decreases by a multiple of the offset.
Did you also observe this kind of behaviour?

My hardware:
FRITZ!DECT 301
Fritzbox 7490, FW 7.57

Hi Nils,

the script works with firmware versions only. If i remember correctly, the last working version is 7.49. As my FritzBox 7490 received the latest update recently too, and temperatures get colder, i’ll have to either reverse-engineer the new calls required or find some code already working with the new ui soon :wink: I’ll let you know when i have a script that works with newer firmware.

Kind regards
Karsten

1 Like

Hi Karsten,
I am very interested in the script. Is there an update for newer versions of Fritz OS? I am currently using version 7.57.
Best regards
Sören