Control a water heater and ground source heat pump based on cheap hours of spot priced electricity

My system has now been up and running without problems for about one month. No changes has been made but today the entsoe.js script didn’t run correctly and no spot prices where added to the influx db. The error message is,

2022-10-24 19:45:28.785 [ERROR] [nhab.automation.script.ui.d023cfcc23] - entsoe.js: Exception parsing spot prices: Cannot read property "period.timeInterval" from undefined

Has anyone else encountered this? Any ideas about what is going wrong?

Entso-E platform has some issues and tomorrow’s prices are not available there, not even their web user interface, see

https://m-transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=GRAPH&areaType=BZN&atch=false&dateTime.dateTime=25.10.2022+00:00|EET|DAY&dateTime.timezone=EET_EEST&dateTime.timezone_input=EET+(UTC+2)+/+EEST+(UTC+3)&biddingZone.values=CTY|10YAL-KESH-----5!BZN|10YAL-KESH-----5&resolution.values=PT60M

Hi all since yesterday data is present in api response for 25th oct for swwden se3 atleast.

Hi,

For information,

Since end of October 2022 the FMI HIRLAM weather forecasting model is no longer in use. The fmi.js script must be updated to get the weather forecast from the HARMONIE forecasts. See info received from FMI below,

Hi,

Thank you for your feedback! The use of the HIRLAM weather forecasting model at the Finnish Meteorological Institute has been ended in the end of October 2022, after which its forecasts are not available through the open data services. HARMONIE (MEPS) forecasts are available through our open data services.

Here you can read more about the changes: The use of the HIRLAM weather forecast model will end in October - Finnish Meteorological Institute

More information on our open data services: Open data - Finnish Meteorological Institute

Best Regards,

…………………….

Nina Kaitemo
Communications

Finnish meteorological institute
Erik Palménin aukio 1, 00560 Helsinki
PL 503, 00101 Helsinki
www.ilmatieteenlaitos.fi

Thanks for sharing the root cause, I was wondering where my weather forecast are but this explains…

I remember that parsing the weather forecast XML was the most painful part of this whole exercise but I now need to redo this part. Let’s see when I will have time for it. I’ll publish a new version of the weather forecast script unless somebody else is faster than me.

Markus

Hi,

I already upgraded the script so you can have it here…

/**
 * Javascript module for reading weather data from FMI API - HARMONIE.
 *
 * Copyright (c) 2022 Markus Sipilä, Updated for HARMONIE Rainer Keto
 *
 * Permission to use, copy, modify, and/or distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
 * REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
 * AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
 * INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
 * LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
 * OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 * PERFORMANCE OF THIS SOFTWARE.
 */

/**
 * Exports.
 */
module.exports = {
    getForecast: getForecast,
};

/**
 * Reads the weather forecase from FMI API.
 *
 * @param string place
 *   Place recoginzed by FMI API.
 *
 * @return string
 *   XML response from FMI API.
 */
function getForecast(place) {
    const xml = makeApiCall(place);
    const points = preparePoints(xml);
    return points;
}

/**
 * Makes an API call to the Finnish Meteorology Institute.
 *
 * @param string place
 *   Place recoginzed by FMI API.
 *
 * @return string
 *   XML response from FMI API.
 */
function makeApiCall(place) {
    const http = Java.type("org.openhab.core.model.script.actions.HTTP");
    console.log('fmi.js: Making an API call to FMI API...');
    const url = 'http://opendata.fmi.fi/wfs?service=WFS&version=2.0.0&request=getFeature&storedquery_id=fmi::forecast::harmonie::surface::point::simple&place=' + place + '&parameters=temperature';
    const xml = http.sendHttpGetRequest(url, 10000);
    return xml;
}

/**
 * Parses the forecasted temperatures from the XML response.
 *
 * @param string xml
 *   FMI response in XML format.
 *
 * @return array
 *   Array of point objects.
 */
function preparePoints(xml) {
    console.log('fmi.js: transforming XML to JSON and parsing temperatures...');
    const transformation = Java.type("org.openhab.core.transform.actions.Transformation");
    let tempPoints = [];

    // Early exit in case XML is null.
    if (xml == null) {
	console.error('fmi.js: XML empty, parsing aborted.')
	return tempPoints;
    }
    try {
	const jsObject = JSON.parse(transformation.transform('XSLT', 'xml2json.xsl', xml));
	const members = jsObject['wfs:FeatureCollection']['wfs:member'];
	for (let i = 0; i < members.length; i++) {
		    let point = {
			datetime: members[i]['BsWfs:BsWfsElement']['BsWfs:Time'],
			value: members[i]['BsWfs:BsWfsElement']['BsWfs:ParameterValue']
		    };
		    tempPoints.push(point);
	}
	console.log('fmi.js: Temperatures parsed!');
    }
    catch (exception) {
	console.error('fmi.js: Exception parsing temperatures: ' + exception.message);
    }

    return tempPoints;
}
1 Like

Thank you @ruxu ! I also updated this to the comment #13 which is marked as the solution for this thread.

Thank your for your effort! I got mine working a couple of days ago and today I had the control connected to the physical layer of house 'lectrics.

Quick info for others regarding the setup I have: I’m running OH 3.3 branch, have the same relay board from Waveshare as Markus with RPi 4B+ on it and have OH running on a separate server. If you make identical steps and as per the guide, you will get it to work out of the box.
Just for the info if you are thinking of using the same relay board with BananaPi, it doesn’t work out of the box as the pigpio and wiringpi are not compatible and would require coding skills to get it to work.
At first I wanted to use the Banana as it has better mass storage than RPi and was aiming to have the OH on it, but after I realized I have to use the RPi already in my possession, because lack of coding skills and understanding, I opted to have the OH running on a ProxMox VM installed in my x3950M4 server to “never” having to worry about sufficient IO capabilities or storage corruption, so I could of just went witha Denkovi ethernet relay or a similar product.


Anyway, now that I have it running, I wondered if you could help me just a bit so I could have my generator started when the spot price is high enough.
I looked for the waterheater.js for a couple of hours and compared it with the rule script to make adjustments and get it done - but it seems like that period of time was similar to a pig watching a satellite trying to understand it…

But still, in case you don’t have the time or interest, thank you again for the effort and making it public. It sure is helpful for many!

Do you mean that you want to start the generator if the hourly spot price is above some threshold?

Or do you want to start it every day for the most expensive hours?

When do you want to stop it?

Markus

Start when price threshold is reached and stop after price decreases below the threshold.

I think there could be a percentage value regarding the shutdown, hence 100% would be the threshold price and if set to 80%, the generator would stop after the price is 0,8*spot price. I’m collecting the heat generated by the exhaust and water jacket so it’s not necessary to stop immediately after the price goes under the set threshold value.

Min running time can be one hour. Other scheduling / start limitations can be set with time based rules for running the script, I think. I have a machine shop and the idea is to cap the peak prices during working hours.

You don’t need to read, clone or modify the waterheater.js that you were wondering earlier.

All you need to do is to create an hourly executed rule with a script action, similar what we now have for the “does the waterheater_control have value 1 for the current hour.”

Conceptually the only change in the script action is that you don’t read waterheater_control, but you read spot_price value for the current hour. And them you don’t compare if it is 1 or 0, you compare it to the price threshold of your choice.

Hope this helps,
Markus

1 Like

I made some major updates to comment #13 as a Christmas present for the community. It is now giving more more guidance to new openHab users who are learning the basic openHab concepts at the same time.

3 Likes

Is there a bug in influx.js.txt ?

Line 41: bucket: autogen’,
Shouldn’t this be
bucket: ‘autogen’,

?

FYI:

It’ll be streamed 23/02/04 but you can already download video in the attachment…

 

I’m also commercially selling this system (see link to a demo link in link above).

And I just got a Tibber tariff myself starting in march (or both of us to develop this system, actually) so we’re now looking to combine our EMS with variable tariffing by then.

Stay tuned :slight_smile:

On a sidenote, as you need to tell your heat pump to run during cheap times you need to have some interface into it.
If you don’t happen to have an OH binding for your heat pump like the OP, use the Smart Grid ready inputs, all new and even most older pumps will have one.

I’ve just deployed a 15€ Shelly Uni for that purpose. It has WiFi and 2 potential free outputs.
If your SGr terminals are 230VAC, you need to decouple and add a another dual relay, 10€ on *bay.

Suurkiitos Markus!

I spotted your solution on the Fissio forum, and decided to give this a go. I am running Openhab on Armbian on an Orangepi 3 LTS. I would still like a walk through of how you made the nice Graphana UI work.

Timo

@timo12357 thanks for spotting the typo. I’ll fix it later today . I might have time later today to elaborate on the UI as well.

1 Like

Actually my heat pump (Nibe F-1226) does not have an OpenHab Binding. I was using the external inputs that my heat pump has, those are the exact ones that could be used to connect it to Smart Grid but at least in Nibe you can choose from many options what you want these inputs to do. For example “prevent compressor” or “prevent tap water heating”.

@timo12357 The typo is now fixed but it’s getting late so I didn’t have time for the UI write-up.

I checked my old notes and here are the tutorials / docs that you might find useful. openHabian OS would have Grafana pre-installed but I understood you are not using it.

Setting this up:

• Remember to set the Authorization header in Grafana data sourcen settings

When it comes to the actual graphs, here are a couple of queries that you might find useful:

Consumption vs. spot prices

I fetch the consumption data from Caruna with GitHub - kimmolinna/pycaruna: Caruna API for Python and store it InfluxDB as “caruna_consumption” measrument.

Queries in plain text:

from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop:v.timeRangeStop)
  |> filter(fn: (r) =>
      r._measurement == "caruna_consumption" and 
      r._field == "value"
      )
  |> map(fn: (r) => ({ _value:r._value, _time:r._time, _field:"Sähkön kulutus (kWh)" }))

And the second one:

from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop:v.timeRangeStop)
  |> filter(fn: (r) =>
      r._measurement == "spot_price" and
      r._field == "value"
      )
  |> map(fn: (r) => ({ _value:r._value, _time:r._time, _field:"Sähkön tuntihinta (c/kWh)" }))

Spot prices and the control values


The spot prices you already have above, it’s just visualized as a bar chart here but the query is the same

Control values that are rendered below:

from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop:v.timeRangeStop)
  |> filter(fn: (r) =>
      r._measurement == "nibe_control" and 
      r._field == "value"
      )
  |> map(fn: (r) => ({ _value:r._value, _time:r._time, _field:"Maalämpöpumpun kompressori" }))

Average price of the day:

You need to change the Grafana “visualization” as a “Stat”

from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop:v.timeRangeStop)
  |> filter(fn: (r) =>
      r._measurement == "spot_price" and 
      r._field == "value"
      )
  |>mean()

Table of spot prices:
Same query as for the line / bar chart, but change the “visualization” as “Table”

Embedding the graphs to OpenHab pages
See the link in the bottom of comment #13 on how to create pages in OpenHab and how to embedd the Grafana dashboards as an “Web Frame” (which means an iframe in practice, if you are familiar with iframes)

And one for the road: Here’s the dashboard I use to monitor my consumption vs. the monthly average. Bars pointing downwards mean that we have consumed on an hour that was cheaper than the month’s average and bar pointing upwards mean that we have consumed on an hour that was more expensive than the month’s average.

Query for average price, visualize as “Stat”

from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => (r["_measurement"] == "spot_price"))
	|> filter(fn: (r) => (r["_field"] == "value"))
	|> mean()

Query for consumption factor (kulutusvaikutus), visualize as “Stat”

avg_price =from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => (r["_measurement"] == "spot_price"))
	|> filter(fn: (r) => (r["_field"] == "value"))
	|> mean()
  |> findColumn(fn: (key) => key._measurement == "spot_price", column: "_value") 

total_consumption =from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => (r["_measurement"] == "caruna_consumption"))
	|> filter(fn: (r) => (r["_field"] == "value"))
	|> sum()
  |> findColumn(fn: (key) => key._measurement == "caruna_consumption", column: "_value")

caruna_consumption = from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "caruna_consumption" and r["_field"] == "value")

spot_price = from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "spot_price" and r["_field"] == "value")

join(tables:{caruna_consumption:caruna_consumption, spot_price:spot_price}, on:["_time"])
  |> map(fn:(r) => ({
      time: r._time,
      _value: ((r._value_spot_price - avg_price[0]) * r._value_caruna_consumption) / total_consumption[0], 
    })
  )
  |> sum()

Bar chart of the consumption factor, visualize as bar chart time series.

avg_price =from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => (r["_measurement"] == "spot_price"))
	|> filter(fn: (r) => (r["_field"] == "value"))
	|> mean()
  |> findColumn(fn: (key) => key._measurement == "spot_price", column: "_value") 

total_consumption =from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => (r["_measurement"] == "caruna_consumption"))
	|> filter(fn: (r) => (r["_field"] == "value"))
	|> sum()
  |> findColumn(fn: (key) => key._measurement == "caruna_consumption", column: "_value")

caruna_consumption = from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "caruna_consumption" and r["_field"] == "value")

spot_price = from(bucket: "openhab")
  |> range(start: v.timeRangeStart, stop: v.timeRangeStop)
  |> filter(fn: (r) => r["_measurement"] == "spot_price" and r["_field"] == "value")

join(tables:{caruna_consumption:caruna_consumption, spot_price:spot_price}, on:["_time"])
  |> map(fn:(r) => ({
      time: r._time,
      Kulutusvaikutus: ((r._value_spot_price - avg_price[0]) * r._value_caruna_consumption) / total_consumption[0], 
    })
)
1 Like