Control a water heater and ground source heat pump based on cheap hours of spot priced electricity

Hi @Marcus_Carlsson !

The error message here means that you are not able to read the forecaster temperatures from the the Finnish Meteorology Institute’s weather forecast. You most probably do not want to read the weather forecast from Finnish Meteorology Institute anyway?

To narrow things down, you can temporarily modify the rule script so that you don’t calculate the number of hours based on the weather forecast. Instead, just put there a static value like 5 when you’re debugging this:

dh = require('kolapuuntie/date-helper.js');
nibe = require('kolapuuntie/nibe.js');

start = dh.getMidnight('start');
stop = dh.getMidnight('stop');

// Comment out these two lines that fetch the weather forecast and calculate how many hours are needed.
// t = nibe.getForecastTemp(start, stop);
// n = nibe.calculateNumberOfHours(t);
// Temporarily define the number of hours as five.
n = 5;
nibe.determineHours(start, stop, n, 3, 0.1);

You can add a couple of debug messages so that you can see from your logs what is going on.

Add the debug lines like this to get the complete XML response written to your log:

function makeApiCall(start, end, area, token) {
   [...]
    priceXml = http.sendHttpGetRequest(url, 10000);
    console.debug('Complete XML response:');
    console.debug(priceXML);
    return priceXml;

Add these debug lines to see if the XML to JSON transformation works:

function preparePoints(priceXml, tax) {
   [...]
	const prices = JSON.parse(transformation.transform('XSLT', 'xml2json.xsl', priceXml));
        console.debug('Complete JSON:');
        console.debug('prices');

These should help you to narrow down what’s going on…

Cheers,
Markus

@Marcus_Carlsson one more thing related to debugging.

If (when) you modify the javascript files, you need to “refresh” your rules so that openHAB will use the updated versions of the js files. This can be done as follows:

  • Go to edit the rule that uses the js file
  • Save the rule
  • Run the rule

If you don’t go to the edit mode and save the rule, openHAB will use the old version of the JS file it had read to memory before.

Hi

Thanks for your efforts to help me out, I managed to get some xml output after changing console.debug to console.log, but somewhere in xml2json there must be a problem, I don’t se any output from extensive console.log in xml2json function. Meybe it wasn’t right to change < to

 &lt;

In the xml2jsonfile, I downloaded an copy an pasted xml2json file again and now it works better. However influx db doesn’t contain much data spotprice came up but only one price point

2022-07-27 15:06:34.088 [INFO ] [org.openhab.automation.script       ] - entsoe.js: Making an API call to Entso-E API...
2022-07-27 15:06:34.591 [INFO ] [org.openhab.automation.script       ] - Complete XML response:
2022-07-27 15:06:34.600 [INFO ] [org.openhab.automation.script       ] - <?xml version="1.0" encoding="UTF-8"?>
<Publication_MarketDocument xmlns="urn:iec62325.351:tc57wg16:451-3:publicationdocument:7:0">
	<mRID>5481dbea73224a0e8c4cd7ec6b1694af</mRID>
	<revisionNumber>1</revisionNumber>
	<type>A44</type>
	<sender_MarketParticipant.mRID codingScheme="A01">10X1001A1001A450</sender_MarketParticipant.mRID>
	<sender_MarketParticipant.marketRole.type>A32</sender_MarketParticipant.marketRole.type>
	<receiver_MarketParticipant.mRID codingScheme="A01">10X1001A1001A450</receiver_MarketParticipant.mRID>
	<receiver_MarketParticipant.marketRole.type>A33</receiver_MarketParticipant.marketRole.type>
	<createdDateTime>2022-07-27T13:06:34Z</createdDateTime>
	<period.timeInterval>
		<start>2022-07-27T22:00Z</start>
		<end>2022-07-28T22:00Z</end>
	</period.timeInterval>
	<TimeSeries>
		<mRID>1</mRID>
		<businessType>A62</businessType>
		<in_Domain.mRID codingScheme="A01">10Y1001A1001A46L</in_Domain.mRID>
		<out_Domain.mRID codingScheme="A01">10Y1001A1001A46L</out_Domain.mRID>
		<currency_Unit.name>EUR</currency_Unit.name>
		<price_Measure_Unit.name>MWH</price_Measure_Unit.name>
		<curveType>A01</curveType>
			<Period>
				<timeInterval>
					<start>2022-07-27T22:00Z</start>
					<end>2022-07-28T22:00Z</end>
				</timeInterval>
				<resolution>PT60M</resolution>
					<Point>
						<position>1</position>
						<price.amount>2.65</price.amount>
					</Point>
					<Point>
						<position>2</position>
						<price.amount>2.16</price.amount>
					</Point>
					<Point>
						<position>3</position>
						<price.amount>2.26</price.amount>
					</Point>
					<Point>
						<position>4</position>
						<price.amount>2.48</price.amount>
					</Point>
					<Point>
						<position>5</position>
						<price.amount>3.23</price.amount>
					</Point>
					<Point>
						<position>6</position>
						<price.amount>5.23</price.amount>
					</Point>
					<Point>
						<position>7</position>
						<price.amount>6.91</price.amount>
					</Point>
					<Point>
						<position>8</position>
						<price.amount>50.05</price.amount>
					</Point>
					<Point>
						<position>9</position>
						<price.amount>200.05</price.amount>
					</Point>
					<Point>
						<position>10</position>
						<price.amount>206.57</price.amount>
					</Point>
					<Point>
						<position>11</position>
						<price.amount>209.92</price.amount>
					</Point>
					<Point>
						<position>12</position>
						<price.amount>209.98</price.amount>
					</Point>
					<Point>
						<position>13</position>
						<price.amount>209.97</price.amount>
					</Point>
					<Point>
						<position>14</position>
						<price.amount>197.19</price.amount>
					</Point>
					<Point>
						<position>15</position>
						<price.amount>204.25</price.amount>
					</Point>
					<Point>
						<position>16</position>
						<price.amount>200.09</price.amount>
					</Point>
					<Point>
						<position>17</position>
						<price.amount>205.73</price.amount>
					</Point>
					<Point>
						<position>18</position>
						<price.amount>214.49</price.amount>
					</Point>
					<Point>
						<position>19</position>
						<price.amount>220.97</price.amount>
					</Point>
					<Point>
						<position>20</position>
						<price.amount>220.96</price.amount>
					</Point>
					<Point>
						<position>21</position>
						<price.amount>214.91</price.amount>
					</Point>
					<Point>
						<position>22</position>
						<price.amount>212.83</price.amount>
					</Point>
					<Point>
						<position>23</position>
						<price.amount>207.25</price.amount>
					</Point>
					<Point>
						<position>24</position>
						<price.amount>179.56</price.amount>
					</Point>
			</Period>
	</TimeSeries>
</Publication_MarketDocument>
2022-07-27 15:06:34.627 [INFO ] [org.openhab.automation.script       ] - entose.js: transforming XML to JSON and parsing prices...
2022-07-27 15:06:34.828 [INFO ] [org.openhab.automation.script       ] - Complete JSON:
2022-07-27 15:06:34.835 [INFO ] [org.openhab.automation.script       ] - prices[object Object]
2022-07-27 15:06:34.847 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-27T22:00:00.000Z 0.265 c/kWh
2022-07-27 15:06:34.854 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-27T23:00:00.000Z 0.21600000000000003 c/kWh
2022-07-27 15:06:34.861 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T00:00:00.000Z 0.22599999999999998 c/kWh
2022-07-27 15:06:34.868 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T01:00:00.000Z 0.248 c/kWh
2022-07-27 15:06:34.875 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T02:00:00.000Z 0.323 c/kWh
2022-07-27 15:06:34.882 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T03:00:00.000Z 0.523 c/kWh
2022-07-27 15:06:34.890 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T04:00:00.000Z 0.6910000000000001 c/kWh
2022-07-27 15:06:34.897 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T05:00:00.000Z 5.005 c/kWh
2022-07-27 15:06:34.904 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T06:00:00.000Z 20.005000000000003 c/kWh
2022-07-27 15:06:34.911 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T07:00:00.000Z 20.657 c/kWh
2022-07-27 15:06:34.918 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T08:00:00.000Z 20.991999999999997 c/kWh
2022-07-27 15:06:34.926 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T09:00:00.000Z 20.997999999999998 c/kWh
2022-07-27 15:06:34.933 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T10:00:00.000Z 20.997 c/kWh
2022-07-27 15:06:34.940 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T11:00:00.000Z 19.719 c/kWh
2022-07-27 15:06:34.947 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T12:00:00.000Z 20.425 c/kWh
2022-07-27 15:06:34.954 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T13:00:00.000Z 20.009 c/kWh
2022-07-27 15:06:34.961 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T14:00:00.000Z 20.573 c/kWh
2022-07-27 15:06:34.968 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T15:00:00.000Z 21.449 c/kWh
2022-07-27 15:06:34.976 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T16:00:00.000Z 22.097 c/kWh
2022-07-27 15:06:34.983 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T17:00:00.000Z 22.096 c/kWh
2022-07-27 15:06:34.990 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T18:00:00.000Z 21.491 c/kWh
2022-07-27 15:06:34.997 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T19:00:00.000Z 21.283 c/kWh
2022-07-27 15:06:35.004 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T20:00:00.000Z 20.725 c/kWh
2022-07-27 15:06:35.011 [INFO ] [org.openhab.automation.script       ] - entsoe.js: 2022-07-28T21:00:00.000Z 17.956 c/kWh
2022-07-27 15:06:35.017 [INFO ] [org.openhab.automation.script       ] - influx.js: Preparing to write points to the database for spot_price
2022-07-27 15:06:35.200 [INFO ] [org.openhab.automation.script       ] - influx.js: Points successfully saved for measurement spot_price

The reason why you’re not seeing more than one value is that your Influx Query is searching for data for “past 1 hour”. Change the range to include tomorrow and you’ll see tomorrow’s spot prices :slight_smile:

Thanks again for all your efforts of helping men out. I will be out travelling for next couple of days but wilöl have spotprice fetching rule up an running every hour for the moment I did indeed have more data if I set a different filer :slight_smile: .

I did get FMI forecast script to runt but it didnt write an mesurement, and I beleive that thats the reason why nibe and waterboiler script fails theres no temperature forecast. I will probably convert this to my local realtime temperature but thought I will get this nice example up and running before I start modifying to much. And I will also integrate this with my Easee EV wallbox.

The waterheater rule is not depending on the weather forecast at all. It should work if/when you have the spot prices stored in your influxDB.

dh = require('kolapuuntie/date-helper.js');
wh = require('kolapuuntie/waterheater.js');

start = dh.getMidnight('start');
stop = dh.getMidnight('stop');

wh.determineHours(start, stop, 4);

The last row means that “between the start midnight and stop midnight, find a 4 hour window when the spot prices are lowest”.

Our hot water consumption does not depend on the outside weather. Four hours should be easily enough to heat the water for the whole day.

  • 1 kWh will be required to heat 864 liters of water by 1 degree
  • My water heater has a 300 liter tank so it will require 300/864 = 0,347 kWh to raise the water temperature by 1 degree
  • My water heater is 3 kW so letting it run for 4 hours means 12 kWh
  • Which means that 12 kWh can raise the water temperature by 12/0,347 = 34,5 degrees

This is easily more than enough because the water heater is “behind” or “after” the Nibe ground source heat pump which pre-warms the water in its 180 liter integrated tank to 48 degrees so the starting point for the water heater is barely ever under 40 degrees.

I’m currently planning to install a couple of DS18B20 temperature sensors to the outer shell of the tank of the water heater so that I could measure how cold / warm the water is already and then determine if I need to allow 1, 2, 3 or 4 hours of heating time for the water heater.

1 Like

Here’s what I’ve been able to achieve so far. Blue bars is my consumption which I’m able to fetch from the power grid operator and green line are the spot prices. You can see nice consumption peaks when the prices are lowest.

July 26 and 27 were hugely exceptional days in Finland in terms of the spot prices as there was a lot of wind power and at the same time the Baltic countries had planned maintenance in their power grid which meant that they could not import electricity from Finland which they normally do. I think we washed laundry something like 4 or 5 times, cleaned the oven with pyrolysis and so on.

My electricity contract is not directly bound to the Nordpool spot prices (i.e. spot price + margin). Instead, I have a contract which is 14.8 c / kWh +/- “consumption factor”. The consumption factor is calculated as follows:

consumption factor = (A-B) / E, where

  • A is the sum of consumption * spot price over every hour of the month
  • B is the total consumption * average price in Nordpool for the month
  • E is the total consumption of the month

In other words, if I use power on hours which are cheaper than the month’s average spot price, the consumption factor will be negative and decrease the price that I pay of the whole month’s power. With these optimizations this means that the 14.8 c/kWh is effectively a ceiling price that I will have to pay under any circumstances, even if the Nordpool would be at 100c / kWh next winter.

Here’s a 7 day trend for the consumption factor. If I would be able to keep this trend, I would pay 14.8 - 5.9 c / kWh but it’s not going to be that much negative because the July 26-27 was so highly exceptional. But anyway, this I will have a double digit discount to the power bill, which is is quite nice :slight_smile:

Hi

It seems like I’m having trouble to write measurments to influxdb other OH dta appears to be right but this data just doesn’t get in there. I’ve verified ip and api token org and bucket name but nothing really changes the game. Influx db version 2.3.0 snapshot.

Do I understand you correctly that

  • You are able to write the spot price points to your InfluxDB
  • But you are not able to write the control points (when the heating should be on / off) to the same InfluxDB?

Cheers,
Markus

Hi I was perhaps unclear, nothing writes for a certain I only got one days of spot price data even thoug logging looks good. I activated a log for whats written to influx db an theres a value=some price anf a blank anf a epoch time (maybe)

I’m willing to help you but you need to be more specific please.

Let’s do this one step at a time.

If you run your “fetch spot price” rule right now, do you get tomorrow’s spot prices stored to your influxDB?

Hi @masipila

First of all thanks for the great work on this topic. Been testing these rules for a few weeks and they seem to work well. I did however have a possibly similar problem as @Marcus_Carlsson of 2-3 null values in results. Tracked it down to the influxdb query that in my setup seems to add three lines of annonations before the header line in the results. See image

Did a workaround in the influx.js ParCSV function by adding a some more lines to be skipped and has worked fine since. Maybe there is a better way to remove annonations, but this has worked so far.

// Ignore the header row and last empty rows. 
    //for (let i = 1; i < n-2; i++) { //Added 3 more lines because of annonations
    for (let i = 4; i < n-0; i++) {

I agree, grat work. Lucky for me that today with a fresh install openhab in docker container most things really works, but when running water heater rule I get an error as below. And that may be related to lack of waterhetaer_control in Influxdb, weatherrforecast is there and nibe control is there but not waterheater_control in influxdb

influx.js: query did not return any data!

And this might have been a real confusion from me, I had combined hourly waterheater script and waterheaterscript. Now all items exist in influxdb, just have to wait until tomorrow and se if boiler turns on/off.

1 Like

Great that you were able to sort it out! If you don’t want to wait until tomorrow, you can re-run the rule that determines the control values, i.e. this:

dh = require('kolapuuntie/date-helper.js');
wh = require('kolapuuntie/waterheater.js');

start = dh.getMidnight('start');
stop = dh.getMidnight('stop');

wh.determineHours(start, stop, 4);

Change the argument of the last row to be 24 hours instead of 4 hours. This will cause your boiler to turn on on the next full hour.

Hi again

This morning it all worked out as it supposed to, so it all must have been som missmatch in first installation in docker.

But wouldn’t you need a script for turning heatpump on/off? similar to the waterheater hourly script?

Yes, absolutely. Thanks for pointing out that it was missing from the Solution of #13. I have now added this simple snippet there.

Cheers,
Markus

@masipila

Nice solution you have created :slight_smile:

I’m facing this issue when running “Fecth Spot Price”

entsoe.js: Exception parsing spot prices: Cannot read property "period.timeInterval" from undefined

Do you have any idea ??

I just wanted to give a short tipp, if you are using Node-RED I would suggest to look into the awesome PowerSaver Nodes

@Nanna_Agesen it means that you did not receive a valid XML response from the Entso-E API. Most probably because you did not update your personal access token.

I realized this (that you need your personal access token) was not properly documented in my comment marked as the solution.

I have now updated all js files in the comment marked as the solution with better documentation. I also made some changes to the rule actions. Please delete the rule actions you had previously, re-load all js files and try again.

Note for other users such as @Marcus_Carlsson and others who had previously got their setups working: these updated js files are not compatible with your existing rule actions because I harmonized the architecture of the code slightly. If you already have your stuff working, you can keep the js files and rule actions that you already have. There are no new features or anyhting like that, just small improvements to the structure of the code.

I also added some photos from the hardware.

1 Like