Algorithm to decide on electricity price is high or low based on historical prices

Yes that was my intention as I don’t want to tag the price only based on the day’s numbers, so I use history as reference number.

I also observed a strong correlation between wind and el price here in Denmark, however I don’t know a reliable source to find the data needed, and to quantify forecast is also difficult. I know tomorrow’s price at 13:00,which gives me about 35 hours to plan. I find it sufficient for daily consumptions.

The future prices are indeed useful for prediction, I will study a bit more, thanks for the link!

Hi Markus,

Thanks for the link and documentation, and it looks very similar to what I want to do with daily mandatory consumption. I don’t have direct control of the compressor of my ground heating and hot water production (heated by ventilation system), and hence I have to control them by temperature. Due to the small size of my water tank I cannot avoid starting the compressor during the peak hours, but I’m trying to avoid electrical heater during the peak hours. There are a few challenges/considerations:

  1. I don’t know how long time I need to run the compressor in order to heat the water to the right temperature, as it depends on many factors (consumption pattern, outdoor temperature, indoor temperature, fan speed of ventilation etc.)
  2. Keep the water temperature also consumes energy, so I try to heat the water close to the peak hours (which is also the period we use most of hot water). Have you thought about if the cheapest hours are midnight and you’re at work during daytime, when you come home the temperature drops significantly and you have to spend energy to heat again?
  3. One of motivation to calculate cheap/expensive price is too further adjust temperature when the energy price is significantly low. For example if I know tomorrow’s price is high, or today price is low, then I’d raise the heating temperature today (or lower during summer).

When your talking about hot water, I assume you are talking about domestic hot water i.e. showers, taps etc. Our house has undefloor water heating so it’s important to differentiate these two…

When it comes to domestic hot water, you can really optimize the heating of it if and only if you have sufficiently big tanks. Our ground source heat pump has a 180 liter tank but in addition to this, we have a 300 liter boiler so the total capacity is 480 liters. This is easily enough for our household for more than 2 days. So in our case I can easily heat the domestic hot water when the electricity is cheapest. Our Nibe ground source heat pump can produce logs to an USB stick so I have learnt that it takes about 1 hour per day to heat the domestic hot water. The boiler usually takes 1.5 hours so I find the cheapest 2 hours and use those to heat the water in the boiler.

When it comes to heating the house, overheating today because tomorrow is expensive is most probably going to cost you more money than what you can save, unless you have very, very thick concrete floor which can preserve the heat very well. The air in the rooms does not preserve the heat almost at all so it’s just pure waste of energy until you can store it somewhere.

I’m about to release a new version of the heating optimization algorithm in the next coming weeks which is by far the best I’ve had so far. That uses a bit of overheating if the weather forecast says that there is going to be a big temperature drop. So instead of being reactive and waiting the temperature drop, the algorithm increases the calculated heating need before the temperature drops. And obviously uses the cheapest possible hours to do so. I’ll inform about this new release in this thread when it’s available:

Cheers,
Markus

If intrested, I get the Entsoe prices without the need for api.

What do you mean by “if interested”?
Don’t you use the binding ?

No, see our chat here.

Ok, while I still have a clear mind, let me summarize what I have in mind, for discussion but also for me to remember when I start the project. I have two type of plans, weekly and daily.

  • Weekly plan is for charging my car, I need to charge 1-2 times a week. The best charging hours are between 6PM-6AM. The 2nd best charging hour is between 9AM-4PM (when I work from home or weekend). I’d like to charge during the cheapest hours (not only on a day but also in the recent period).
  1. When I receive the electricity price for tomorrow, I will calculate how long time to fully charge my car based on battery percentage. If the battery is relatively full (80%?) I skip.
  2. I then try to find best charging time between 6PM and 6AM, compare with best avg prices in the pas week (or days), if it’s lower, send myself a notification to charge during the period.
  3. If the car has less battery (30%?) I will find the cheapest price between now and tomorrow and remind myself to charge the car when I arrive at home (I have location based automation so it’s ideal for this)
  • Daily plan for heating/cooling the house + domestic hot water. Some requirements:
  1. Heating is slow so I’d prefer to find a period of at least 3 hours to start the heatpump
  2. The domestic hot water should be produced as close to usage as possible - which means in the afternoon
  3. Both 1 and 2 should be dependent on the weather (temperature, sunny or not). Our house is well isolated, on a sunny winter day there is almost no heat loss, which means I don’t need to keep the heating working very hard

To be continued…

As already mention by not just myself, there are no price patterns you can rely on. Daily pricing, including distribution, will be different every time, and unpredictably so.
The main learning from years of operating energy management systems that optimize for hourly changing tariffs is that trying to apply patterns is doomed to fail in terms of getting the savings you expect from this.

Long story short: you should NOT be making plans.

Instead, you should be retrieving Entso-E prices every day, sort hours of the day by price, and charge your EV during the cheapest hours.
Heat pump heating, same.

Hi, thanks for the comment but I’m not sure where do I rely on patterns? All my decision are based on known info (historic data or published price for tomorrow), without assuming which hours are cheaper than others. The only exception is for charging my car, if the price is not cheap enough I can wait for 2-3 days, in worst case I will be forced to charge when battery is low (and price is expensive) which I have to accept, but in that case I don’t have to charge the car to full and continue to wait for better price, so economic effect is not worse than just charging any day that I need to charge. Or do you see any problem doing so?

I think this only works for consumption that you cannot avoid during the day (heating, hot water as my 2nd point). For charging car I’m not sure this works as you probably don’t connect your car to the charger all the time, and what if the best price is at 8AM and 2nd best price is 8PM? In general I also want to avoid this to start stop compressor on hourly basis (as I’m not sure if it can cause problem longer term).

Comparing with historic data is a pattern.
“I try to find best charging time between 6PM and 6AM, compare with best avg prices in the pas week (or days)”
You assume price ranges are the same every day so if it’s the same as last week and last week it was cheapest it’s cheap this time, too. But it’s not, that’s a fallacy.

This was just the principle. Make it configurable how many cheapest hours you want to charge or heat then adapt this number as needed to heat/charge enough in time.

Thanks for the reply! I understand what you’re saying but don’t see how I could solve this otherwise. It’s the flexibility I have in terms of which day I choose to charge the car, if I need to charge every day then I’d definitely just find the cheapest on the day and do the charging. However if I have flexibility that I can charge on different days, I’d rather to make sure I pick a day that has “relatively” lower price. I agree with you that I make an assumption that in X days I can always find a day that have lowest price. The failure of this assumption is when the electricity price keeps going up every day, in which case I’ll make a loss. However if my assumption is right most of the time I will save money overall.

@somy I have a very similar use case with the car that you have since I don’t need to charge it every day.

I have solved my need with an user interface like this:

Yesterday evening our car was had only 30% so I had a need to charge it for 3 hours. So I updated an openHAB Item to have value 3. The change of this value invoked a Rule which found the cheapest 3 hours last night.

When you have more flexibility in when to charge, you really cannot use historical prices reliably. You can try, but both @mstormi and myself have years of experience with this and let us assure that the historical price patterns are not predicitng future very well.

As I mentioned in my previous post, I use the wind power forecasts as a crystal ball to look further to the future than just 1 day. For example, at the moment it looks like Monday evening and the night between Monday and Tuesday is going to be significantly cheap because there is going to be so much wind power production that the prices will go to zero (if not negative).

Unfortunately this data is not open data in a sense that I could get it to openHAB so I need to manually check the wind power forecast from the website and then I can manually determine when to charge and when not (i.e. when to put the charging need to 0 in openHAB and when to adjust the Item to a higher value).

There’s also the solarforecast binding you could use, depending on how much wind and pv you have in your country.
So if by all means you want to have an indication ahead of time which day to charge your EV,
you can use wind forecast if you use the car during the day and charge at night and use solar forecast if you work from home :slight_smile: Then ON that day, just use the pricing data.

But betting on weather forecast is really really advanced stuff and risky in terms of gettings savings out of it.

PS: reminds me of one of my all time favourite movies.
You know the scene in Back to the future part II when it’s raining cats and dogs and Doc has a look at his wristwatch and tells Marty to wait some more seconds before getting out of the car?

Yep, totally agree. And there are so many factors that the algorithm can’t possibly know e.g. I dont need the car tomorrow but I need 150 kilometers the day after tomorrow. So it’s just so much easier to leave the final decision to a human brain (and easy UI like in my example above)

Okay, concrete example from the upcoming weekend.

I know that I will drive at least 250 km during the weekend.

I see tomorrow nights prices and can say that they are moderately cheap. Based on the wind power forecast I can see that the night between Sat and Sun is most likely going to be more expensive. But the night between Sunday and Monday is most likely going to be much cheaper than these two upcoming nights.

Conclusion: I charge my car only 1 hour this night. That will be enough for the weekend rides. This means I have to charge on the night between Sunday and Monday no matter what but I’m okay with this because my confidence level based on my previous experience is high enough that the assumptions I’m making are good enough.

If I would like to automate this, I would need the following:

  • sophisticated machine learning model with shitloads of training data to train the model for correlations *)
  • I would still need to feed the driving needs manually one way or another so that the result would be good

→ it’s so much simpler and gazillion times morecreliable to click the plus/minus button to value 1 for next night and to value 3 for the night between Sun and Mon.

*) There is actually a machine learning model available in Finland for forecasting the prices for the next 5 days, see https://sahkovatkain.web.app/ (use Google translate to understand the Finnish texts).

This model was originally scraping the Foreca wind power predictions that I check from the web. With that data, the model was able to reach like 98% accuracy with the price predictions. Foreca (the private weather company) did not appreciate the scraping of their wind power forecast data and forbid the usage so now the machine learning model is using normal weather forecast data available as open data. And as you can see from the graphs, the accuracy is nowhere close to the 98% anymore → it’s pretty much useless at least for me.

This example illustrates how difficult the prediction is even for very sophisticated machine learning models. You need super, super accurate data, and shitloads of it to train the model. There’s absolutely no way how you could do that with simple “if value is this, then that” rules.

Markus

Still continuing on the money aspect. I just by coincident analyzed our electricity bills from the past 2 years. By scheduling the energy consumption, we got 725 EUR discount from the electricity bill. This is roughly 25% percentage wise.

From the discount, I would say at least 90% is based on day-ahead optimizations (optimizing tomorrow after the prices have been published) with the spot-price-optimizer I linked above. By far less than 10% is achieved with these human-brain based decisions to longer future like the car example above.

Exactly. With an EV it’s not just the production price pattern but also the consumption pattern
that is just not predictable.
And with car sharing, electric bikes/scooters, improvements in public transport and fast-lane life in general, it’s getting ever less predictable which means of transport you use and when.

About the same for me.
Here’s a study from well-renowed German think tank on dynamic tariffs
(there’s also an english version, just search).
They forecast 10% of household consumption to be scheduleable, heat pumps and EVs these are, and 600 EUR of annual savings for an average consumer household.

I find it really interesting to read your posts here @mstormi and @masipila, you obviously have put a lot of time and thought into this area. I would also like to hear your thoughts on domestic heating, which is my main focus of optimization. For EV charging, it’s quite straight forward to say “I need to charge for X hours between Y and Z, find the cheapest ones”.

But for heating it’s harder to fulfill the heating needs of a house in just a few hours, and the alternatives are not heat/don’t heat but rather heat more/less, e.g., by raising or lowering the thermostat setpoint. The question is, how much should the price affect the setpoint? And even if all the energy the house needs for heating could be provided in, say, 4 hours, you also need to take comfort into account. Would I be okay with raising the temperature at 23 degrees and then gradually sink to 19 to keep an average of 21? Have you solved this in a good way?

As an additional complexity factor, my heatpump has a maximum effect of about 1.5kW, which gives about 4-5kW of heat (COP is stated as 4.5, but that is at more optimal frequencies, not at max output), but when that is not enough the electric add-on kicks in, adding up to 3kW of energy for the same amount of heat. This is obviously something to avoid happening as much as possible, but would certainly happen if I would only run heating for some of the hours of the day during spring/autumn (it’s inevitable when outside temp is < 0, and I don’t need any heating at all in summer).

Would love to hear your thoughts on this as well.

1 Like

I have a ground source heatpump and 20 m3 of concrete floor where I can “store” or “charge” or “load” the thermal energy, which means our house can survive several hours without the compressor running.

This is very, very different game compared to air-to-air heat pumps where you don’t have pretty much anything that would preserve the energy. Plus, the COP is different at different outdoor temperature, whereas with a ground source heat pump it’s not a significant factor.

For the ground source heat pump use case I managed to develop a kick-ass algorithm which balances the cost optimization with comfort in a very nice balance. (I simulated whole last winter with it and I was very happy with the results). I’ll publish this version hopefully soon, need to polish and harden the code still a bit when I have time. Curling season is on again and I have 7 ice sessions this week :slight_smile: