This is probably a more statistics question, but would like to bring some discussion here as I keep learning new/better ways from the community.
I live in a country where the electricity prices are updated on hourly basis, and the taxation on certain hours are higher (17-21 for example) than other hours (0-6 for example). I usually know tomorrow’s electricity price around noon today. I currently store all today’s hourly prices in OpenHAB, plus:
(1) Lowest price today
(2) Highest price today
(3) Average price today
When I receive tomorrow’s price, I’d like tag each hourly price cheap, normal or expensive based on statistics from previous days (I plan to use a week’s data). This is because electricity price changes with seasons, weather, temperature etc. What I need is to find 2 threshold for cheap and expensive prices. The challenge is that how to best balance the distribution of prices and take that into the formula. For example, we could be in a period with many hours at very low price, in which case I want the cheap threshold skew towards cheaper end. What I have in mind is:
For cheap threshold, I calculate average cheapest price (1) during past 7 days, add margin (maybe standard deviation), and plus/minus the skew
For expensive threshold, I calculate average expensive price (2) during past 7 days, deduct margin (maybe standard deviation), and plus/minus the skew
I’m not very good at statistics, but would like to calculate the skew based on if majority of hours in the past days are expensive or cheap, so I’m thinking maybe I can use the historic data on (3) average price today. Not sure how but any suggestion is appreciated
A simple approach would be to calculate the mean price for today, and the standard deviation for the last 7 days. Anything in the interval (mean +/- stddev) is considered average, below that is low and above is high.
The benefit of using a longer period for the stddev is that if the prices are stable during the day, everything is considered average instead of changing from low to high basen on a change of only a few cents. E.g., the built-in price adaptation in my heatpump doesn’t do this, it only looks at a single day, which has on occasion led to big changes in requested temperature when the prices are varying less than 1¢.
That’s also my consideration - in general I want most prices in “normal” range, cheap and expensive are exceptions so I do something extra. However I don’t want to only look at today’s price, hence the average today is not used in my calculation. For example if in the past week average is much lower, then I would define all hours today normal->expensive, and it will continue for a couple of days until the average kicks in.
Something that I consider:
When price is cheap and my car is below 70%, it sends alert to charge my car
When price is cheap I set heating/cooling running with higher/lower temperature
When price is expensive, I pause heating/cooling
When price is normal, just run the normal routine
I also want to have a range of hours in the same category, so I don’t need to start/stop charging, stop/stop heating frequently.
But what if prices have been low for a week, and then gets high for two days, should heating be totally disabled leading to a cold home? The reason why I use the daily mean is for comfort, the total amount of heating should be the same, just shifted between hours to optimize the cost.
For EV charging it’s totally possible to skip entire day’s however, provide you have enough charge for daily commute etc.
I am working on the same stuff since some months already. I do had/have the same questions and decisions to make.
I decided not to take in “long term” past averages in to count. So “normal”, “high” and “low” only exist in the future (max 32h). Main reason for this, is that on days that prices are cheap all day long, you will be charging at any price of that day (and vice versa) and thus also at the highest price(s) of that day.
I am lucky that the car only needs charging once a week (so charging is done Saturday and Sunday afternoon).
For all other I just look in the future. So dishwasher, dryer, washing machine,… all go to work some hours in the future, when prices are low. Same will be done in the near future for a 10 kW heating element in a water buffer (that heats the house and takes care of hot tap water).
The real question stays : what is “normal”, “high” and “low” (and highest and lowest).
I decided that
normal is the average.
high is above the median (not the average).
low is below the median (not the average).
highest and lowest explain itself.
So, none of the hourly prices are normal. One is highest, one is lowest, the rest is high or low.
Only thing is, I did not have calculated the median yet. If somebody has a rule (or even better, a function build in so a group can be used just like max, min, avg) ?
Maybe open an enhancement request for this on github (core repo). I expect it is more efficient to implement median in a core persistence extension than in a rule.
Charging my car. I only need to charge 2 times a week and hence I’m looking for absolutely cheapest period to charge
Heat/cool my house: here I have to define a temperature range, say 18-24. In summer I lower the temperature when the electricity is cheap, in winter I higher the temperature when the electricity is cheap. During the expensive period I’ll do the opposite. In my country peak cost is significantly higher, so the electricity is always expensive between 17-21. What I normally do is to adjust in the afternoon - turning up the heating/cooling, raise the hot water temperature etc. to prepare for the supply during evening.
Now I start to realised the plan of using electricity in my house is either weekly (1) or daily (2), and hence it makes the problem more difficult to solve. Maybe I should more focus on (2) and just send alert to charge my car when electricity price is at the bottom each week. However even for daily plan, I still need somehow to check if today’s price is high or low comparing to previous and next days, and I don’t need my house to be at best temperature all the time.
I have lots of data besides electricity price which is good thing but also make decision harder.
I also charge in the weekend afternoon, usually Sunday afternoon the cheapest.
However I noticed the pattern can change with season - in Winter time sometimes it’s cheaper to charge during the week.
There are also other factors to consider: the remaining battery and how many hours you need to charge (and do you find enough hours in low price).
Reading this thread for the first time. While I’d like to restate that being able to retrieve the median is a prerequisite to many algorithms - so the feature request is well worth implementing -, it’s no good solution in this use case.
You may know or not that I’ve built and am commercially selling an energy management system that optimizes power cost with dynamic tariffs just like you attempt here.
Thing now is, from my experience of operating the system for more than a year, there’s no price patterns you should rely on. That means you should not deduce from historic data if today’s pricing distribution and in particular the level is high or low. Doing so will drive you into trouble one way or the other. Plus, the median really isn’t a meaningful figure in this case either.
Without that I want to advertise my product, be warned that developing a good algorithm and even more so a proper implementation of that is hard, there’s lots of pitfalls.
I generally agree, and hence my initial thought is to use not longer period of historical data and only use as a reference. It’s still meaningful to some extend as it tells me I should have charged my car 2 days ago as tomorrow price will be higher. However the prices are not random, in the country where I live I see strong correlation of the following:
When it’s windy tomorrow, the el price will be lower. It doesn’t matter if it turns out windy or not, I think they set on price based on forecast. This is by far the strongest correlation.
In the weekend the price is usually lower - 99% case I have observed.
The highest price on any day will be between 17-21 due to peak tax very high
The lowest price on any day with be either between 9-17 or 0-6.
Yes prices aren’t random either but thing is whatever pattern you believe you have identified, it WILL fail at times. And when that day comes, you lose a lot of money.
That’s why you must not decide by patterns but only ever by the fixed price forecast you can get from your provider when the energy stock exchange has fixed it for the next day.
That’s reliable and soon enough for about any ‘moveable’ consumption.
Use the Tibber or the Entso-E binding to get next day prices by 1300 today, and you’re safe.
Note that’s sufficient to make optimum consumption decisions the next day.
You can not forecast if it’s cheaper in 3 days than in 2 or tomorrow so some decisions you cannot base on that like which day to charge your EV.
The best indication for that would be weather forecast but weather forecasts >1 day are so unreliable that you should not rely on those.
Plus there’s non-weather factors to impact the price like vacation, day of week, plus speculation on the energy stock exchange.
Again, don’t bother trying.
Yeah I wouldn’t “predict” how the price look like 2-3 days from now, and I know tomorrow’s price by 13.00 each day. I think my intention when I started discussion here is how to decide on cheap/normal/expensive hours when I know the price tomorrow.
One easy solution is to set some hard thresholds, however as el price are volatile and hence I need to adjust those threshold frequently. So what I plan to do is to look at past week’s price and use statistics for find optimal threshold.
Again, no. Don’t mess with historic prices as reference. Only ever get next day prices using the Entso-E binding.
Then you can compute the cheapest hours and move your moveable consumption there.
It is only about moving within a day (not across days).
You need not determine cheap/expensive in absolute terms, only relative compared to the other hours of the same day. So all you need to do is sort next day hours by price when you get them.
Do you mean that you want to compare tomorrow’s day ahead prices to the past couple of days to concluded if tomorrow’s prices are “cheap”, “normal” or “expensive”?
I’ve been doing spot price optimizations for house heating, heating domestic hot water, charging an electric vehicle, washing dishes and washing laundry for two years now.
Based on my experience during these 2 years, the relation of tomorrow’s prices to the past N days is completely irrelevantti, the only thing that matter is how tomorrow’s prices relate to the future.
There are some loads that I don’t necessarily need to do tomorrow, if the day after that is likely to be cheaper than tomorrow. In Finland the spot prices have a huge correlstion to the wind power but luckily there is a wind power prediction available for the week ahead [1].
If I see from the wind power forecast that there is going to be a huge peak in the amount of wind power production, I might sometimes postpone something compared to what my automation scripts have calculated. But that is tinkering with decimals, in the big picture it’s quite irrelevant - I can easily achieve a -25% discount compared to the monthly market average during the winters and up to -50% discount during the summer.
Oh, forgot to mentioned that the markets have also price futures which are traded. This prices are the best available market guestimate how the prices are gonna look like in the months to come.
Divide by 10 to convert EUR/MWh to c/kWh and multiple with your local VAT.
I believe these are the correct futures, I usually follow them from my power dealer’s mobile app.
