Math experts wanted: analyse humidity curve and distinguish between bath and shower

Yes, it is. Using KNIME (or similar tools) and ‘historical data’ one could train a model and let a predictor decide whether the current event is ‘shower’ or ‘bathtub’ or something else. By using time series analysis it would even be possible to predict the next shower/bathtub event. :slight_smile:

2 Likes

I’m not a math expert, but looking at your graph I see that the shower peaks and then drops sharply, whereas the bath does not. I feel like that time difference is your best indicator of shower or bath, so I would:

  1. Trigger your rule when the humidity rises above a certain threshold
  2. Set a timer within the rule, equivalent to how long it takes for the humidity to stop dropping
  3. After the timer has elapsed, check if the humidity has dropped below a second threshold. If so, turn on the towel heater.

I’m also wondering if you have a separate bathtub and shower, or if it’s a single bath/shower. If the former, you could put in a second humidity sensor with one closer to the bath and one closer to the shower. Since the steam from the shower is more contained, the sensor close to it should peak faster than the one close to the bath. This should also minimize the false positives from brushing your teeth.

BTW, if you’re running the water while brushing your teeth (which is the only way I can imagine the humidity rising), I encourage you to turn off the tap when you start brushing. That should also decrease false positives while conserving water. :wink:

@powerpolly
We would need to see more historical data to recommend a simple and reliable algorithm … If you are willing to post machine readable data (timestamp/humidity pairs spanning a week/a month), we could have a deeper look into your problem.

Edit:
A more general approach would be to use a realtime peak detector (see, e. g., algorithm - Peak signal detection in realtime timeseries data - Stack Overflow).

1 Like

Hi, i think what you would like to do is differential analysis.
You’ll need some samples in time to find the “direction” and the amount of change over a defined period.
Leta say you store 3 samples with an interval of 30sec:

var Humid1; //most recent sample
var Humid2; //sample 30sec older
var Humid3; //sample 60sec older
var D;             //Derative weight

You can get the direction and amount of change:

Change = (humid1 * D) + (humid2 * -2*D)+ (humid3 * D);

If you have some ventilatiesystemen you may use it, for example, as:

FanSpeed += Change;

You probably can see the difference between bath and shower by the weight of Change (shower will be much higher than bathing).

If(Change > x) showering;
else if(change>y) bathing;

You can just fill the formule in in excell to get some feeling with it.

You couls also try it with 2 samples so you could use something like the item oldstate and item newstate in rules:

var Humid1; //item new state
var Humid2; //item old state
var P = 1;
var BathingLevel =20;.    //Shower level
var ShowerLevel=10;.     //Bathing level

Change = (humid1 * P) - (humid2 * -P);
FanSpeed += Change;

If(Change > ShowerLevel) 
    showering;
else if(Change > BathingLevel) 
    bathing;

Or for the 3 samples something like

var Humid1;.  //insert most recent sample
var Humid2;.  //insert sample 30sec older
var Humid3;  //insert sample 60sec older
var D = 1;       //Derative weight
var BathingLevel =20;.    //Shower level
var ShowerLevel=10;.     //Bathing level

Change = (humid1 * D) + (humid2 * -2*D)+ (humid3 * D);

FanSpeed += Change;  //need some limits here 

If(Change >ShowerLevel )
      showering;
else if(change>BathingLevel ) 
      bathing;

That’s how I should solve it mathematical, but i’m not sure how to get the historical data.

great to see so many people trying to solve my use case, many many thanks for that!!!

here`s a csv export of the humidity sensor series over the last month:

a temp sensor beyond the bathtube is definitely worth to try, other approaches like sensors on the hot water supply won`t work here as water withdrawal for a second bathroom or cooking would negatively influence the bath rules then.

the hum sensor position is fine. it`s an AM2301 at the highest possible position in the bath, right beyond the ceiling. this temp&hum sensor plus a motion sensor are connected to a magichome led controller. the cheap sensor might not deliver the most accurate values, but I only care on seeing humidity changes. shower and bathtube are in opposite corners of the bathroom.

have kids? :rofl: but great idea, I`ll add brush teeth detection to the rules and advice alexa to remind my daughter to turn off the tap.
Alexa is already part of my bathroom rules and starts relaxing music when bath is detected… raising WAF.

I like the machine learning approach, but I guess this is some levels above my skills… I would be the perfect husband if I could predict what my wife plans to do :stuck_out_tongue_winking_eye:

detecting the peak is too late for the shower use-case. humidity can still raise when shower finished, but I need the towel heater start while I’m showering. so it`s the climb in humidity that needs to be triggered.

interesting idea, will do some experiments with the derivation approach.

currently I simply compare old and current values. a shower will raise humidity by 14-17%RH per minute. a bath will raise it approx 5%RH per minute.

this works: LED_Bad_hum.historicState(now.minusSeconds(30)).state as DecimalType

Don’t go there… plans always change…

The humidity signal is quite noisy. Just for reference (x-axis: days):

Higher resolution:

Event ‘shower’:

Event ‘bathtub’:

gnuplot commands used:

set autoscale
#set xrange [2.8:2.9]
set xrange [1.25:1.55]
x0=NaN
y0=NaN
plot 'ts1.csv'  every ::1::4000 using (dx=$4-x0,x0=$4,$4-dx/2):(dy=$5-y0,y0=$5,dy/dx/1000) w l t 'dy/dx/1000',\
        'ts1.csv'  every ::1::4000 using 4:5 with lines

Edit:
High resolution SVG file: test1

What kind of sensor do you use and what’s the update frequency?

it’s an AM2301 connected to a magichome controller with tasmota flashed firmware. reporting every 5min unless motion is detected, then I go down to 10sec.

switch LED_Bad_motion.state {
                case ON: {
                    mqttActions.publishMQTT("cmnd/LED_Bad/Teleperiod", "10")
                    teleperiod = createTimer(now.plusMinutes(8), [ |
                        if (LED_Bad_motion.state == ON) teleperiod.reschedule(now.plusMinutes(2))
                        mqttActions.publishMQTT("cmnd/LED_Bad/Teleperiod", "300")
                    ])

With a simple threshold on the derivative it should work in my opinion? Maybe a little trial and error but it seems pretty clear,

Well, you have more options like

  1. some (pre) averaging over some samples.
  2. have a higher amount of samples for the derative function (5 instead of 3)
  3. you could double the time between samples (30sec → 1 min) in combination with setting the D value to 0.5.

Best regards

Implementation in Lua:

-- Please note: x values must be equidistant!
-- x scale: days

threshold_shower = 20  -- shower, if 1st derivative is above threshold_shower

time_start = 1634374075761000000

x0 = math.huge / math.huge
y0 = math.huge / math.huge


for l in io.lines( 'ts.csv' )
 do

  _, t, h = string.match( l, '(.*),(.*),(.*)' )

  t = ( t - time_start) / 1E9 / 86400 -- convert timestamp to days

  dx = t - x0
  x0 = t

  dy = h - y0
  y0 = h

  change = dy / dx / 1000 -- scale change for gnuplot ...

  if change > threshold_shower
   then
	print( t, h, t * 86400 )
   end

 end

Output (grouped for better readability, columns: current time in days, humidity, current time in seconds) and discussion:

2.8380930324089 66.6 245211.23800013
grafik

OK.

4.4009757407407 80.7 380244.304
4.4010916898163 84.1 380254.32200013
4.4012079050933 86.5 380264.36300006

grafik

OK

grafik

Not detected, below threshold.

8.4040727314815 72.6 726111.884
8.4044655787052 80.7 726145.82600013
8.4046975694459 85.4 726165.87000013

grafik

OK.
8.8431113888889 70.4 764044.824
8.8432276157422 73.6 764054.86600013
8.8433431712978 76.3 764064.85000013
8.8434590046311 78.7 764074.85800013
8.8435047453719 79.7 764078.81000013

grafik

Probably a shower event - OK.

10.845959444444 71.3 937090.896
10.846074988427 73.7 937100.87900006

grafik

OK.

11.887368935185 68.1 1027068.676
11.887484293982 72.1 1027078.6430001

grafik

Inconclusive - stange event.

12.958152453704 69 1119584.372
12.958268275464 72.6 1119594.3790001
12.958384525464 76 1119604.4230001

grafik

First event below threshold, second event OK.

14.051926886575 66.8 1214086.4830001

grafik

Detected as shower event. OK?

16.15609494213 69.3 1395886.6030001
16.15621042824 71.7 1395896.5809999
16.156326354166 74.1 1395906.5969999
16.1564425 76.6 1395916.632

grafik

Bathtub event followed by a shower event. OK.

grafik

Below threshold.

17.862331319446 66 1543305.4260001
17.8624475 68.6 1543315.464
17.862898009259 76.7 1543354.388
17.863013761573 79.1 1543364.3889999

20.177188668981 65.2 1743309.1009999
20.177304050927 68.2 1743319.0700001
20.177420277778 71.4 1743329.112
20.17751244213 74.2 1743337.0750001

20.867646504631 62.9 1802964.6580001
20.867761886575 65.3 1802974.6270001
20.867877650462 67.7 1802984.6289999
20.867993888889 70.6 1802994.672
20.868109247686 73.5 1803004.6390001

23.845237418981 61.3 2060228.5129999
23.84570087963 69.4 2060268.556
23.845816180557 72.1 2060278.5180001
23.84593212963 75 2060288.536
23.846047789351 77.4 2060298.5289999

24.88141204861 65.4 2149754.0009999
24.881527835647 68.6 2149764.0049999
24.881644594907 71.3 2149774.0929999
24.881759421298 74.4 2149784.0140001
24.881875277778 76.9 2149794.024
24.88199164352 79.8 2149804.0780001
24.882107187499 82.4 2149814.0609999
24.882223148148 84.8 2149824.08
24.882339525464 87.2 2149834.1350001
24.882454108797 89.6 2149844.0350001

25.931703368056 67.3 2240499.1710001
25.931819282409 70.3 2240509.1860001

28.22510787037 69.2 2438649.32
28.225223726853 73.7 2438659.3300001
28.225339768519 78.5 2438669.356
28.22545539352 82.5 2438679.3460001
28.225571006945 85.1 2438689.3350001

29.065296423612 62.8 2511241.6110001
29.06587537037 66.4 2511291.632
29.065990173612 69.6 2511301.5510001
29.066338425926 76.2 2511331.64
29.066453877316 78.6 2511341.6150001
29.06648832176 79.3 2511344.5910001
29.06683545139 86.1 2511374.5830001

Quite a nice result …

2 Likes

my initial approach was to lower the sample rate to identify hum rise more quickly. unfortunately the lower the sample rate is, the lower is also the difference between samples and therefor false reports were more often. I’ll give 2 & 3 a try.

How about water sensor in the shower? - The flooding ones

WOW, that’s clearly an extrem mathematical approach :+1:
I’m impressed by the hit rate you achieved in a couple of hours, I’m spending days if not weeks on this problem already. never heard about Lua before, let me see if I find out how to feed live data into this script. might cost me another weeks, but it’s all about a hobby for me :rofl: thanks!!

Visualization of the points of detection (threshold 20):

For threshold 20:
test1

CNP due to upload limit …

Threshold 10 is too low - false positives:

SVG files for threshold 15 and 10:
Threshold_15_10.zip.txt (526.2 KB)

gnuplot source code:

set autoscale
set terminal svg size 15000x500
set output 'test1.svg'
set xtics 0, 0.1
set style circle radius 0.025
#set xrange [2.8:2.9]
#set xrange [1.25:1.55]
x0=NaN
y0=NaN
#plot 'ts1.csv'  every ::1::4000 using (dx=$4-x0,x0=$4,$4-dx/2):(dy=$5-y0,y0=$5,dy/dx/1000) w l t 'dy/dx/1000',\
#        'ts1.csv'  every ::1::4000 using 4:5 with lines 
plot 'ts1.csv'   using (dx=$4-x0,x0=$4,$4-dx/2):(dy=$5-y0,y0=$5,dy/dx/1000) w l t 'dy/dx/1000',\
'ts1.csv'  using 4:5 with lines,\
'th20.txt' using 1:2 with circles lc rgb 'red' t 'shower event'

Hi, actually the rise between samples will become larger (relativeChange / time). Increase the time will increase the relativeChange. For example twice the time.

To retain the amplitude you half the Derative value (D/2) which will also resuce the impact of noise (half).

If that’s the case, I think we’re putting too much focus on the bath when the question is, “has the shower turned on?” A humidity/temperature sensor in/above the shower would spike almost immediately due to the steam funneling up to the ceiling. The bath doesn’t actually matter since you don’t want to trigger any rules based on it.

Nope, but I’m sure that I ALWAYS left the water running when I was a kid (keeping in mind that we didn’t think about water conservation nearly as much in the 80’s). I had to retrain myself to turn off the tap when I started brushing, and now it seems weird to leave it on. So, there’s hope for your daughter. :wink:

2 Likes

in the initial graphs you dont specify when the bath/shower ends and it might be helpful. how long do you need the towel heated for during a shower? it’s obviously pointless to run it anytime after the end of the shower but maybe its ok to run it for a little at the start of a spike in humidity but switch off when it realizes its a bath instead then focus on predicting the end of the bath.

i’m curious if the drop in humidity during shower is caused by fan or end of shower. can you note when fan turns on also as it might be useful.