Math experts wanted: analyse humidity curve and distinguish between bath and shower

[powerpolly (Matthias P)]

I`ve installed a humidity sensor in my bathroom and want to differentiate if someone is bathing or taking a shower. having a shower results in a steep rise of humidity while a bath shows a more flat increase. the attached screenshot show shower and bath times…
to get a more accurate messurement I shorten the samples of the humidity sensor to 10sec intervall when movement is detected in the bathroom.
the challenging thing is that I need to differentiate between shower and bath as quick as possible as I want to start a towel heater while having a shower.
the rules I attached at the bottom have a high failure rate. sometimes the rule triggers for a bath when I just brush my teeth.
did anybody ever tried something similar or has some more mathematically approached rules for this use case?

rule "shower oder bathtube"
    when
        Item LED_Bad_hum changed
    then
        val Number Badhum_t1 = LED_Bad_hum.historicState(now.minusMinutes(2)).state as DecimalType
        val Number Duschhum_t1 = LED_Bad_hum.historicState(now.minusSeconds(30)).state as DecimalType
        val Number Badhum_t0 = LED_Bad_hum.state as DecimalType
        val Number Baddiff = Badhum_t0 - Badhum_t1
        val Number Duschdiff = Badhum_t0 - Duschhum_t1
        if (Baddiff >= 10 && duschcounter == 0 && Badewanne.state == OFF) { 
            bathtube.postUpdate(ON)
            badecounter = badecounter + 1
            createTimer(now.plusMinutes(10), [ |
                    ...do something
                ])
            sendNotification("xxx", "someone takes a bath")
            }  
        else if (Duschdiff >= 8) {    
                duschcounter = duschcounter + 1
                Dusch_counter.sendCommand(duschcounter)
                Dusche.sendCommand(ON)
                sendNotification("xxx", "someone takes a shower")
                if (LED_Bad_t.state <= 23 ) {
                    Bad_Radiator.sendCommand(ON)
                    createTimer(now.plusMinutes(5), [ |
                        Bad_Radiator.sendCommand(OFF) 
                    ])
            }
        }
    end

[BobMiles]

Hi,

I don’t have exactly the same, but I do have a shower detection. Additionally to the humidity sensor, I use a self-made readout sensor on our gas meter to figure out if water is heated in our boiler. That cut away a lot of false positives.
First of all I would start calculating the “derivative” of your humidity level and post that value into a new item. Then look at that graph for shower and baths and see if there’s a threshold you can set.

How does your humidity rise when you brush your teeth? Either you have a very fancy toothbrush or you might think about a better placement for the sensor.

I also don’t understand why you wouldn’t want to heat the towel rack during a bath

I honestly don’t think a math expert alone will solve the problem…

[powerpolly (Matthias P)]

@BobMiles

I also don’t understand why you wouldn’t want to heat the towel rack during a bath

it is an electric heater with a quite loud fan which I don`t want to run all time. my wifes spends hours in that bathtube

@BobMiles

How does your humidity rise when you brush your teeth? Either you have a very fancy toothbrush or you might think about a better placement for the sensor.

unfortunately the toothbrush is not yet smart home capable, otherwise I could check the runstate and integrate into the humidity detection rule

[m4rk (Mark)]

You could add a temperature probe attached to the hot water supply, shower or bath, or on the underside of the bath, to detect hot water being used. I use Mobile-Alerts and extract the results from a web page or even an API call for the pro probes. There are posts here on how to do that.

[Dibbler42 (Thomas Bail)]

Isn’t that a perfect scenario for machine learning?

[anon71759204]

Interesting problem - you would have to identify the time intervals [t_start … t_end] in which the humdity is strictly monotonically increasing - a bit tricky, but doable by comparing previous and current values.

Let hum_start be the humdity at t_start and hum_end the humidity at t_end.

Judging from your graph:

• Let threshold_hum be two ‘squares’ of humidity.
• Let threshold_shower be 0.2 ‘squares’ of time.

Pseudo code:

if ( hum_end - hum_start ) > threshold_hum -- filter out small peaks
 then
  if t_end - t_start < threshold_shower -- short and steep rise in humidity
   then
    SHOWER
  else - long and steep rise in humidity
    BATHTUB

By fine-tuning threshold_hum and threshold_shower it should be possible to achieve an acceptable accuracy (assuming that all the shower and bathtub peaks look ‘similar’ …).

As an alternative I would attach a temperature sensor to the outside of the bathtub (dismantle the sensor and glue the sensor element to the outside of the bathtub, use heat-conducting glue/paste to minimize the reaction time).

[anon71759204]

@Dibbler42

Isn’t that a perfect scenario for machine learning?

Yes, it is. Using KNIME (or similar tools) and ‘historical data’ one could train a model and let a predictor decide whether the current event is ‘shower’ or ‘bathtub’ or something else. By using time series analysis it would even be possible to predict the next shower/bathtub event.

[rpwong (Russ)]

@powerpolly

the challenging thing is that I need to differentiate between shower and bath as quick as possible

I’m not a math expert, but looking at your graph I see that the shower peaks and then drops sharply, whereas the bath does not. I feel like that time difference is your best indicator of shower or bath, so I would:

1. Trigger your rule when the humidity rises above a certain threshold
2. Set a timer within the rule, equivalent to how long it takes for the humidity to stop dropping
3. After the timer has elapsed, check if the humidity has dropped below a second threshold. If so, turn on the towel heater.

I’m also wondering if you have a separate bathtub and shower, or if it’s a single bath/shower. If the former, you could put in a second humidity sensor with one closer to the bath and one closer to the shower. Since the steam from the shower is more contained, the sensor close to it should peak faster than the one close to the bath. This should also minimize the false positives from brushing your teeth.

BTW, if you’re running the water while brushing your teeth (which is the only way I can imagine the humidity rising), I encourage you to turn off the tap when you start brushing. That should also decrease false positives while conserving water.

[anon71759204]

@powerpolly
We would need to see more historical data to recommend a simple and reliable algorithm … If you are willing to post machine readable data (timestamp/humidity pairs spanning a week/a month), we could have a deeper look into your problem.

Edit:
A more general approach would be to use a realtime peak detector (see, e. g., algorithm - Peak signal detection in realtime timeseries data - Stack Overflow).

[tjarco (tjarco Boerkoel)]

Hi, i think what you would like to do is differential analysis.
You’ll need some samples in time to find the “direction” and the amount of change over a defined period.
Leta say you store 3 samples with an interval of 30sec:

var Humid1; //most recent sample
var Humid2; //sample 30sec older
var Humid3; //sample 60sec older
var D;             //Derative weight

You can get the direction and amount of change:

Change = (humid1 * D) + (humid2 * -2*D)+ (humid3 * D);

If you have some ventilatiesystemen you may use it, for example, as:

FanSpeed += Change;

You probably can see the difference between bath and shower by the weight of Change (shower will be much higher than bathing).

If(Change > x) showering;
else if(change>y) bathing;

You can just fill the formule in in excell to get some feeling with it.

You couls also try it with 2 samples so you could use something like the item oldstate and item newstate in rules:

var Humid1; //item new state
var Humid2; //item old state
var P = 1;
var BathingLevel =20;.    //Shower level
var ShowerLevel=10;.     //Bathing level

Change = (humid1 * P) - (humid2 * -P);
FanSpeed += Change;

If(Change > ShowerLevel) 
    showering;
else if(Change > BathingLevel) 
    bathing;

Or for the 3 samples something like

var Humid1;.  //insert most recent sample
var Humid2;.  //insert sample 30sec older
var Humid3;  //insert sample 60sec older
var D = 1;       //Derative weight
var BathingLevel =20;.    //Shower level
var ShowerLevel=10;.     //Bathing level

Change = (humid1 * D) + (humid2 * -2*D)+ (humid3 * D);

FanSpeed += Change;  //need some limits here 

If(Change >ShowerLevel )
      showering;
else if(change>BathingLevel ) 
      bathing;

That’s how I should solve it mathematical, but i’m not sure how to get the historical data.

[powerpolly (Matthias P)]

great to see so many people trying to solve my use case, many many thanks for that!!!

@anon71759204

We would need to see more historical data

here`s a csv export of the humidity sensor series over the last month:

LED_Bad_hum(last30days).csv

a temp sensor beyond the bathtube is definitely worth to try, other approaches like sensors on the hot water supply won`t work here as water withdrawal for a second bathroom or cooking would negatively influence the bath rules then.

the hum sensor position is fine. it`s an AM2301 at the highest possible position in the bath, right beyond the ceiling. this temp&hum sensor plus a motion sensor are connected to a magichome led controller. the cheap sensor might not deliver the most accurate values, but I only care on seeing humidity changes. shower and bathtube are in opposite corners of the bathroom.

@rpwong

I encourage you to turn off the tap when you start brushing

have kids? but great idea, I`ll add brush teeth detection to the rules and advice alexa to remind my daughter to turn off the tap.
Alexa is already part of my bathroom rules and starts relaxing music when bath is detected… raising WAF.

@Dibbler42

Isn’t that a perfect scenario for machine learning?

I like the machine learning approach, but I guess this is some levels above my skills… I would be the perfect husband if I could predict what my wife plans to do

@anon71759204

A more general approach would be to use a realtime peak detector

detecting the peak is too late for the shower use-case. humidity can still raise when shower finished, but I need the towel heater start while I’m showering. so it`s the climb in humidity that needs to be triggered.

@tjarco

You can get the direction and amount of change:

@BobMiles

First of all I would start calculating the “derivative” of your humidity level

interesting idea, will do some experiments with the derivation approach.

currently I simply compare old and current values. a shower will raise humidity by 14-17%RH per minute. a bath will raise it approx 5%RH per minute.

@tjarco

but i’m not sure how to get the historical data.

this works: LED_Bad_hum.historicState(now.minusSeconds(30)).state as DecimalType

[tjarco (tjarco Boerkoel)]

@powerpolly

I would be the perfect husband if I could predict what my wife plans to do

Don’t go there… plans always change…

[anon71759204]

The humidity signal is quite noisy. Just for reference (x-axis: days):

Higher resolution:

Event ‘shower’:

Event ‘bathtub’:

gnuplot commands used:

set autoscale
#set xrange [2.8:2.9]
set xrange [1.25:1.55]
x0=NaN
y0=NaN
plot 'ts1.csv'  every ::1::4000 using (dx=$4-x0,x0=$4,$4-dx/2):(dy=$5-y0,y0=$5,dy/dx/1000) w l t 'dy/dx/1000',\
        'ts1.csv'  every ::1::4000 using 4:5 with lines

Edit:
High resolution SVG file:

[tjarco (tjarco Boerkoel)]

What kind of sensor do you use and what’s the update frequency?

[powerpolly (Matthias P)]

it’s an AM2301 connected to a magichome controller with tasmota flashed firmware. reporting every 5min unless motion is detected, then I go down to 10sec.

switch LED_Bad_motion.state {
                case ON: {
                    mqttActions.publishMQTT("cmnd/LED_Bad/Teleperiod", "10")
                    teleperiod = createTimer(now.plusMinutes(8), [ |
                        if (LED_Bad_motion.state == ON) teleperiod.reschedule(now.plusMinutes(2))
                        mqttActions.publishMQTT("cmnd/LED_Bad/Teleperiod", "300")
                    ])

[BobMiles]

With a simple threshold on the derivative it should work in my opinion? Maybe a little trial and error but it seems pretty clear,

[tjarco (tjarco Boerkoel)]

Well, you have more options like

1. some (pre) averaging over some samples.
2. have a higher amount of samples for the derative function (5 instead of 3)
3. you could double the time between samples (30sec → 1 min) in combination with setting the D value to 0.5.

Best regards

[anon71759204]

Implementation in Lua:

-- Please note: x values must be equidistant!
-- x scale: days

threshold_shower = 20  -- shower, if 1st derivative is above threshold_shower

time_start = 1634374075761000000

x0 = math.huge / math.huge
y0 = math.huge / math.huge


for l in io.lines( 'ts.csv' )
 do

  _, t, h = string.match( l, '(.*),(.*),(.*)' )

  t = ( t - time_start) / 1E9 / 86400 -- convert timestamp to days

  dx = t - x0
  x0 = t

  dy = h - y0
  y0 = h

  change = dy / dx / 1000 -- scale change for gnuplot ...

  if change > threshold_shower
   then
	print( t, h, t * 86400 )
   end

 end

Output (grouped for better readability, columns: current time in days, humidity, current time in seconds) and discussion:

2.8380930324089 66.6 245211.23800013

OK.

4.4009757407407 80.7 380244.304
4.4010916898163 84.1 380254.32200013
4.4012079050933 86.5 380264.36300006

OK

Not detected, below threshold.

8.4040727314815 72.6 726111.884
8.4044655787052 80.7 726145.82600013
8.4046975694459 85.4 726165.87000013

OK.
8.8431113888889 70.4 764044.824
8.8432276157422 73.6 764054.86600013
8.8433431712978 76.3 764064.85000013
8.8434590046311 78.7 764074.85800013
8.8435047453719 79.7 764078.81000013

Probably a shower event - OK.

10.845959444444 71.3 937090.896
10.846074988427 73.7 937100.87900006

OK.

11.887368935185 68.1 1027068.676
11.887484293982 72.1 1027078.6430001

Inconclusive - stange event.

12.958152453704 69 1119584.372
12.958268275464 72.6 1119594.3790001
12.958384525464 76 1119604.4230001

First event below threshold, second event OK.

14.051926886575 66.8 1214086.4830001

Detected as shower event. OK?

16.15609494213 69.3 1395886.6030001
16.15621042824 71.7 1395896.5809999
16.156326354166 74.1 1395906.5969999
16.1564425 76.6 1395916.632

Bathtub event followed by a shower event. OK.

Below threshold.

17.862331319446 66 1543305.4260001
17.8624475 68.6 1543315.464
17.862898009259 76.7 1543354.388
17.863013761573 79.1 1543364.3889999

20.177188668981 65.2 1743309.1009999
20.177304050927 68.2 1743319.0700001
20.177420277778 71.4 1743329.112
20.17751244213 74.2 1743337.0750001

20.867646504631 62.9 1802964.6580001
20.867761886575 65.3 1802974.6270001
20.867877650462 67.7 1802984.6289999
20.867993888889 70.6 1802994.672
20.868109247686 73.5 1803004.6390001

23.845237418981 61.3 2060228.5129999
23.84570087963 69.4 2060268.556
23.845816180557 72.1 2060278.5180001
23.84593212963 75 2060288.536
23.846047789351 77.4 2060298.5289999

24.88141204861 65.4 2149754.0009999
24.881527835647 68.6 2149764.0049999
24.881644594907 71.3 2149774.0929999
24.881759421298 74.4 2149784.0140001
24.881875277778 76.9 2149794.024
24.88199164352 79.8 2149804.0780001
24.882107187499 82.4 2149814.0609999
24.882223148148 84.8 2149824.08
24.882339525464 87.2 2149834.1350001
24.882454108797 89.6 2149844.0350001

25.931703368056 67.3 2240499.1710001
25.931819282409 70.3 2240509.1860001

28.22510787037 69.2 2438649.32
28.225223726853 73.7 2438659.3300001
28.225339768519 78.5 2438669.356
28.22545539352 82.5 2438679.3460001
28.225571006945 85.1 2438689.3350001

29.065296423612 62.8 2511241.6110001
29.06587537037 66.4 2511291.632
29.065990173612 69.6 2511301.5510001
29.066338425926 76.2 2511331.64
29.066453877316 78.6 2511341.6150001
29.06648832176 79.3 2511344.5910001
29.06683545139 86.1 2511374.5830001

Quite a nice result …

[powerpolly (Matthias P)]

@tjarco

1. you could double the time between samples (30sec → 1 min) in combination with setting the D value to 0.5.

my initial approach was to lower the sample rate to identify hum rise more quickly. unfortunately the lower the sample rate is, the lower is also the difference between samples and therefor false reports were more often. I’ll give 2 & 3 a try.

[garyfree]

How about water sensor in the shower? - The flooding ones