Rules with a wait for changing state back

Setup, Configuration and Use Scripts & Rules
6

Pay attention to hysteresis but for an example of how to do specifically what you are asking for see the antiflapping timer in Generic Presence Detection and Design Pattern: Motion Sensor Timer