Sky cloudiness detection for automated shading

Hi Community, gladly I give something back to the great Openhab Community…

I was searching for a solution to automatically detect the sky cloudiness for my automated shading (blinds). I found the weather forecast not precise enough, therefore I decided to have a solution based on illuminance (Lux) measurement.

Requirements

  • Accurate detection of the cloud level
  • Quick reaction, within minutes
  • Works all year around without adjustment
  • Affordable Investment < 150€, low maintenance

Challenges
For a proper cloudiness detection the lux sensor need to point towards the sun, hence it needs at least 3 sensors to cover a full day. It is also not so easy to find a spot, where it is sunny the whole day, sensors are not an obstacle, and they are easy reachable for maintenance etc. Finally, on my latitude the sun ecliptic and therefore the illuminance changes a lot during the year. This needs to take in to consideration when calculating the cloudiness.

Approach
After some experiments I landed with the new Philips Hue outdoor sensors. Motion sensors measure the Lux level as they usually only need to report motion when it is dark. Via the hue bridge the lux level is transmitted every 5mins to Openhab. As they are battery driven and wireless, they can be placed everywhere on the house or the garden. I placed 2 on the carport and one on a garden wall. The one pointing to south I slightly modified. As the sun is high while in the south I had to saw the cover off (and sprayed it white, to preserve my wife’s nerves).
20190922_103946

The Lux level changes during the day dependant on the elevation of the sun. So it needs to be “normalized”. I do that by dividing with the calculated theoretical max sun radiation on a vertical surface. The sun radiation is calculated from sun elevation, azimuth,… (data provided by the Astro binding). Finally the calculation gives a Cloud Index for each direction.

The 3 sensor need to be masked, as the measurement of sensor shall only contribute when the sun is really in this direction. I found the good old Pythagorean identity sin^2(alpha) + cos^2( alpha ) = 1 helpful, as you can use for alpha the sun azimuth (Astro Binding) which gives a nice weighting and always sums up to 1. Keep in mind, for this approach the sensors need to be orthogonal to each other. This means if the first points to the east (azimuth=90°) the second needs to point to south (azimuth=180°)… But you can point your sensors also to other directions, eg 80°, 170° (but always +90° )…
This gives finally an overall Cloud Index.

Out of the Cloud Index I calculate a FuzzyCloudiness state. If CloudIndex>30 -> “sunny”, if 15…30 -> “partly cloudy”, <15 “heavy cloudy”. This threshold values are dependant on your taste and the calibration of your sensors.

Finally I give the FuzzyCloudiness state over to the blinds control. To avoid an up/down for every cloud passing by there is a hysteresis of 15mins (adjustable) a new state needs to be active.

Solution
Input: Lux Measurement of 3 sensors around the house
Output: Calculated FuzzyCloudiness state: “Sunny”, “Partly Cloudy” or “Cloudy”

The screen to check the current status. Not often necessary as everything is automatic :slight_smile:


Items Definition (you need also additional Astro Items -> see Astro Binding)

Number CloudIndex                   "Cloud Index [%.1f] "                       <sun_clouds>
Number CloudIndexAvg                "Cloud Index Avg [%.1f]"                    <sun_clouds>
DateTime CloudIndexChg              "Cloud Index Update [%1$ty-%1$tm-%1$td    %1$tH:%1$tM]" <time>

Number  IlluminanceEast             "Lichtintensität Ost [%.0f Lux]"            <sun_clouds>                     { channel = "hue:0106:1:LightLevelEast:illuminance" }
Number  BatteryIllEast              "Batterien Ost Status [%s %%]"              <battery>                        { channel = "hue:0106:1:LightLevelEast:battery_level" }
DateTime IllEastLastRead            "Letzte Aktualisierung  [%1$ty-%1$tm-%1$td    %1$tH:%1$tM]"  <time>  (gMonitor)       { channel = "hue:0106:1:LightLevelEast:last_updated" }
Number  TempEast                    "Temperatur Ost [%.1f°C]"                   <temperature>                    { channel = "hue:0302:1:TempEast:temperature" }   
Number  CloudIndexEast              "Cloud Index Ost [%.1f] "                  <sun_clouds>
Number  ClearSkyRadiationEast       "Max Strahlung Ost [%.0f W/m2]"            <solarplant>


Number  IlluminanceSouth            "Lichtintensität Süd [%.0f Lux]"            <sun_clouds>                     { channel = "hue:0106:1:LightLevelSouth:illuminance" }
Number  BatteryIllSouth             "Batterien Süd Status [%s %%]"              <battery>                        { channel = "hue:0106:1:LightLevelSouth:battery_level" }
DateTime IllSouthLastRead           "Letzte Aktualisierung [%1$ty-%1$tm-%1$td    %1$tH:%1$tM]"  <time> (gMonitor){ channel = "hue:0106:1:LightLevelSouth:last_updated" }
Number  TempSouth                   "Temperatur Süd [%.1f°C]"                   <temperature>                    { channel = "hue:0302:1:TempSouth:temperature" }   
Number  CloudIndexSouth              "Cloud Index Süd [%.1f] "                 <sun_clouds>
Number  ClearSkyRadiationSouth       "Max Strahlung Süd [%.0f W/m2]"            <solarplant>

Number  IlluminanceWest             "Lichtintensität West [%.0f Lux]"            <sun_clouds>                     { channel = "hue:0106:1:LightLevelWest:illuminance" }
Number  BatteryIllWest              "Batterien West Status [%s %%]"              <battery>                        { channel = "hue:0106:1:LightLevelWest:battery_level" }
DateTime IllWestLastRead            "Letzte Aktualisierung [%1$ty-%1$tm-%1$td    %1$tH:%1$tM]"  <time>  (gMonitor){ channel = "hue:0106:1:LightLevelWest:last_updated" }
Number  TempWest                    "Temperatur West [%.1f°C]"                   <temperature>                    { channel = "hue:0302:1:TempWest:temperature" }   
Number  CloudIndexWest              "Cloud Index West [%.1f] "                   <sun_clouds>
Number  ClearSkyRadiationWest       "Max Strahlung West [%.0f W/m2]"            <solarplant>

Rules for max sun radiation and CloudIndex calculation (for each direction rule is necessary).
Remark: The max sun radiation could be also used for solar panel efficiency calculation!

import java.time.ZonedDateTime // needed for getDayOfYear

rule "Calculates CloudIndex for East Facade "
  when 
		 Item IlluminanceEast changed 
	then
		 //Astro Input         
         val Number vdayOfYear = (ZonedDateTime::now().getDayOfYear());
         val Number vSunElevDeg = (Elevation.state as Number).floatValue
         val Number vSunElevRad = Math.toRadians(vSunElevDeg)           
         val Number vSunAzimuthDeg = (Azimuth.state as Number).floatValue
         val Number vSunAzimuthRad = Math.toRadians(vSunAzimuthDeg)

         //Collector Input
         val Number vCollectorTiltDeg = 90  //Collector Tilt Angle towards Ground. Must be between 0 (horizontal)..90°(vertical)
         val Number vCollectorTiltRad = Math.toRadians(vCollectorTiltDeg)  //Collector Tilt Angle Radians
         val Number vCollectorAzimuthDeg = 90  // Collector Azimuth (90°=East, 180°= South, 270°=West)
         val Number vCollectorAzimuthRad = Math.toRadians(vCollectorAzimuthDeg)

         //Ground Reflectance guidance values: 
         //Gravel roof=0.1  ¦   Ordinary ground/grass=0.2  ¦  Desert sand=0.4   ¦    Ocean ice=0.6     ¦   Fresh snow=0.8
         val Number vGroundReflect = 0.2

        //Calculations  (Are tested against excel-exact same values. Formulas seem to me from plausibility check not so precise for tilt<>90° )
         val Number vAirMass = Math.abs(1/(Math.sin(vSunElevRad)))   //Air Mass Ratio
         
         //val Number vExtraTerraDay = Math.cos(Math.toRadians(((360*vdayOfYear)/365).doubleValue)) //Intermediate Value (not used)
         //val Number vExtraTerrRadiation = 1370 * (1 + (0.034 * vExtraTerraDay))  //Extraterrestrial Solar Radiation (not used)
         
         val Number vAppaInt1 = (360F / 365F) * (vdayOfYear-275F) 
         val Number vAppaInt2 = Math.sin(Math.toRadians(vAppaInt1.doubleValue))
         val Number vAppaExtraTerrRadiation = 1160 + 75F * vAppaInt2   //Apparent Extraterrestrial Solar Radiation
         //logInfo("Radi","DayofY: " + vdayOfYear + "   AirMass: " + vAirMass  + " ExtraTerra " + vExtraTerrRadiation   + "   Apparen Extra Terra " + vAppaExtraTerrRadiation)

         val Number vOptInt1  = (360F/365F)*(vdayOfYear-100F)
         val Number vOpticalDepth = 0.174 + (0.035 * Math.sin(Math.toRadians((vOptInt1).doubleValue)))
         val Number vSkyDiffuseF = 0.095 + (0.04 * Math.sin(Math.toRadians((vOptInt1).doubleValue)))
         
         var Number vClearSkyBeamRad = 0
         if (vSunElevDeg>0) { vClearSkyBeamRad = vAppaExtraTerrRadiation * Math.pow(2.71828, -(vOpticalDepth*vAirMass).doubleValue) }
         //logInfo("Radi2","  ODepth: " + vOpticalDepth + "   vSkyDiffuseF: " + vSkyDiffuseF + "  CSBeam: " + vClearSkyBeamRad) 

         val Number vInAngInt1 = Math.cos(vSunElevRad)
         var Number vInAngInt2 = Math.cos((vSunAzimuthRad-vCollectorAzimuthRad).doubleValue)
         if (vInAngInt2<0) vInAngInt2 = 0  // If Sun Azimuth and Collector Azimuth are more than 90° apart no contribution to the value 
         val Number vInAngInt3 = Math.sin(vCollectorTiltRad)
         val Number vInAngInt4 = Math.sin(vSunElevRad)
         val Number vInAngInt5 = Math.cos(vCollectorTiltRad)
         val Number vIncidenceAngle = vInAngInt1*vInAngInt2*vInAngInt3 + vInAngInt4*vInAngInt5
    
         var Number vBeamInsoCollector = 0
         if (vIncidenceAngle>0) { vBeamInsoCollector = vClearSkyBeamRad * vIncidenceAngle }

         val Number vDiffuseRadiInt1 = (1F + vInAngInt5) / 2F
         val Number vDiffuseRadi = vSkyDiffuseF * vClearSkyBeamRad * vDiffuseRadiInt1
         //logInfo("Radi3","vIncidenceAngle: "+ vIncidenceAngle + "  vBeamInsoCollector: " + vBeamInsoCollector + "  vDiffuseRadi: "+vDiffuseRadi )

         val Number vReflectRadi = vGroundReflect * vClearSkyBeamRad * ( (vInAngInt4 + vSkyDiffuseF)*((1F-vInAngInt5)/2F)   )

         val Number vCOllectorTotalRadiation = vBeamInsoCollector + vDiffuseRadi + vReflectRadi
         //logInfo ("Radiation", "Sun Elev="+vSunElevDeg + " Sun Azi="+ vSunAzimuthDeg + "  BeamCollector="+vBeamInsoCollector+ "  Collector Radiation="+vCOllectorTotalRadiation)
         //logInfo ("Radiation", "Col Tilt="+vCollectorTiltDeg + " Col Azi="+ vCollectorAzimuthDeg +"  DiffuseRadi="+vDiffuseRadi + "   vReflectRadi" + vReflectRadi)
		 
		 //Calculate Cloud Index
		 if ((vCOllectorTotalRadiation)>100)  //avoid Sunrise/Sunset Calculations
		 { val Number vCloudIndex = (IlluminanceEast.state as Number) / vCOllectorTotalRadiation
		   CloudIndexEast.sendCommand(vCloudIndex) 
		 } //end if TotalRadiation>1
		 ClearSkyRadiationEast.sendCommand(vCOllectorTotalRadiation)
	end

Rules for masking, FuzzyCloudiness and Hysteresis

rule "Calculates FuzzyCloudiness (Cloud Cover) from Cloud Index "
  when 
    	 Item CloudIndexEast changed or
		 Item CloudIndexSouth changed or
		 Item CloudIndexWest changed 
  then
         var float vCloudIndex = 1.0 
         var float vAzimuthDeg = (Azimuth.state as Number).floatValue
         var float vAzimuthRad = Math.toRadians(vAzimuthDeg).floatValue
         var float vMaskingEast = vAzimuthRad + 0 // if your sensor is not pointing to East (Azimuth=90°) you can adjust the value here. Keep in mind that sensors must be South=East+90° and West=East+180°
         var float vMaskingSouth = (vMaskingEast-Math.PI()/2).floatValue // never change as this masking is based on the formula Sin(a)^2+Sin(a-90°)^2 = 1
         var float vMaskingWest =  (vMaskingEast-Math.PI()).floatValue // never change as this masking is based on the formula Sin(a)^2+Sin(a-90°)^2 = 1

         if ((vAzimuthDeg>=0.0)&&(vAzimuthDeg<=90.0)) //Azimuth 0..90
         {    vCloudIndex = (CloudIndexEast.state as Number).floatValue //as there is no LUX sensor on north facade it can not contribute to the calculation
         }    // end Azimuth 0..90

        if ((vAzimuthDeg>90.0)&&(vAzimuthDeg<=180.0)) //Azimuth 90..180
         {    vCloudIndex = ((Math.pow(Math.sin(vMaskingEast),2)*(CloudIndexEast.state as Number))+((Math.pow(Math.sin(vMaskingSouth),2))*(CloudIndexSouth.state as Number))).floatValue                        
         }    //Azimuth 90..180

        if ((vAzimuthDeg>180.0)&&(vAzimuthDeg<=270.0)) //Azimuth 180..270
         {    vCloudIndex = ((Math.pow(Math.sin(vMaskingSouth),2)*(CloudIndexSouth.state as Number))+((Math.pow(Math.sin(vMaskingWest),2))*(CloudIndexWest.state as Number))).floatValue                        
         }    //Azimuth 180..270

         if ((vAzimuthDeg>270)&&(vAzimuthDeg<=360.0)) //Azimuth 270..360
         {    vCloudIndex = (CloudIndexWest.state as Number).floatValue //as there is no LUX sensor on north facade it can not contribute to the calculation
         }    // end Azimuth 270..360

         //logInfo("FuzzyCloudiness", "Azimuth: " + vAzimuthDeg + "  IndexSouth: " + CloudIndexSouth.state + " IndexWest: " + CloudIndexWest.state + " CloudIndex: " + vCloudIndex)
		 CloudIndex.sendCommand(vCloudIndex) 
		 CloudIndexChg.postUpdate(now.toString)   //Set Update Timestamp

		     if (vCloudIndex>=vCISunny)   // If Sunny Detected vCISunny
		     {   if (FuzzyCloudiness.state != 3) FuzzyCloudiness.postUpdate(3)	 
		 	     TimeStampLastSunny.postUpdate(now.toString)  //Set Time Last Sunny
		     } //end if Sunny

            if ((vCloudIndex<vCISunny) && (vCloudIndex>=vCICloudy))      // If Cloudy Detected 
		     {   if (FuzzyCloudiness.state != 2) FuzzyCloudiness.postUpdate(2)	 
		 	     TimeStampLastCloudy.postUpdate(now.toString)  //Set Time Last Cloudy
		     } //end if Cloudy

            if (vCloudIndex<vCICloudy)      // If heavy cloudy Detected 
		     {   if (FuzzyCloudiness.state != 1) FuzzyCloudiness.postUpdate(1)	 
		 	     TimeStampLastHeavyCloudy.postUpdate(now.toString)  //Set Time Last HeavyCloudy
		     } //end if Cloudy
end

Sitemap with some charting

                     Text item=AutoLux
                { Frame label="Zusammenzug"
                  {   Switch item=AutoLux
                      Text item=CloudIndex
                      Text label=">30=Sunny(3)  >15=Cloudy(2)" 
                      Text item=CloudIndexChg
                      Selection item=FuzzyCloudiness mappings=[1="Stark Bewölkt", 2="Bewölkt", 3="Sonnig"]
                      Text item=Time2LessShadingMin
                      Text item=Time2MoreShadingMin
                      Text item=TimeStampLastSunny
                      Text item=TimeStampLastCloudy
                      Text item=TimeStampLastHeavyCloudy

                  Frame label="Bewölkungsgrad Ost"
                  {   Text item=CloudIndexEast
                      Text item=IlluminanceEast
                      Text item=ClearSkyRadiationEast
                      Text item=IllEastLastRead
                      Text item=BatteryIllEast
                      Text item=TempEast
                  }   //End Frame  

                  Frame label="Bewölkungsgrad Süd"
                  {   Text item=CloudIndexSouth
                      Text item=IlluminanceSouth
                      Text item=ClearSkyRadiationSouth
                      Text item=IllSouthLastRead
                      Text item=BatteryIllSouth
                      Text item=TempSouth
                  }   //End Frame  

                  Frame label="Bewölkungsgrad West"
                  {   Text item=CloudIndexWest
                      Text item=IlluminanceWest
                      Text item=ClearSkyRadiationWest
                      Text item=IllWestLastRead
                      Text item=BatteryIllWest
                      Text item=TempWest
                  }   //End Frame  
                  
                  
                  Frame label="Auswertung Cloudiness" // Chart
                  { Switch item=Heat_E_Chart_Period mappings=[0="Stunde", 1="Tag", 2="Woche"]
                    Chart  item=CloudIndex period=h refresh=10000  legend=false visibility=[Heat_E_Chart_Period==0, Heat_E_Chart_Period=="Uninitialized", Heat_E_Chart_Period==NULL]
                    Chart  item=CloudIndex period=D refresh=30000 legend=false visibility=[Heat_E_Chart_Period==1]
                    Chart  item=CloudIndex period=W refresh=30000 legend=false visibility=[Heat_E_Chart_Period==2]

                    Chart  item=FuzzyCloudiness period=h refresh=10000  legend=false visibility=[Heat_E_Chart_Period==0, Heat_E_Chart_Period=="Uninitialized", Heat_E_Chart_Period==NULL]
                    Chart  item=FuzzyCloudiness period=D refresh=30000 legend=false visibility=[Heat_E_Chart_Period==1]
                    Chart  item=FuzzyCloudiness period=W refresh=30000 legend=false visibility=[Heat_E_Chart_Period==2]

                    Chart  item=CloudInputShading period=h refresh=10000  legend=false visibility=[Heat_E_Chart_Period==0, Heat_E_Chart_Period=="Uninitialized", Heat_E_Chart_Period==NULL]
                    Chart  item=CloudInputShading period=D refresh=30000 legend=false visibility=[Heat_E_Chart_Period==1]
                    Chart  item=CloudInputShading period=W refresh=30000 legend=false visibility=[Heat_E_Chart_Period==2]

                  }
                } //End Text Item
13 Likes

Yes I can’t wait for the rest

1 Like

Thomas, we need the solution :grinning:
Very interesting device, didn’t know about it yet. Does it really report the lux values and is the openHAB binding able to read those values?

Here we go, have fun with sun and shade … :sunglasses:

1 Like

Yep. The sensor provides illuminance, motion, and even temperature. Additionally “last updated” and “battery level”. Cheers Thomas

1 Like

I bought my first outdoor sensor for testing, this is a really great device, well build, thx for pointing to it.
Did you disassemble the sensor to paint it? (I have to paint it white, too).
If yes, I hope it will still be weatherproof after reassembling :sunglasses:

To paint i used some masking tape to cover the white plastic of the sensor. I took fine sandpaper to scratch the black plastic parts (paint holds better). Then I sprayed it. Dissassemble is in my opinion not necessary.

1 Like

I am thinking of placing it just upside down as I don’t need the motion feature.
Then I don’t have to saw the cover off.
A bit of silicone should fill the gap between the cover and the sensor …

There is a little hole facing down the main body (to drain water if seals are not 100% close) . If you place the device upside down, you need to seal this. Its probably wise to make a new one on the other side. The same for the cover, if you silicon the gap I would drill a hole on the other side to drain. Thats why I thought I better cut the cover off. This goes quite easy with a metall handsaw which you find in every bigger tool case.

1 Like

BTW, the HUE outdoor sensor works great with zigbee2mqtt and there is now a lux channel available (no need for conversion anymore):

grafik

1 Like

Hadn’t seen this thread before. I have posted a pretty similar solution here quite some time ago. It`s my oldest example and I haven’t really reworked it so probably not the most elegant one in terms of coding, but hey, it works.

I’ve also experimented a lot with weather cloud forecasts and lux sensors in- and outside.
Inside is just not reliable. The crux with an outside sensor is placement and water proofness. I learned the hard way, killing one or two multi sensors along the way.
I now have one 4-in-one (gen 5) and one (latest gen) Aeotec multisensor 6 in operation.
The older one has a lux detection limit of 1000 (which is quickly surpassed when it ain’t cloudy) while the new one can do do up to 32.000 which is fine. The older one is said to be somewhat waterproof at least while the new one clearly is not, so what I did was to get a waterproof enclosure for it. I actually bought some outdoor spotlights and replaced the bulb with my sensor.

By the way, consider using persistence and a sliding window instead of a hard hysteresis, no matter if it’s about the lux or cloudiness values. See my linked thread.
That really helps a lot if you don’t want your shades/shutters to constantly keep toggling.

1 Like

Hi Markus

This Hue outside motion sensors are marketed as weather proof and so far (9 months in use) I did not have any water issues. What is a bit strange, they report still 100% battery level. Either they have super efficient electronics (with solar charging?) or the battery level report is just wrong. We will see. In any case, 9+ months is not bad as they update the lux levels every 5 mins and even almost instantly, if there is a dramatic level change. I just chart the cloud index, but from the values I assume there must be Lux levels over 100k measured. I will check on a sunny day :sunny:

To calculate the cloud index I divide the lux level by “the clear sky radiation” (total radiation from the astro binding). I turned out to be precise until autumn. But when winter came (and sun was low at the sky) I was not happy anymore. I turned out that the Astro binding total radiation is for square meter on horizontal ground. But my sensors are vertical! So I had to calculate the clear sky radiation for a vertical surface. Unfortunately nothing in the Astro Binding, so I calculate them from scratch. I updated now this article with the new formulas. The last 2 months since the update I had very good result!
BR Thomas

2 Likes

WeatherFlow outdoor sensors are perfect to determine when your home is in direct sun.
Love my WeatherFlow

Code updated for Openhab 3.0+ compatibility. (Should work also with OH 2.5+, but not tested)