"Modular" Sitemap?

[meingraham (Michael Ingraham)]

With some openHAB “components”, one can devise a scheme for how to segment those components. For example, you can have all items for each room in a separate file. Or all types of devices (switches, plugs, lights, etc.) in separate files. Similarly for things, rules, etc. Except for Sitemaps.

My “admin” Sitemap is gradually becoming “unruly” to maintain or modify. As I add more controls and functions, the file has grown and grown. I know that one can have multiple Sitemap files, but this requires a user to “manually” select the desired Sitemap. Is there a way to “include” Sitemap “definition” files into a single master Sitemap?

I know that there is a school of thought that a Sitemap is not needed or ought not to be used in everyday operation of a home’s automation environment. I could make the case that some things require a human decision (desired temperature or fan speed) or no amount of sensors and “intelligence” can avoid triggering for the wrong reason (e.g., my dog walks in the room - I haven’t implemented my AI computer vision yet ) Nevertheless, I want a master interface where I can control things if I need to for any reason. I would never ask my wife or guests to use it. Thus, I want to be able to modularize the Sitemap file to make it easier on me to maintain.

Mike

[Thedannymullen (Danny mullen)]

I second this request!

[rlkoshak (Rich Koshak)]

@meingraham

Is there a way to “include” Sitemap “definition” files into a single master Sitemap?

Unfortunately not. Some people have written scripts that take a bunch of parts of sitemap files and merges them into the one big unruly one, but no one has taken on the task of writing something built into OH. I think there has been an Issue open on that for awhile though I could be wrong.

@meingraham

Thus, I want to be able to modularize the Sitemap file to make it easier on me to maintain.

The Group element can partially address this though not completely given the limitations of the using the Group element (i.e. you only get the default and cannot modify the icon, labels, order, or what element is used to display the Group’s members).

[domeninini (Dominik B)]

Is there a feature like this in the meantime since then ?

[rossko57]

Nope. Sitemap structure is not likely to change in the lifetime of openHAB-2.

[mbronk (Mateusz)]

@rossko57

Sitemap structure is not likely to change in the lifetime of openHAB-2.

How about OH3?

[Chrischan (Christian)]

I made a workaround therefore.
My script allows you to create a sitemap-item.

Sitemap name="example"

It will search for a file called “example.sitemap” an inserts all it’s contents instead of the tag.
finally it will create a file named by the name in the sitemap definition of the root-file.
The root element is a file called “main.sitemap” in the folder sitemapparts.

Exmple:
sitemapparts/main.sitemap

sitemap defaultAutoGenerated  label="Home"
{
    Frame label="Wohnraum" icon="firstfloor" {
        Sitemap name="livingroom.sitemap"
    }
}

sitemapparts/livingroom.sitemap

Frame label="Wohnraum" icon="firstfloor" {
    Text item=gTadoBatteryLow visibility=[gTadoBatteryLow==ON]
    Sitemap name="wohnraum/temp" 
    Sitemap name="wohnraum/lstg"
}

sitemapparts/wohnraum/lstg.sitemap

Group    item=gHeizungLstg{
    Text item=HEAT_Wohnzimmer_heating_power     label="Wohnzimmer"
    Text item=HEAT_Esszimmer_heating_power      label="Esszimmer"
    Text item=HEAT_Kueche_heating_power         label="Küche"
    Text item=HEAT_Badezimmer_heating_power     label="Badezimmer"
    Text item=HEAT_Toilette_heating_power       label="Toilette"
    Text item=HEAT_Arbeitszimmer_heating_power  label="Arbeitszimmer"
    Text item=HEAT_Schlafzimmer_heating_power   label="Schlafzimmer"
}

Generated output
sitemaps/defaultAutoGenerated.sitemap.

// Autogenerated sitemap by Sitemapscript located in /etc/openhab2/scripts/watchdir.py
sitemap defaultAutoGenerated  label="Home"
{
    Frame label="Wohnraum" icon="firstfloor" {
        Frame label="Wohnraum" icon="firstfloor" {
            Text item=gTadoBatteryLow visibility=[gTadoBatteryLow==ON]
            //File wohnraum/temp.sitemap doesn't exists. Therefore ignoring it. 
            Group    item=gHeizungLstg{
                Text item=HEAT_Wohnzimmer_heating_power     label="Wohnzimmer"
                Text item=HEAT_Esszimmer_heating_power      label="Esszimmer"
                Text item=HEAT_Kueche_heating_power         label="Küche"
                Text item=HEAT_Badezimmer_heating_power     label="Badezimmer"
                Text item=HEAT_Toilette_heating_power       label="Toilette"
                Text item=HEAT_Arbeitszimmer_heating_power  label="Arbeitszimmer"
                Text item=HEAT_Schlafzimmer_heating_power   label="Schlafzimmer"
            }
        }
    }
}

You can use subfolders. The script will also add whitespaces in front of each row, so that the imported sitemaps will fit to the format.
The export file will be stored as sitemaps/defaultAutoGenerated.sitemap.
There will be now warning before overwriting so take care of your files.

How to:

1. Create a dir /etc/openhab2/sitemapparts

sudo mkdir /etc/openhab2/sitemapparts

2. Install depencies systemd and watchdog

sudo pip3 install systemd
sudo pip3 install watchdog

3. Create the python script below as /etc/openhab2/scripts/watchdir.py

#!/usr/bin/env python
import watchdog.events 
import watchdog.observers 
import time 
import re
import os.path
import logging
from systemd.journal import JournaldLogHandler

#Path to store sitemaps
global outputPath 
outputPath = r"/etc/openhab2/sitemaps/"

#Debounce to avoid multiple runs
global lastBuild
lastBuild=0

# get an instance of the logger object this module will use
logger = logging.getLogger(__name__)

# instantiate the JournaldLogHandler to hook into systemd
journald_handler = JournaldLogHandler()

# set a formatter to include the level name
journald_handler.setFormatter(logging.Formatter(
    '[%(levelname)s] %(message)s'
))

# add the journald handler to the current logger
logger.addHandler(journald_handler)

# optionally set the logging level
logger.setLevel(logging.INFO)

class Handler(watchdog.events.PatternMatchingEventHandler): 
    def __init__(self): 
        # Set the patterns for PatternMatchingEventHandler 
        watchdog.events.PatternMatchingEventHandler.__init__(self, patterns=['*.sitemap'], 
                                                             ignore_directories=True, case_sensitive=False) 

    #if any event in current item hapens
    def on_any_event(self, event):
        global lastBuild
        #and the last item was created minimum a second ago
        if int(time.time())-lastBuild>1:
            logger.debug(event)
            print (event)
            #build the sitemap
            buildSitemapFile()

#Builds the Sitemaps file
def buildSitemapFile():
    if not os.path.isfile('main.sitemap'):
      print('Root file "main.sitemap" does not exist.')
    else:
      content = getFileContent('main')
      #Don't continue if file is empty
      if content is None:
        return
      
      #Set current time to avoud debounce
      global lastBuild
      lastBuild = int(time.time())

      #Search for the name of the sitemap to set the filename
      filename = 'noname.sitemap'
      regex = r"sitemap\s*(\w*)"
      matches = re.finditer(regex, content)
      for matchNum, match in enumerate(matches, start=1):
          filename = match.group(1)+'.sitemap'

      content = "// Autogenerated sitemap by Sitemapscript located in " + os.path.realpath(__file__) + "\n\r" + content

      #write sitemap to path
      f = open(outputPath+filename, "w")
      f.write(content)
      f.close()
      logger.info('wrote sitemap to '+outputPath+filename)
      print ('wrote sitemap to '+outputPath+filename)

# This function reads the content of a sitemapfile.
# filename: the name of the sitemap without the .sitemap
# prespaces: a string to be set before every row to get a nice formatted file
def getFileContent(filename, prespaces=''):
    filename += '.sitemap'
    logger.debug('Processing file: '+filename)
    print('Processing file: '+filename)
    #If the file doesnt exists leave a comment in the target sitemap
    if not os.path.isfile(filename):
      return '//File '+ filename + ' doesn\'t exists. Therefore ignoring it.'
    else:
        data = ""
        #Read the target file line by line and extend it by the pre-spaces
        with open(filename, 'r') as fp:
            line = fp.readline()
            while line:
                data += prespaces + line
                line = fp.readline()
        
        #If the file was empty retun None
        if data == "":
          return None

        #otherwise search for sitemap items
        regex = r"^([\t ]*)Sitemap\s*name=\"([\\\/\w]*)\""
        matches = re.finditer(regex, data, re.MULTILINE)
        for matchNum, match in enumerate(matches, start=1):
            #if a sitemap item was found call getFileContent wit the sitemapname and 4 extra spaces
            content = getFileContent(match.group(2),prespaces+'    ')
            #replace the Sitemap item with the filecontent
            data = data.replace(match.group(),content)
        return data

if __name__ == "__main__":
    src_path = r"."
    event_handler = Handler() 
    observer = watchdog.observers.Observer() 
    observer.schedule(event_handler, path=src_path, recursive=True) 
    observer.start() 
    try: 
        while True: 
            time.sleep(1) 
    except KeyboardInterrupt: 
        observer.stop() 
    observer.join() 

4. Create a service for the script

sudo nano /lib/systemd/system/sitemapWatchdog.service

The content of the file is

[Unit]
Description=Watchdog and sitemap-generator for /etc/openhab2/sitemapparts
After=multi-user.target

[Service]
Type=simple
ExecStart=/usr/bin/python3 /etc/openhab2/scripts/watchdir.py
WorkingDirectory=/etc/openhab2/sitemapparts
Restart=on-abort
StandardOutput=inherit
StandardError=inherit
User=openhab

[Install]
WantedBy=multi-user.target

5. Activate the service

sudo chmod 644 /lib/systemd/system/sitemapWatchdog.service
chmod +x /etc/openhab2/scripts/watchdir.py
sudo systemctl daemon-reload
sudo systemctl enable sitemapWatchdog.service
sudo systemctl start sitemapWatchdog.service

6. Create your sitemapfiles.

[mbronk (Mateusz)]

Thanks @Chrischan! If one wants to resort to a separate watcher process, what you provided is a very neat idea!
BTW. You might consider posting it in the #tutorials-examples section so it is easier to find for other folks.

---

My question stands though. I am thinking OpenHab should eventually (3.x, 4.x, ???) bring first-class support for ‘Sitemap fragments’ (very much in line with what @meingraham proposed in the original post).
Till we get there, the above idea is a very decent workaround… I don’t believe it should be considered anything more than that, though. IMHO this kind of support belongs in the framework itself (and the fact people need it so much they will even resort to separate scripting… only seems to validate this need ).