Howto: Generate similar item/sitemap configurations using templates

Ratimer · August 21, 2019, 6:21pm

I have got lots of door and window sensors and radiator controllers. They are all are configured in a similar way, having several items. So I was looking for a way to avoid typing the configurations again and again. After some investigation I found a nice Java tool named STST http://hardlikesoftware.com/weblog/stst/ . This combines templates and data to generate any kind of text files. It reads data from a JSON format, which is very concise and handy. STST is based on the very powerful template engine StringTemplate https://www.stringtemplate.org/ .

Here is an excerpt of my generated file to configure my window items:

Group gWindows "Alle Fenster"

// Window_Kitchen "Fenster Küche"
Contact Window_Kitchen "Fenster Küche" <window> (gWindows, gContacts) {channel="zwave:device:77ecbab9:node19:sensor_door"}
Number Window_Kitchen_battery "Batterie Fenster Küche [%d]" <battery> (gBatteries) {channel="zwave:device:77ecbab9:node19:battery-level"}
Number:Temperature Window_Kitchen_sensorTemperature "Temperatur Fenster Küche [%d]" <temperature> (gDeviceStatuses) {expire="13h", channel="zwave:device:77ecbab9:node19:sensor_temperature"}
Switch Window_Kitchen_alarmPower "Batteriewarnung Fenster Küche" <lowbattery> (gPowerAlarms) {channel="zwave:device:77ecbab9:node19:alarm_power"}

// Window_Office "Fenster Büro"
Contact Window_Office "Fenster Büro" <window> (gWindows, gContacts) {channel="zwave:device:77ecbab9:node42:sensor_door"}
Number Window_Office_battery "Batterie Fenster Büro [%d]" <battery> (gBatteries) {channel="zwave:device:77ecbab9:node42:battery-level"}
Number:Temperature Window_Office_sensorTemperature "Temperatur Fenster Büro [%d]" <temperature> (gDeviceStatuses) {expire="13h", channel="zwave:device:77ecbab9:node42:sensor_temperature"}
Switch Window_Office_alarmPower "Batteriewarnung Fenster Büro" <lowbattery> (gPowerAlarms) {channel="zwave:device:77ecbab9:node42:alarm_power"}

…

You see the definition of one Group on top of the file and two nearly equal sensors. Item IDs are in English, but labels in German.

This is the template I used to generate it:

Group gWindows "Alle Fenster"

$windows : {
// Window_$it.id$ "Fenster $it.label$"

Contact Window_$it.id$ "Fenster $it.label$" <window> (gWindows, gContacts) {channel="$device$:$it.node$:sensor_door"}
Number Window_$it.id$_battery "Batterie Fenster $it.label$ [%d]" <battery> (gBatteries) {channel="$device$:$it.node$:battery-level"}
Number:Temperature Window_$it.id$_sensorTemperature "Temperatur Fenster $it.label$ [%d]" <temperature> (gDeviceStatuses) {expire="13h", channel="$device$:$it.node$:sensor_temperature"}
Switch Window_$it.id$_alarmPower "Batteriewarnung Fenster $it.label$" <lowbattery> (gPowerAlarms) {channel="$device$:$it.node$:alarm_power"}
}$

The data file looks like this. It only holds variable parts.

{
    "device" : "zwave:device:77ecbab9",
    "windows": [
        {
            "id": "Kitchen",
            "label": "Küche",
            "node": "node19"
        },
        {
            "id": "Office",
            "label": "Büro",
            "node": "node42"
        }
    ]
}

Let’s have a deeper look at the template:

You see variables and logical statements are surrounded by $ signs.

The first one is $windows{ … }$ . This defines an iteration over all elements of the array named “windows” in the JSON file.

Next is $it.id$. $it is the current item of the iteration. Separated by a dot follows the name of the JSON sub-element holding the data. In this case it is “id”.

Congruously the next statement $it.label$ is the “label” element of the iteration object.

$device$ does not reference an iteration object. So it gets replaced by the value of the “device” object outside of the window array, at the beginning of the file.

Now it should be clear what $it.node$ means

As mentioned before the template language is very powerful. A use case I’ve got is to add some Items to additional groups, but some windows must not be added to this group. So I added an additional field named groups to the data structure. Not all window definitions contains it. The data for one window like this looks like this:

        {
            "id": "LivingroomRight",
            "label": "Wohnzimmer rechts",
            "node": "node24",
            "groups": "gWindows_Livingroom"
        },

I modified the first item definition in the template like this:

Contact Window_$it.id$ "Fenster $it.label$" <window> (gWindows, gContacts$if(it.groups)$, $it.groups$$endif$) {channel="$device$:$it.node$:sensor_door"}

The start of the optional part is marked as $if(it.groups)$ which means that the following part is only included when the data contains a value for the “groups” attribute.

The end of the optional part is marked with $endif$. Everything in between is used as a template with all it’s template logic.

More about the template language can be found at https://www.stringtemplate.org/

Of course the template mechanism can not only be used to generate items. Sitemaps can be generated the same way or persistence configurations. Potentially you can combine the same JSON data file with different templates to generate separate types of target files. Attributes that are only required in one template, do hurt if they are not used in another one. They are simply ignored. So you can have a mix of data for both template in just one data file.

Template files have the file extension .st . Data files in JSON format usually have the extension .js.

In my OH conf directoryI created a subfolder named generator in parallel to the items and sitemap folder. Here I store all my templates and data files and a batch file that generates all target files and stores the directly to the correct OH conf directory. STST is a command line tool. stst -h lists all supported command line options.

For the example above the command line looks like this:

stst -o ../items/windows_generated.items windows_items.st windows.js

Using templates helps to meet your own naming conventions without typos and similar items are named in a similar way. This helps keeping overview and ensures generic coding (see Design Pattern: Associated Items)

While your OH system grows, the configurations need to be adapted from time to time. With the templates it is easy to change all scripted items in one go. Just modify the template and start the generator. So it totally makes sense to reimplement similar configurations using templates, even if you already have an working configuration written manually. It’s so easy!

vossivossi · August 21, 2019, 6:38pm

Excellent tutorial! I also already thought about creating some kind of config file generator myself, but the presented approach could be the solution.
Just one question: How do you maintain the data files? I would like to have the data stored in a kind of database, is this also possible?
Thank you very much for your contribution!

Ratimer · August 21, 2019, 6:44pm

Hi vossivossi,

I myself just have JSON files.
StringTemplate is a Java framework. If you’re familiar with writing Java code, it should not be to hard, to write a small program, that reads data from each source you can connect to.
Take a look at https://www.stringtemplate.org

marcel_erkel · August 21, 2019, 8:43pm

You could create the JSON in Excel or Libre Calc using a =CONCAT() “formula” or you could simply copy the table from your spreadsheet and past it into the input box of ‘Step 1’ at csv-to-json. Below ‘Step 5’ press the ‘Convert to JSON’ button.

If you like to try it, copy the following table into the input box of ‘Step 1’ and press the ‘Convert to JSON’ button below ‘Step 5’:

id	label	node
Kitchen	Küche	node19
Office	Büro	node42

Kim_Andersen · August 21, 2019, 9:04pm

Will it work on sitemaps as well? I suspect you´ll have to define each item type in sitemaps as well. Not always to you use the same type in items and in sitemaps… Like this:

.items

Switch    alarm_totalalarm     "IHC Total Alarm tilkoblet [%s]"       <switch>        (alarm)                                                                     { channel="ihc:controller:elko:alarm_total" }

.sitemap

sitemap alarm label="Alarm og bevægelsesensore"
{
	Frame label="Alarm" {

		Text item=alarm_totalalarm
	}
}

Notice the type in items is Switch, but in sitemap I use it as Text.

Ratimer · August 22, 2019, 4:47am

Hi Kim,
you need two different templates. One to generate items and another one for the sitemap. The type Switch and Text are static part of the templates.

Your example might look like this:
File alarm.js

{
    "alarms": [
        {
            "id": "totalalarm",
            "label": "IHC Total Alarm tilkoblet",
            "node": "ihc:controller:elko:alarm_total"
        },
        {
            "id": "firealarm",
            "label": "Firealarm",
            "node": "ihc:fantasydevice:heat_alarm"
        }
    ]
}

File alarm_items.st:

$alarms : {
Switch alarm_$it.id$ "$it.label$ [%s]" <switch> (alarm) { channel="$it.node$" }
}$

File alarm_sitemap.st:

sitemap alarm label="Alarm og bevægelsesensore"
{
    Frame label="Alarm" {
$alarms: {
        Text item=alarm_$it.id$
}$
    }
}

(not tested!)

Kim_Andersen · August 22, 2019, 1:37pm

I think I understand. But I wonder if it saves any work using a template. Or is the template just meant to be use for making things the same (equal) ?

Ratimer · August 23, 2019, 4:49am

Hi Kim,

templates only make sense to generate lots of similar entries. Of course it does not make sense to create a template for each single individual item. The main goal is to reduce typing effort.

psyciknz · August 23, 2019, 5:31am

I bookmarked this the other day to look at. Could this be used for creating thing files from items (thinking of the effort of creating mqtt things/channels from existing items).

Ratimer · August 23, 2019, 5:48am

Hi psyciknz,

if you want to use STST the data have to be in JSON format. Theoretically you could use StringTemplate directly and write your own Java program to parse the items file, but I guess the effort for this is not justified.

If you have lots of similar items it makes more sense rewrite your items to be generated by templates and use the same data with another template to generate mqtt configuration. Creating a template just takes a view minutes when you know how the target file should look like.

I am not familiar with mqtt, so I can’t promise if it’s configuration can be generated. But the chances are not bad .

psyciknz · August 23, 2019, 5:56am

Or use the rest api… i just used your sample, like you I use zwave, and quite often add a new device where theres a bunch of common channels.

Hmm, I sense a playing session coming on.