Load things from yaml file

Is it possible to load things and possibly items as well as yaml? I see that the new UI uses yaml all over the place and the thing configs look like this:

UID: modbus:tcp:e3dc
label: E3DC Modbus TCP
thingTypeUID: modbus:tcp
configuration:
  rtuEncoded: false
  timeBetweenTransactionsMillis: 60
  connectMaxTries: 1
  reconnectAfterMillis: 0
  port: 502
  timeBetweenReconnectMillis: 0
  host: 192.168.178.21
  connectTimeoutMillis: 10000
  id: 1
  enableDiscovery: false

I really like this format, but I don’t want my configuration to be managed by the UI. Currently using the standard .things and .items files, problem is just that lines can get quite large in that format. YAML is way more elegant.

TLDR: can I put .yaml files in the things or items folder?

AFAIK, you can’t.

I wrote a utility to generate the .items and .things files based on yaml + template.

Link to forum post: Another Things and Items file Generator

The yaml file looks like this:

LivingRoom_Window1:
  template: aqara-contact
  thingid: livingroom-window1-contact
  icon: window
  groups:
    - gWindows
    - gLivingRoom
  tags:
    - Window

Hallway_Light1:
  template: tuya-rgbtwlight
  thingid: hallway_light_1
  power:
    groups:
      - gSmartLights
    metadata:
      - HardwareSwitch: Hallway_Light_Switch
  groups:
    - gHallway
  color:
  ct:

The structure of the yaml relates to the template in use so you pretty much decide what and how it should be structured.

No. But you can maintain the source in YAML then copy’n paste it into the UI “Code” tab.
Note YAML support was built to store UI-configured components. It is by design not available as an input format. For file provided config use the existing format.
Then again file configs are very prone to errors which is why interactive config through UI is way preferrable anyway.
But that’s even more so true when you don’t provide inputs in the file format the designers intended for it (.things/.items).
What’s the point in that, boredom? Then there’s many other areas of OH your time is better spent on.

@JimT thank you, I will consider that. Might be to much overhead for me.

@mstormi the point in that is a less error-prone config format with better readability and good editor support.

No. But you can maintain the source in YAML then copy’n paste it into the UI “Code” tab.

hmm, that’s not worth it form me.

Then again file configs are very prone to errors which is why interactive config through UI is way preferrable anyway.

I think this has been discussed on the forum enough already with many arguments for both ways. Personally I prefer the files config for version control, seperation and bulk editing.

But that’s even more so true when you don’t provide inputs in the file format the designers intended for it (.things/.items).

That is what my question was about, if YAML is supported as it used by the the new UI. Anyway, thanks for your clarification.

1 Like

I wouldn’t think many people agree that YAML has better readability than .items format.
The point though is that as you cannot use YAML file input when you don’t want to copy’n paste.
So you would need to build some software that converts YAML into .items format, with potential bugs/errors in handling both of these.
That’ll effectively double the error probability and will not even provide any benefit in return.

Don’t mix this, this is about a different aspect:
YAML is internally used to store those UI-defined items+things.
As I said you can NOT use YAML to define items and things.
YAML is also used for UI (widget) definition. That is where you would copy’n paste, and that’s well worth it as it is the only way to have a textual representation to store.
Then again, and that’s to consider, for widget definition there is no real point in version control, separating and bulk editing like there might be for items.
(well there’s actually a lot less need for/benefit for that in items and things as well than people who “just like text” tend to think, but for UI definition there’s really no point).
You would use widget templates there rather than replicate and do bulk changes.

I am not mixing it, you said UI configuration is preferable anyway. I have started with paper UI, migrated everything to openhab 3 UI and found it extremely hard to manage. I then switched to everything in .things and .items and I am happy with it. I was just wondering if YAML is an option and I understood that it is not without workarounds.

Just a bit of background info.

The YAML you see in MainUI only exists in MainUI. Nowhere else in OH is YAML used for these things.

The native representation of everything is JSON. And JSON is really hard to work with for humans, especially for things like rules. For example, your scripts will all have to fit in one JSON entry. Since newlines are not allowed your script must be all on one line using \n instead of actual newlines. Yuck!

But, if you are OK with that, you can indeed save a .json file under $OH_CONF/automation and OH will pick it up and load it like any other rule. I don’t know if there is the same for Things and I’m pretty sure there isn’t similar support for Items. Items are hard because one line in a .items file might touch three different internal stores in OH’s memory (the ItemRegistry, MetadataRegistry, and Links) and the current REST API doesn’t even have the end points (maybe now? it’s been a few version since I checked last) to easily get everything associated with a given Item.

The same old reply for future readers of this post (i.e. I’m not trying to convince you to change anything). All three of these are also possible through the UI too. You may prefer text files and that’s fine. But I can’t let stand statements that imply that these are not possible through the UI too.

Not really. It’s generated on the fly by MainUI from the original JSON. Nowhere is YAML formatted stuff stored anywhere. It’s converted from JSON to YAML in the browser because, despite all its faults, it’s way easier for a human to read than the original JSON.

For example, the following YAML rule:

configuration: {}
triggers: []
conditions: []
actions:
  - inputs: {}
    id: "1"
    configuration:
      type: application/javascript;version=ECMAScript-2021
      script: >-
        var logger = log('Alarm');


        var hour = time.ZonedDateTime.now().hour();

        var currToD = items.getItem('TimeOfDay').state;


        logger.debug('Current ToD = {} Current Hour = {}', currToD, hour);

        if(currToD == 'NIGHT' || hour < 5) {
          logger.info("The alarm went off and it's night time, turning on the lights for ten minutes");
          items.getItem('TOD_Lights_ON_MORNING').sendCommand('ON');
          actions.ScriptExecution.createTimer(time.ZonedDateTime.now().plusMinutes(5), function() {
            items.getItem('TOD_Lights_ON_MORNING').sendCommand('OFF');
          });
        }

        else if(currToD == 'BED'){
          logger.info('Good morning!');
          items.getItem('TimeOfDay').sendCommand('MORNING');
        }

        else {
          logger.warn("The alarm went off but it's not BED or NIGHT")
        }
    type: script.ScriptAction

Actually looks like this in the raw JSON:

  "alarm_script": {
    "class": "org.openhab.core.automation.dto.RuleDTO",
    "value": {
      "triggers": [],
      "conditions": [],
      "actions": [
        {
          "inputs": {},
          "id": "1",
          "configuration": {
            "script": "var logger \u003d log(\u0027Alarm\u0027);\n\nvar hour \u003d time.ZonedDateTime.now().hour();\nvar currToD \u003d items.getItem(\u0027TimeOfDay\u0027).state;\n\nlogger.debug(\u0027Current ToD \u003d {} Current Hour \u003d {}\u0027, currToD, hour);\nif(currToD \u003d\u003d \u0027NIGHT\u0027 || hour \u003c 5) {\n  logger.info(\"The alarm went off and it\u0027s night time, turning on the lights for ten minutes\");\n  items.getItem(\u0027TOD_Lights_ON_MORNING\u0027).sendCommand(\u0027ON\u0027);\n  actions.ScriptExecution.createTimer(time.ZonedDateTime.now().plusMinutes(5), function() {\n    items.getItem(\u0027TOD_Lights_ON_MORNING\u0027).sendCommand(\u0027OFF\u0027);\n  });\n}\nelse if(currToD \u003d\u003d \u0027BED\u0027){\n  logger.info(\u0027Good morning!\u0027);\n  items.getItem(\u0027TimeOfDay\u0027).sendCommand(\u0027MORNING\u0027);\n}\nelse {\n  logger.warn(\"The alarm went off but it\u0027s not BED or NIGHT\")\n}",
            "type": "application/javascript;version\u003dECMAScript-2021"
          },
          "type": "script.ScriptAction"
        }
      ],
      "configuration": {},
      "configDescriptions": [],
      "uid": "alarm_script",
      "name": "Alarm Clock Execution",
      "tags": [
        "test"
      ],
      "visibility": "VISIBLE",
      "description": "Called when the alarm clock goes off"
    }
  }

Even better is to install from the Marketplace. I encourage anyone who posts an example widget to consider posting it to the marketplace where it can simply be installed by users instead of copy and paste.

As with everything else, for widgets too the YAML only really exists in the browser. It’s saved and processed as JSON. I think you could even post the JSON in a marketplace entry in addition to YAML. Not sure you’d want to but it’s supported. I know that it is definitely supported for rule templates.

But it is true that the YAML is the only way to copy and paste MainUI Widgets because the editors you’d be pasting into only support YAML.

This point cannot be stressed enough. If you are copying and pasting YAML for widgets in your own config, a “custom widget” created under Developer Tools → Widgets is the better approach. If you are coping, pasting and editing lots of rules the better approach would be to post the rule to the marketplace as a template (there isn’t yet support for local rule templates but I’ve plans to try and push support for that). It’s way better to install, instantiate, and configure than it is to copy/paste/edit. Failing that, a local library is preferably to copy and paste.

The more advanced users we can get posting widgets and especially rule templates, the closer we can get to a point where the average user rarely needs to code rules at all. They mostly just need to install and configure them.

The thing about rule templates sounds really promising!
I have one question: how would you do seperation and bulk edit in UI?

When I used to use UI config I had a very long list (100+) of items that I could either sort by binding or name. For text files I have one .items file per room and some others like functional_groups.items. Is there any way to achieve something similar to this in UI?

For bulk edit:
How would you rename an item in UI or delete it with all its references?
In my current setup (remote vscode with openHAB extension) I just go to search and type in the item name, then replace all occurrences or delete the lines / files.

EDIT: I forgot to mention the ability to comment everything so you don’t forget what which config is doing.

It depends on why you want to do those things. What your end goal is will dictate the best approach.

For some ways to do separation:

  • Items, Rules, and Widgets can be assigned tags which can be searched on.

  • The Developer Sidebar (alt-shift-D or bring it up from the Developer Tools page) can search for and pin Things, Items, and Rules. So, for example, if you are working on your HVAC system you can pin all the relevant stuff to the sidebar and it’s all right there one click away.

  • Naming conventions can be used to group stuff together. When using “create equipment from Thing” to create a bunch of Items all at once, the name of the Thing will be used for the default names of the Items.

Personally I use tags and when I want to see everything that is related, I search for the tag and the config page will only show the Items or Rules with the given tag. For Things there is already an option to group the Things by binding. If you need some other grouping, I’ve used tags on the Items linked to the Things. From the Item you can navigate from the Item to the Link and on to the Thing. Though mostly I use the Developer Sidebar and just pin the ones I need at that time.

I like this way of achieving separation because I don’t have to decide ahead of time what I want the separation to be. I can create an adhoc grouping of anything relevant to what I’m working on.

Firstly, I do not consider this to have anything to do with bulk editing. So perhaps there is a difference in definitions. Bulk edits to me is, for example, adding a tag to a bunch or related Items or creating a bunch of similar MQTT Things or something like that.

Anyway, for this rename case, this is a pet peeve of mine. There is no such thing as renaming an Item in the UI or text files. Every time you modify a .items file, all of the Items defined in that file are deleted from OH and then reloaded from scratch.

So to “rename” an Item in the UI you need to do the same thing. Delete the Item and recreate it. To delete you can delete the link and the Items linked to a Thing’s Channel from the Things page or the Link page. When the Item is deleted this way my understanding is that the metadata registry will clean itself up as well (I don’t think managed Item metadata gets cleaned up at all for text based Items). You can choose to remove the link or not but you should.

From a bulk edit perspective, you can do this for all the Items linked to a Thing all at once using the options at the bottom of the Things Channels page.

Then you can use the “Add Equipment to Model” or “Add Points to Model” and you can recreate all those Items in one go with the new settings/names/etc. What’s nice about this is not only can you crate and configure all the Items for that Thing in one go, there are reasonable defaults for a lot of things and it will create the Equipment Group for you and let you select the Location.

But whether you are doing this in a text file or through the UI, renaming an Item should be done with great care. Not only will you potentially need to update rules and sitemaps and such but you will disconnect that Item from its persistence and it’s metadata. Renaming an Item should not be taken on lightly. I personally think it’s too easy to do so in .items files but not enough to harp on it. But it is a source of “leaks” in openHAB because by design not everything gets cleaned up on the unload/reload.

After renaming the Item, you can close OH, open the JSONDB files in VSCode and do the find and replace just like you are used to, or you could do it though the API Explorer in some cases though I find that less easy to use.

Because of the disruption caused by “renaming” Items is so great, doing so should be done rarely and only under significant need.

Things are a little less easy to do but still not too bad to bulk edit and create a bunch of similar ones. The fastest way would be to generate one example Thing, query for it in the API Explorer, copy the JSON, edit it, and post it back to OH through the API Explorer. You can create dozens of similar MQTT Things in a few minutes that way. Less quick would be to use the Code tab YAML and copy/paste/edit.

There are not too many bindings that do not support automatic discovery of Things so the above only applies to those bindings that do not auto-discover (e.g. MQTT, KNX, modbux, etc.). Auto discover everything else.

Over all though, in my experience the number of times I’ve ever had to do a bulk edit in these ways are pretty limited. Usually it’s only when adding new devices to the home automation (or decomissioning them). The last time I renamed an Item was when I moved to OH 3 (the names of the Items don’t really matter that much anymore compared to the labels of the Item). I use dynamic UI widgets (posted to the marketplace) for a lot of my devices that rely on Group membership and/or Item tags so there it doesn’t matter what my Items are named, the widgets adjust dynamically.

One last piece of bulk edit. You likely have a bunch of Items that you want to look the same way in MainUI. So create the widget once under Developer Tools. Then you can apply that widget to each Item using the Item’s metadata. Now, if you decide to make a change, you can modify that widget under Developer Tools and the change will be picked up by all the Items that are using it. I’ve posted a bunch of these types of widgets to the marketplace too.

Now for comments.

Things

I find using meaningful names and the location fields are more than sufficient when combined with the binding and the channels to determine what a Thing is for and what it represents.

Items

Here too, the Item name, label, type, link, Group membership, semantic tags and position in the semantic model, and additional tags make it clear what the Item represents and what it’s for. Were I to need more, I’d probably use Item metadata to capture some notes.

For example:

It’s pretty clear that Nate’ Christmas Tree controls the lights of a tree in the Family Room. Notice the Item name doesn’t even appear in this tree (you can click on it and get the Item name easily enough though if you need it).

Rules

For UI created rules you have lots of places to add comments and additional information. There’s a name and description where you should explain what the rule does over all. Rules can have tags too.

Each rule can have one or more triggers, actions, and conditions. Each of these also have a name and a description. They are often filled in with a default that makes sense but you can modify them as needed.

If you have a Script Action or Script Condition, you can use inline comments per normal too in the script.

// No alarm scheduled
var type = (typeof(require) === "function") ? DateTimeType : DateTimeType.class;
if(time.class != type || time.getZonedDateTime().isBefore(ZDT.now())) {
  logger.debug("No alarm scheduled for " + item);
  if(this.timer !== null) {
    logger.info('Cancelling alarm');
    this.timer.cancel();
  }
}
// create or schedule a timer to run at the configured time
else {
  logger.info("Scheduling alarm at " + time + " for " + item);
  
  if(this.timer !== null) {
    logger.info("Rescheduling alarm for " + time);
    this.timer.reschedule(time.getZonedDateTime());
  }
  else {
    logger.info("Setting a new alarm for " + time);
    var map = new java.util.HashMap();
    map.put("triggeringItem", item)
    this.timer = ScriptExecution.createTimer(time.getZonedDateTime(), callScriptGenerator(map, script));
  }
}

The one place where comments are not really supported are widgets. I know there is an issue but it’s a hard problem since JSON, which is the underlying format, doesn’t support comments. It’s a known problem without a solution yet.

I use a fairly simple workaround for widget comments. Any keys in a component other than the basic config and slots get thrown out as far as I can tell, but additional keys within the config do not (presumably because in most cases the config object is bound to the underlying f7 component and the UI doesn’t bother to do any validation of those values). This means that you can just add a comment key to the config of any component with the comment text and it will be preserved along with the widget but also not impact the widget function.

Here you can see one of my comments in the JSON form saved in the ui jsondb file to show that this information does persist:

{
  "component": "f7-block",
  "config": {
    "comment": "** HVAC mode menu ** Flex direction is reversed so that css sibling selector works from buttons to block showing the icon\n",
    "style": {
      "align-items": "center",
      "aspect-ratio": 1.0,
...

It’s not an official comment syntax, but it works more than well enough. I suspect that it might not work with some of the more specialized components (those that don’t bind the config to an f7 component), but I haven’t tested it with everything because it’s usually just required in the container blocks to help track whatever wacky thing is going on inside them.

1 Like

The whole reason for creating the generator tool is to make it easier, quicker, and less tedious, and reduce the possibility of errors. You only need to pay attention to the .items and .things format once, when creating the template.

Because I have lots of similar devices, using a template makes sense. For example all Tasmota based lights have similar .things and .items, so one template to create them and you can have as many devices defined in your yaml that use the same template. I also have a lot of Zigbee2mqtt sensors, so just one template for that to create 20+ sensors.

This results in a consistent, and less error-prone output. And when I wanted to tweak my .things or .items, for example, adding an extra metadata common to all my items, I just edit ONE thing: the template, and regenerate, and voila, my 100+ items / things are changed. It beats doing a mass search/replace and I cannot even begin to imagine how this can be achieved using UI.

To make it even easier for me, I created a rule that watches for a change in the yaml file. So whenever I need to add or remove a device, I just edit my yaml file, save it and my .items and .things are automatically updated.

Maintaining the YAML file is far simpler than maintaining the .things and .items file or the MainUI’s YAML, because it only holds the high level data in it. The repetitive stuff is stored once in the template files.

I’d suggest you give it a try. It has made dealing with .items and .things easier for me.

It isn’t a 100% commitment. I still have quite a few unique .items and .things files that I manage manually without the generator. I only use the generator to maintain items / things that are highly repetitive.

Furthermore, if you want to stop using it, just take the generated .items and .things file and throw away the generator, and continue maintaining the generated files the old way.

Example, if I wanted to add another motion sensor, all I needed to do is add this into my yaml:

ToolCupBoard_PIR:
  template: aqara-pir
  groups:
    - gHallway
  motion:
    groups:
      - gMotionSensors

Then the corresponding .things and .items will be generated for me automatically. I don’t have to think about the nitty gritty details of things channels and linking them to the multiple items that I need. If I decided I want to rename it, just rename ToolCupBoard_PIR to GarageCupboard_PIR. If I want to remove this device, just remove that block and I’m done. How can it be easier than this?

When I decided that ALL my motion sensors need to have an additional item, say XXX_SignalStrength that belongs to a gSignalStrength group, I would add that to the template, and regenerate, then voila, all the motion sensors now have an extra Signal Strength item associated with them.

Lastly, if you don’t like my generator, you can write your own. The main thing is the idea to invest the time upfront to make it easier in the future.

I did have a look on the topic of YAML and found a combination of libraries which can generate a proper YAML output while calling some macros and loops in areas where UI is still falling short.

These are:

  1. handlebars.java - generates dynamic yaml
  1. jackson-dataformat-yaml - parses it into openHAB structures

I use first to have a basic templates and macros and later to push results into openHAB at runtime. So far I did it only for pages and widgets where I needed some flexibility. Obviously with a tiny amount of work jackson dataformat yaml could be embedded into openHAB to let people copy-paste definitions between UI and files. Most of jackson components are already deployed into OH (jackson annotations, core and databind), so its practically two jars - jackson-yaml + snakeyaml.

that sounds pretty handy! Does it support multiple yaml input files and multiple .items outputs?

Yes, and yes. I prefer dumping it all into one output but you can have it as granular as you want.

I haven’t read the whole thread, but did you have a look into the following:

1 Like

This looks promising! Thanks for pointing it out

Just a dirty side note:

YAML is a superset of JSON
So you probably could write YAML and push it to the API manually

(I am currently using ansible to create the configuration files but am annoyed with the limitations of the things format, eg nested code for the widget definitions of the items )

i think it works only in the other direction. YAML is a superset of json, so a yaml parser must accept json, but a json parser does not accept YAML.

Hi all,

joining a bit late to this party as I’m an OpenHab newbie, but let me share my 2 cents.

I come with a Drupal background (I used to be a Drupal core sub-system maintainer). In Drupal, every piece of configuration done in the UI can be exported to a YAML file. And every piece of YAML configuration can be imported. Which effectively means two wonderful things:

  1. All configurations done in the UI can be very easily version controlled
  2. It’s easy to do the dev-staging-prod workflow of configurations

I’m not planning to have a dev-staging-prod workflow for my home automation since I only have one environment (I can only afford one house :slight_smile:), but I definitely want to have all my configs under version control. I do understand that Openhab has backup capabilities but to me, version controlling and backups are two different concepts.

After reading this thread, I’ll tackle the version controlling so that I’ll create subdirectories like /etc/openhab/things/yaml and manually copy-paste the Things configured in the UI over there as YAML so that I can have my Things under version control. And the same things for Items, Rules, and so on.

However, in an Ideal World™ , we would have the following capabilities in OpenHab so that one would not need to copy-paste stuff manually.

Config exports capability:

  • A command line tool / web UI tool called “config export” which would take arguments like “all”, “all things”, or even an individual thing’s ID.
  • This would generate (and overwrite if present) the YAML files in a standard locations, for example /etc/openhab/things/yaml

Config import capability:

  • A command line tool / web UI tool called “config import”.
  • This would read the YAML files from the standard location and create (or update if present) the configuration items.

This would allow advanced users to have a systematic way of working like:

  • make configuration changes that belong together
  • export configurations to YAML
  • commit to version control

If somebody’s interested, here’s additional information on the Drupal configuration management:
https://www.drupal.org/docs/configuration-management

Cheers,
Markus

1 Like

This seems excessive. Why not just version control $OH_USERDATA/jsondb? Those files are text JSON and very amenable to source control.

Anything is possible as long as someone volunteers to do it.

But keep in mind that OH “speaks” JSON. YAML pretty much only exists in the MainUI. I’m not sure what converting everything to YAML for import/export does for you except forcing you to be tied to MainUI for import and export operations. If you stick with JSON you can use the already existing REST API instead of needing to add code to core, the REST API, and MainUI.

While the comparison isn’t completely apples to oranges, Drupal has a system that was designed from the start to work a certain way (or was reworked at some point to conform to a certain design). OH’s configs have grown much more organically and there are at least four different formats/locations for OH configs: text based (e.g. .things files), the JSONDB, Karaf’s $OH_USERDATA/config, and Karaf’s $OH_USERDATA/etc.