New tool: a purely JS parser for item definitions

Ah, sorry I realize I wasn’t clear enough, what I meant was, openHAB should do the cleanup in this case, automatically (and generally make sure all references between objects are valid), not the user :slight_smile:

1 Like

Here’s a pure-UI scenario you can try if your things are already there and you need to create items for them:

  1. create a hierarchy of locations (groups) using the semantic model builder:

  1. select a location and choose the “Create Equipment from Thing” option with the lower-right-corner button:

  1. this will add a Group item representing a Thing (Equipment), and the channels you choose as Point items. You have to configure the name, labels and so on for each item that you’re going to create. This feature basically replaces the “simple mode” but with more control.
    And yes, there needs to be a way to configure the prefix of items’ names, and a “check all” option for channels…

  1. When you’re finished, click Add and the hierarchy of items will be added to the model under the group (location) you selected as the parent.

  1. If you want to undo this, including links and items, the best way is to go to the Thing page, in the Channels tabs, you will find a “Unlink all and Remove Items” button:

Hopefully it will work without problems, if a single deletion request fails (link or item) it will be reported but it won’t show which one. Note that it also won’t remove the equipment group (there is no real link between the Thing and the group). A better way would be to be able to remove a group including all members and links recursively from the model…

As long as it lets you define the names of the Items and doesn’t hide the fact that there are Items I’m happy. As it exists right now, simple mode provides too much “magic” and not enough clarity causing confusion and leading to limitations. This approach doesn’t appear to have the same limitation. I look forward to experimenting with it.

Would it make sense to just do something simple and if you have a “Clean Groups” button somewhere that finds and removes (with confirmation of course) Groups that have no members. A Group without members isn’t all that useful most of the time. Those users who are dynamically changing Group membership through Rules should be advanced enough to know not to remove their empty Groups.

2 Likes

Exactly, it lets you choose where in the hierarchy you wish to insert the items, which channels you need, and change the name of the items before they are created (the names of the items created by the simple mode are often ugly, and you can’t change them afterwards…).

The other similar option, “Add Points from Thing”, is basically the same but without creating the group - instead you select an existing group in the tree view first, and the items will be added to that group. For instance, if you originally decided against linking some channels and now want them, or if you add some channels to an extensible Thing like a MQTT thing, you can then use this option on the original equipment item.

Removing a group and all members recursively is not that hard, you can get the list of items with a simple API call and retrieve all the links and remove those which concern the items you’re about to remove. But a “remove empty groups” feature could very well end up for instance in the developer tools, which is destined to be a pot-pourri of various functions for power users that don’t belong anywhere in the main UI…

2 Likes

And where can I place these generated jsons for openHAB to parse it? What about tags on items? If I’m right currently there is no option in PaperUI to set it, can it be modified later from the json file?

You use the REST API Docs to submit the JSON to OH. Tags are included in the JSON this tool generates.

You can modify the tags using the REST API Docs.

In OH 3 when the above tool becomes part of OH’s UI, I’m certain that there will be ways to see and modify tags on Items directly in the UI.

Thanks I hoped that there is some way to manually insert it into the DB. Problem is that I can’t install the REST Docs right now…

Thanks for the clarification anyway!

REST API works without the docs or the UIs would not function. You could install a separate OH instance (without the troublesome bindings) to access the API docs there. The syntax would be the same as the broken system.

There is but it’s risky. You need to stop openHAB and then you can manually edit /var/lib/openhab2/jsondb/org.eclipse.smarthome.core.items.Item.json. But it’s really easy to make a mistake so if you try to go down that path I suggest that you make a backup first and be prepared to restore it. One missing comma will corrupt the whole file.

You are better off waiting for the REST API docs to be fixed or run a separate version of OH where the docs are not broken and copy/paste the curl command generated by the docs to submit the JSON to your running OH like Bruce suggests.

Yes thanks for the correction! I know it… However usually people use the UI and JSONDB to store items? When I started using openHAB, I have started with PaperUI to manage items, but I had a lot of problems with it and someone suggested to move to items file and yes, all problems were gone (and not just that I was unable to add tags to it, but some items disappered from the Items list, etc). Are these problems are gone even on a larger scale setup?

Thanks! Is there any advantage of this anyway? For me personally I would be happy to drop all my items file (it is easier to edit through UI on a tablet or smaller device - and also after adding Things usually from PaperUI it would be easier to add Item to these Things). But is there anything else? Or it might be just a preparation for the upcoming OH3?

Items file are much easier to back up and, if you have an older copy, to revert bad edits.

I disagree. It’s just as easy to backup /var/lib/openhab2/jsondb/org.eclipse.smarthome.core.items.Item.json as it is to backup /etc/openhab2/items/*. It’s stored in plain text just like .items files so you can use git or the source control of your choice to track changes and such. openhab-cli backup also backs up everything in the JSONDB, as does Amanda or any other automated backup system. And unlike .items files, OH generates backups of the JSONDB files automatically (/var/lib/openhab2/jsondb/backups) whenever you make a change. Thus, it’s actually even easier to revert a bad edit than with .items files because the older copies are created for you automatically. And because the editing of the JSONDB isn’t usually done directly, it’s much more difficult to make a bad edit as the core will prevent anything syntatically incorrect from ever being written in the first place.

The only thing that is harder is manually editing them with a text editor, which is reason enough to stick to .items files. But backup and restore is not a compelling reason as it’s not harder than it is for .items files and in some ways superior.

It largely depends. If you have 1.x version bindings I don’t know if it is known yet whether the binding configs will still work when you migrate to Managed Items. They load faster than .items files and I don’t think they are as prone to experiencing the startup timing issues you often see (e.g. Rules start running before all the Items are loaded). Indeed you will have the ability to modify the Items through PaperUI and future UIs.

When you go into more esoteric stuff, when you have Managed Items if you are using Item metadata in Rules (only possible through NGRE) the metadata will persist across OH restarts. Any dynamically created metadata gets wiped out for Items loaded from .items files. See Design Pattern: Using Item Metadata as an Alternative to Several DPs for details.

Thanks for the clarification.
Yes I think backing up can’t be a problem with this. Plus if I’m right, auto-backups are already there, so maybe you are already one step closer to not losing your data. Once my Thing Json got corrupted and I was able to restore it from an auto-backup.

I’m only using the HTTP binding which is still a 1.x binding (but sometime maybe the new version will be here).
I will read the linked post. Little off topic but maybe not that much: Do you have any reference on how to start to migrate to JSR223? What is mostly used there? Jython?

Is it worth to also upgrade to JSR223 yet? As the linked post says, ther are already functions which the DSL does not support. Any other advantages with JSR223?

Hopefully soon. And if not, you can do anything the HTTP binding can do in a Rule using the sendHttp*Request Actions in the mean time if you decide to go down that path.

Jython is most commonly used and the Helper Libraries for it are the most mature. But JavaScript is also pretty popular. Groovy is supported but not widely used.

I think it’s worth it now. Some advantages include:

  • library support (e.g. I’ve already submitted a number of reusable scripts and modules to the helper libraries that you can just use, no need to copy/paste/edit code from the forum) https://github.com/openhab-scripters/openhab-helper-libraries/pulls

  • it’s faster, though that almost never matters in this context

  • they do not suffer from the thread pool limitations described in Why have my Rules stopped running? Why Thread::sleep is a bad idea

  • the Helper Library delays the loading of the rules to avoid the startup problem where Rules start running before Items are loaded

  • they use standard languages which means there will be tons more resources in books and online to learn the languages (e.g. Free "Python for Beginners" video series)

  • also, because they are standard languages, you can use common programming concepts not available in Rules DSL like classes, functions, first class data structures, etc.

  • there are a number of OH things you can do that can’t be done in Rules DSL including:

    • enable/disable rules (see https://openhab-scripters.github.io/openhab-helper-libraries/Guides/But%20How%20Do%20I.html#enable-or-disable-a-rule-by-uid)

    • dynamically create rules (i.e. Rules that create Rules)

    • create/edit/delete Items from Rules (look at the OpenWeatherMap forecast library in the Helper Libraries or the test for the Countdown Timer module PR I’ve submitted)

    • access and modify Item metadata (link above)

    • there is a script unloaded function you can add to a file where you can clean things up when the script is unloaded/reloaded; very handy for cancelling running Timers to avoid errors caused by orphaned Timers

    • Rule triggers can be dynamically created at load time (see the drop in replacement for Expire 1 binding PR for an example)

    • they support a config file so, for example, you can define information in configuration.py and import that into scripts (see the generic presence detection PR for an example where the Groups/Items that indicate presence are defined in configuration.py and used to construct the triggers and access the right Items in the Rule, making it reusable as is, no editing required)

  • they are compatible with (i.e. run on the same NGRE) Managed Rules (rules defined through the UIs) which will probably become the default way to define Rules in OH 3

  • there is the ability to create unit tests for your Rules built in.

I’m sure there are more good reasons. There are some good reasons not to move though:

  • it’s a lot of work

  • the tooling in VSCode is not as good as it is for Rules DSL meaning you will have more errors that won’t be detected until run time so testing Rules is more important (on-the-other-hand, as time goes on you will probably find you will be reusing other people’s code more than writing your own)

  • you’ll have to potentially learn a new programming language

  • most of the examples here on the forum and elsewhere are still in Rules DSL (I’m working to add at least Python examples to all of my design pattern posts and trying to build libraries out of those that warrant it (e.g. the Gatekeeper DP has been submitted as a PR already)

  • installation is a little awkward

1 Like

Thanks for this! I’m more than concerned that I will move to. Mainly because it is more natural as you said, using functions is a really big help for me.

Learning a new language is always not that bad. I’m a CE student so I already have to know the most of the languages :slight_smile:

Only the migration seems a lot of time and mostly a lot of time until I get used to this…

I could be wrong, but I don’t think so, unfortunately.

I’m moving the tool out of the developer tools section, by the way (it’s on the items list now, as an option besides “add item” and is now called “Add items from textual definition”), and making it a little more user friendly:

Now if the parsing is successful it will show the results in a table, and it’s also able to tell you if an item with the same name already exists, and whether it’s editable or not (because it’s provided by an .items file). If you’re trying to overwrite a non-editable item it won’t let you proceed - it wouldn’t work anyway. Not done yet, but I suppose it could also check the channels, making sure they exist.

1 Like

By the way, one nitpick from the Paper UI. in Z-Wave device properties, please do not convert the manufacturer id, device type & device id to decimal. Hexadecimal is the standard & we just have to convert it back to be useful.

I don’t think that the Channels not existing should be a blocking problem. I could see a case where someone has temporarily removed a binding or a Thing with the intention to restore it later. In that case, the fact that the Channel doesn’t exist wouldn’t really be a problem. It’d be nice to know about though.

1 Like

That happens quite often to pick up new changes in the zwave binding, for instance. the Thing is re-discovered with the same thingid so Items do not break. It should be a warning or caution, but not blocking.