You see, it’s not just not just items and things. Everything in my conf directory is under version control – html, icons, items, persistence, rules, scripts, services, sitemaps, sounds, things, transforms. I have three installations, so keeping track of all the changes to the configs becomes really important.
I assume that quite a lot users are using some sort of version control system for their configuration. That’ s one of the reasons why I started to use text files for my item definitions some weeks ago. I even store the JSONDB in git.
But some sort of version control could also be implemented in a graphical environment. Maybe even using git in the backend.
After following the discussion for some time I have the feeling that it goes round in circles and will not come to any conclusion (it reminds me a bit of the BREXIT discussion in Britain).
I think we can agree that most of us have the following requirements (in addition to generally improved graphical UI):
some sort of version control for all definitions
backup/restore function for all relevant definitions
export/import option to make it easy to copy definitions to other OH installations
copy/paste of definitions plus the support of “mass changes”
efficient editing especially for large installations with hundreds of items
I can understand @David_Graeff 's arguments (at least partially ). If you want to implement new configuration options it is in most cases easier to add new options to a GUI than to “invent” some sort of syntax for a textual representation. If you look at the item definitions and all the different types of brackets, then we already have a syntax that is not so easy to understand.
I don’t know whether it is possible to implement all required functions in a GUI that satisfies both new and experienced users. But we should a least try to.
Indeed, runtime. I’m not interested in the runtime configuration. I don’t even see the need to store it to disk. My source of truth are the configuration files that I write that are under my control. From security perspective it would even be good if openHAB didn’t have write access to them.
Each time I start openHAB I want to be sure that it uses my configuraton, not some automagically updated config hidden in an unreadable format (I know its JSON but that doesn’t make it readable).
But I guess this is where we have a fundamentally different opinion on what the source of truth should be. In your opinion it should be the JSONDB, in mine the text configuration files.
Is always the jsondb for openHAB. .things/.items files are creating a second source of truth here.
A configuration file in contrast is read once at start-up. .thing/.items files are not configuration files at the moment, they are like a 2nd backend/database.
If you would have to have your .things/.items as the one and only source of truth, the implication would be that openHAB writes back on any changes like discovered and accepted things on the UI etc.
Personally I don’t care how it works. I’m a fairly new user and my needs are simple, I’m comfortable with text file and once I figured out that Items and rules beliong in text files and things belong in PaperUI I did fine setting up what I needed to please my wife.
But, if you expect this product to grow, you need to make it so my wife can make changes which means it all has to be through a GUI. The way it is now and if you keep requiring text files, 99 of every 100 people who come here will move on.
Making text files an option so that the people stretching the edges can keep stretching makes perfect sense, but home automation should be for everyone and openHAB needs to realize that or something else will come along.
How it works underneath should not be a concern of the users because if it works, the majority of the users will not know or care. If you think it matters, become a developer and show the team a better way, isn’t that what open source is about.
Hm but that is how OH2 is build internally, it allows multiple ways to access the internal database (jsondb), but that has to happen in a structured way. The console and the rest interface are using the correct interface. And the next logic step is to make the file inputs also use that interface. Please accept that technical fact.
You are probably just used to the OH1 way of thinking. But this is OH2.
This topic is turning into the same conversation like in the other discussion. There’s a poll in the beginning, saying that 90% of people like GUI and textual ways.
6% GUI only and the rest textual only.
The poll unfortunately doesn’t differentiate between people who want a textual interface for copy/paste, search/replace, cloning or real text files.
Nonetheless. OH and also the developer environment must become more newby friendly (but stay powerful of course). Please ask yourself what compromises you are willing to offer for that goal.
I think you are still not understanding what I am proposing. The yaml file is a drop in replacement for the jsondb. But with the yaml file structure it is possible to move configurations into another yaml file or while it still can be used with the yaml parser. This provides the flexibility for proper configuration.
Example:
Openhab does auto discovery and detects a z-wave device and links some items. This is what gets created
Note that the openhab.yaml is still valid and can still be written to by the parser.
Openhab registers the file changes and loads everything back.
A part of the gui configuration became now manual configuration.
→ Benefit of db & gui but still allows full manual configuration
With a good concept there is no compromise needed!
I think David’s point is you cannot have static read only text configs and automatic discovery of Things. That is a hard break. Since removal of automatic discovery of Things is not up for discussion we require OH to be able to write this information to disk.
This is the choice. Either get rid of automatic discovery, in which case you may as well go back to OH 1.8, or come up with a more reasonable way to handle the fact that configuration information can come from both the user and OH itself.
See the Hue binding. You configure just the bridge, and for the light and control Things, auto-discovery works just peachy. I believe this would also hold true for MQTT things via the Homie convention. So, in my mind, it is possible to find proper sweet spots between all-text and all-GUI. This is also the way I would expect my dream HA system to work.
The “static read only text configs” provide only just the minimum necessary for the rest of the system to work. Which does not preclude us die-hard greybeards from fully defining all attributes of the Things ourselves, of course
And where do those automatically discovered Things get saved? The binding doesn’t go out and rediscover the Things every time OH restarts. It discovers them, you accept them out of the inbox, and they get saved.
Unless you are wanting to have all the Things be rediscovered every time, in which case that makes it impossible to set any configuration parameters on discovered Things because there is nowhere for those parameters to get saved.
Yes, but tl;dr that will not work. Let me elaborate.
I got you so far and I now understand why you think that this would be possible.
Yaml, Json, XML doesn’t matter. All those markup languages have “external reference” syntax, not just yaml. So in theory, we can do that today with just json and json-schema. But that is actually unnecessary if you watch an entire directory anyway, isn’t it?
You imagine that openhab.json would continue to be the runtime storage and your side-files are now “manually maintained”.
And that is where you are lacking technical implementation details. Json DB as we call it is actually just a serialized copy of an in-memory state (java objects).
The file system is only used as “backup” and is read once on start-up to fill the in-memory state.
And it can only serialize the exact state of the jave objects, no artificial file boundaries are considered.
This is fundamental to openHAB and it can’t be changed over night, nor should it be changed. In-memory is always faster then to wait for the filesystem to confirm a write transaction. Currently the jsondb file is written out every 5 seconds (if a change happened in that cycle).
Discovered things can simply be saved to .things files. If you can save them to a database, then you can save the same data to a flat file.
Currently they get saved to a database which also contains application runtime state. Mixing application runtime state with application configuration is in my opinion a very bad idea.
How are you going to do a clean start? By first manually editing the JSONDB and removing everything that has to do with state? I don’t know if that is even possible without either missing something or accidentally breaking something.
Why do we like Docker so much? Because it packages everything the application needs and allows you to basically run everywhere, but also because it gives us the possibility to start with a clean sheet each and every time we run the container. Someting goes wrong with the container, we simply throw it away and start fresh. We know that the new container instance will run because the last time we started with a new container it also ran. We start from the exact same baseline.
If I start openHAB with a JSONDB that contains runtime state information from a previous instance then I can never start from a known working baseline because that baseline has been contaminated.
Why do we even store state to disk? As soon as openHAB is stopped we must consider the state invalid because the world around openHAB has changed and we cannot guarantee that the state that openHAB persisted is still valid. It’s the same as the Wasp in the Box principle when the box is opened. Until the box is closed and we see or hear a life sign of the wasp inside the box we must assume the fact that the wasp is no longer in the box.
The only benefit I can see why we need to persist runtime state information is for debugging. I had to edit the JSONDB once because for some reason removing a binding using PaperUI didn’t go well and parts remained in the database. This would not have been necessary if the JSONDB had been just an in-memory database. In that case a simple restart would have sufficed.
Storing to disk is slow and doing this every 5 seconds (if a change happened in that cycle) causes unnecessary wear on the the SDcard used by many SBC’s.
When storing a file to disk there is always the chance of data corruption. And guess what, we’re storing runtime state to the same file that contains our carefully crafted application configuration. We’re taking that risk up to 12 times a minute, 720 times per hour, 17280 times per day, which leads to 6.307.200 times each year. That worries me.
Why not? He already included a storage association in his design study. Simply save the Thing to file and I can set the file attributes to read-only as a hardening measure. So I don’t think that is what he meant.
I said “'nuff said” because I’m getting tired of this UI only tunnel vision. Everything needs to be done via the UI, even to the extend that VS Code is now integrated into the PaperUI-NG design study. Why?! Microsoft releases new versions of VS Code every month or so. There are two or three openHAB releases per year. The included version of VS Code is already old by the time a new openHAB version is released. It’s not exactly DRY (or better DRO; Don’t Repeat what Others can do better ).
There are so many great editors out there already.
But I think I’m going to step out of this discussion for a while and see how things develop, and if needed, build my own tooling around this (should I still feel the need).