Ah, o.k., my bad. Did not think about the “accept” part and futher manual customizing (which I tend to avoid if at all possible)
I literally made a suggestion how auto discovery and manual configuration can be implemented in the post above yours.
Yes, but tl;dr that will not work. Let me elaborate.
I got you so far and I now understand why you think that this would be possible.
Yaml, Json, XML doesn’t matter. All those markup languages have “external reference” syntax, not just yaml. So in theory, we can do that today with just json and json-schema. But that is actually unnecessary if you watch an entire directory anyway, isn’t it?
You imagine that openhab.json would continue to be the runtime storage and your side-files are now “manually maintained”.
And that is where you are lacking technical implementation details. Json DB as we call it is actually just a serialized copy of an in-memory state (java objects).
- The file system is only used as “backup” and is read once on start-up to fill the in-memory state.
- And it can only serialize the exact state of the jave objects, no artificial file boundaries are considered.
This is fundamental to openHAB and it can’t be changed over night, nor should it be changed. In-memory is always faster then to wait for the filesystem to confirm a write transaction. Currently the jsondb file is written out every 5 seconds (if a change happened in that cycle).
Cheers
Discovered things can simply be saved to .things files. If you can save them to a database, then you can save the same data to a flat file.
Currently they get saved to a database which also contains application runtime state. Mixing application runtime state with application configuration is in my opinion a very bad idea.
How are you going to do a clean start? By first manually editing the JSONDB and removing everything that has to do with state? I don’t know if that is even possible without either missing something or accidentally breaking something.
Why do we like Docker so much? Because it packages everything the application needs and allows you to basically run everywhere, but also because it gives us the possibility to start with a clean sheet each and every time we run the container. Someting goes wrong with the container, we simply throw it away and start fresh. We know that the new container instance will run because the last time we started with a new container it also ran. We start from the exact same baseline.
If I start openHAB with a JSONDB that contains runtime state information from a previous instance then I can never start from a known working baseline because that baseline has been contaminated.
Why do we even store state to disk? As soon as openHAB is stopped we must consider the state invalid because the world around openHAB has changed and we cannot guarantee that the state that openHAB persisted is still valid. It’s the same as the Wasp in the Box principle when the box is opened. Until the box is closed and we see or hear a life sign of the wasp inside the box we must assume the fact that the wasp is no longer in the box.
The only benefit I can see why we need to persist runtime state information is for debugging. I had to edit the JSONDB once because for some reason removing a binding using PaperUI didn’t go well and parts remained in the database. This would not have been necessary if the JSONDB had been just an in-memory database. In that case a simple restart would have sufficed.
Storing to disk is slow and doing this every 5 seconds (if a change happened in that cycle) causes unnecessary wear on the the SDcard used by many SBC’s.
When storing a file to disk there is always the chance of data corruption. And guess what, we’re storing runtime state to the same file that contains our carefully crafted application configuration. We’re taking that risk up to 12 times a minute, 720 times per hour, 17280 times per day, which leads to 6.307.200 times each year. That worries me.
1 Like
Why not? He already included a storage association in his design study. Simply save the Thing to file and I can set the file attributes to read-only as a hardening measure. So I don’t think that is what he meant.
I said “'nuff said” because I’m getting tired of this UI only tunnel vision. Everything needs to be done via the UI, even to the extend that VS Code is now integrated into the PaperUI-NG design study. Why?! Microsoft releases new versions of VS Code every month or so. There are two or three openHAB releases per year. The included version of VS Code is already old by the time a new openHAB version is released. It’s not exactly DRY (or better DRO; Don’t Repeat what Others can do better 
).
There are so many great editors out there already.
But I think I’m going to step out of this discussion for a while and see how things develop, and if needed, build my own tooling around this (should I still feel the need).
I am not talking about external reference but rather custom classes which do load and merge the other files.
This is fundamental and why this concept works.
!!load_from_file items.yaml is not an external referance but a class that gets constructed with the external filename. It is then used to merge the manually created configuration with the automatically created one.
The class gets serialized to !!load_from_file items.yaml with a custom resolver but actually does contain the file contents because these get loaded in the constructor.
I am fully aware that the files on the disk are always just a serialized copy of in-memory state.
But this is no problem.
Let’s play a load and a dump through.
Assumptions
- openhab does serialize to/from openhab.yml and it contains a gui-configured thing “GuiThing”
- a custom thing “ManualThing” is defined in mything.yml
- in openhab.yml there is a class registered to load mything.yml
- there is a file-watcher in place for openhab.yml
- if there was a dump to openhab.yml the consequent file events will be ignored
Load
Openhab.yml is loaded. Custom class with external files is constructed and loads configuration for ManualThing. Configuration is then merged in memory and contains configuration for “GuiThing” and “ManualThing”.
Dump
Since all things/items/etc. have unique names dumping is easy, too.
First, check if changed thing is in the custom class. If yes, don’t serialize these changes and show warning message. This step is optional.
Serialize data to openhab.yml.
The external, manual files get serialized to !!load_from_file mything.yml so there is no need to know about file boundaries.
Conclusion
It is possible to create configuration through the GUI, but a manual configuration will always have more priority then a configuration through the GUI. There is only one source of truth. Automatic loading and dumping still works.
1 Like
You suggest to change half of the core. Please go for it, you know where the source code is.
Good - so we agree that this approach is valid and possible.
Tbh I only think this would affect configuration and serialization.
But this is the development section and we’re here to discuss concepts. And when introducing new concepts I expect that there are changes to be made.
While these changes may not be “low hanging fruits” they solve the problem of two truths while providing the possibility to configure through GUI and textual files. And this is the consens how everyone here wants to configure openhab.
1 Like
openHAB is prepared to use different storage providers. That is the easy part to switch to yaml.
But everything after that means a lot of work. Things would need to remember their storage association etc. Because the storage service doesn’t know any meta data about the objects its going to serialize.
But as I said, have a look at the code. Personally I’m happy with the internal state database as it is so will not do anything in this regard. I’m more interested in a fast start-up and a good UI. And currently .thing/.items files are parsed via XTend/XText which pulls in a gigantic 3prd party library that is slow and evolves slow (we can’t update to newer java versions at the moment because of that) and I just want to get rid of that mess and instability that it inflicts.
I was under the impression we are talking about configuration not about storage of items/things.
It is perfectly fine to have only one way of doing so ( and this is the status quo as you keep mentioning that everything gets loaded from jsondb).
Nobody wants the keep .thing/.items files nor the XTend libraries. The only dependency here is the yaml-parser and configuration would be done using the yaml syntax.
1 Like
And I was under the impression that you want to replace jsondb with yamldb. Because that is the only way to archive a single source of truth.
No - I want a consistent, logical and flexible way to configure things and I provided a concept how it can be done.
If that means to get rid of jsondb it’s fine with me.
1 Like
here’s a actual good UI nobody will ever ask for textual files again. Pretty sure.
You keep saying that, yet many times people have said that it’s not true.
I chose OpenHab because of textual configuration.
yes, I do believe there are powerusers who prefer GUI configuration, just as there are beginners who prefer text.
In the poll on this page most people selected both interfaces.
I’m not against a great GUI, I am pro also text configuration.
1 Like
I think David’s point is you cannot have static read only text configs and automatic discovery of Things.
The discussion on how the handle the automatic discovery is happening, so I’m ignoring that part here.
For me I don’t use automatic discovery from things.(especially because i use the same configuration on two openhabs instances that swap between production and test)
so having static read only without automatic discovery would work for me.
yes this means i need a way to set the all that config info also in the static files.
yet when I can do that, I’m ok with not being able to access the rest of the current state.
and yes I’m personally OK when that means that I’m limited to a reboot of openhab to do this.
My use of a source control system is more important then to have the current state of whatever openhab needs besides the configuration in files.
just out of curiosity, what bindings do you use?
It’s not clear who you are replying to. I always try to tag the user or quote from their post just to make it clear.
I doubt it was me but just in case:
Actively use:
- Bindings: zwave, astro, expire1, network, http1, mqtt1, mqtt, networkupstools1, nest, openweathermap,
- UIs: paper, basic, habmin,
- Persistence: mapdb, influxdb
- Transformations: map, jsonpath, regex, javascript
- Misc: openhabcloud, restdocs, market
Installed but not actively used, mainly legacy left overs or stuff I’m learning:
- Bindings: zigbee
- UI: habpanal, habot
- Actions: mail
- Transformations: xslt
Sorry Rich, I was asking Yves
thanks anyhow… I believe you’ve told me your personal setup before
binding = astro,exec,hue,mqtt1,netatmo,ntp,samsungtv,fronius,anel1,neato
hue and ntp not activly used at this moment.
anel1 is the biggest part.
ui = basic
persistence = influxdb
action = mail
all the bindings that need things are configured using .things
Why are you asking?
Sorry… as stated… just curious
I’m new to OpenHAB, still learning. This is an amazing discussion. I’m watching the development of this platform and am in awe of the work you guys are doing. Your opinion on the configuration is important to this discussion and I wondered how you used the platform
An example of why .thing files are just super confusing for people:
And another one:
Doesn’t seem to be so hand-edit friendly the current .thing file syntax, is it?
1 Like