In my very first proposal I saw an import process as a replacement process. JSONdb would only contain what is specified in thing/item files.
In later proposals this is more like an import. So only adding / modifying.
But actually you can have both. Just imagine there are two triggers. One will append/modify, the other will remove/modify/add.
Summaries might be/are biased, so to get the full picture Iād recommend reading through.
There are some interesting ideas in this thread and it would not be just to condense them to something they are not.
I tend to agree with Sebastian. It will be painful but there are a number of unique perspectives here that will be hard to summarize.
I think David presented the summary of his position pretty well, though I think part of what riled up a bunch of users was his going in position that perhaps we donāt need text config at all and just use the GUI.
Iām sure Iāll forget some things but in my mind the requirements that have come out of this whole thread are (sorry for not providing attribution, itās just too much to go back and find who posted what). Note some of these are contradictory:
For each of the above there is at least one user who feels strongly that the requirement should be met and has good reasons for their belief.
I think if I had to make a tl;dr, developers want a text based config that allows them to work with OH like any other development platform using source control, repeatable automated deployment, and the like. This is at odds with the current JSONDB internal database which has a mix of runtime and config based stuff and no way to get stuff out of it in a way that supports this approach.
One thing that David has treated as a fixed quantity is that JSONDB shall remain and JSONDB shall be the āsingle source of truthā for OH config. But based on your comment on the Architecture thread that mentioned this, I donāt think that it is fixed. Replacing JSONDB in whole or in part I think absolutely is on the table. Perhaps we can find more common ground if that is taken as a consideration. Can we meet more of the requirements listed above (and any I missed) with a different approach to how OH stores itās internal config and internal state?
@David_Graeff, assuming that you could make JSONDB work however you want it to work, would this change anything about your approach? Donāt worry for the time being about who would do the work to change out the DB. Letās just brain storm how you would design the āsingle source of truthā storage and how that might change anything recommended above. I get the feeling that a lot of the restrictions you placed upon yourself stem from treating JSONDB as a given. What if you didnāt?
@Spaceman_Spiff, can you elaborate more about how you would design OHās internal config strorage down to the next level?
Anyone else please feel free to join in.
Letās try to be constructive, friendly, and cooperative. No one is going to unilaterally make any decisions on this topic.
If I could over night change OHs code base then @Spaceman_Spiff suggestion is very reasonable. The runtime data (thing properties, channels, object references etc) is stored in fast binary fashion. The configuration in separate files. But the configuration would still be written to by openHAB in some cases, like auto discovery or when a binding stores thing configuration (like session tokens etc).
But thatās a only a dream considering how openHAB internally works.
If we can get everyone on board perhaps it is a dream we can make a reality. We have the opportunity now to present a case for why the way it internally works is holding us back or causing other problems. Iād hate to come up with a suboptimal solution just because we didnāt consider all the options.
Perhaps Iām taking this statement beyond what was intended but I read Kaiās bullet point from Directions for openHAB 3
- Simplify switching from UI to textual configuration and vice versa - maybe through a new database format, by export mechanisms or whatever.
to indicate that major changes or replacing the JSONDB is not outside the range of consideration.
Yes Kai is brainstorming here, I read I like you as well. But that subsystem was just replaces in oh 2.2. Why not first address persistence and sitemaps. An import/export service like I have proposed it is written in a weekend. And this one conversion tool of that other guy already allows to import and export items for example. So migration wouldnāt be the issue.
Thanks @David_Graeff and @rlkoshak for your summaries.
I am indeed open to ANY solution and my bullet point that you mentioned above was indeed referring to this discussion here - whatever the outcome is.
Maybe some additional infos:
This sounds like the multiple truth problem which came up in the thread a couple of times.
Imho the problem is that there is an internal database which not only stores runtime data (otherwise the file could just be deleted) but it also stores configuration data. It is essential that there is a differentiation between runtime cache (e.g. to improve startup speed) and configuration.
Maybe a ConfigProvider and a CacheProvider so everything is still pluggable?
From what I have read in the thread nobody actually wants Xtext code. It just happens to be the technology which is currently used. Removing Xtext for config files (only) could also be a smaller task since itās quite easy to write a parser for the file format (I wrote an config checker in python which checks for logical errors a while ago). Creating and writing to the files is a whole other story though.
Itās because json is actually a machine-interchange format which just happens to be readable 
.
The current format is indeed very compact, but as a trade-off the structure is different for items, things, sitemaps, persistence, etcā¦
I therefore think itās best to use a universal file format which can be used for all these configuration, even though it means that it will be more explicit and less compact.
As I pointed out in my earlier post yaml is the way to go because when done correctly it even provides the flexibility to the user to split configuration across multiple files as he wants (e.g. moving certain sections on any level to another file, not just merging files on a top level together). It also is human editable and allows serialization/loading of custom classes.
The JSON DB is not about any runtime data, it is pure configuration. And if you do not use any DSLs, they are the one and only truth in your system.
Removing Xtext for config files (only) could also be a smaller task since itās quite easy to write a parser for the file format
Well, as mentioned, you are losing the LSP feature with it, which is imho very helpful. And having a custom parser can also be complex as soon as you start doing changes to the file format. With Xtext, it is actually pretty easy to add additional properties as all you need to do is to change the grammar.
As I pointed out in my earlier post yaml is the way to go because when done correctly it even provides the flexibility to the user to split configuration across multiple files as he wants
Would be cool if youād come up with a POC for a yaml syntax and a parser. As pointed out above, we could easily add this as a second (experimental) option, so that people could give it a try how it feels.
I see, but if I use another source of configuration there is the need to keep them in sync. As a user however I only maintain one source of configuration so this will always be the cause for problems.
Imho caching of configuration should be delegated to a ConfigProvider or at least not be persisted (e.g. if done manually).
Iād happily provide something, but I am neither fluent nor good in Java.
I could provide a POC in python however if that would be fine, too?
It would be just a converter to the current *.items etc files
Not strictly true. Markus has valid concerns about the disruption that will occur when/if all the current users of OH are forced to recreate their entire configuration and Rules again. Iāve not voiced them as much but I have such concerns too. Itās a thorny problem. I do believe the majority of commentors in the thread do agree that we are OK with it going away, as long as we have tools and aides to help us transition to the new format.
I like this idea. If we have some devs willing to try it out we can prototype it and see how it goes.
Not strictly true. Markus has valid concerns about the disruption that will occur when/if all the current users of OH are forced to recreate their entire configuration and Rules again
I think that was adresse when @Kai said:
I would actually LOVE to get rid off the Xtext code (while not necessarily changing the file syntax, this would be another question).
Indeed, a nuance I missed. Thanks!
I was just going to say the same thing. I think itās important to distinguish between xtext and the grammar. Thereās no linkage between the two other than the current implementation. It seems everyone is ok with getting rid of xtext. Itās the file format (i.e. the current grammar definition) that we would like to be preserved.
Exactly - next time Iāll try to be more specific because thatās exactly what I meant. 
I also read somewhere (in this thread) :
Related to that, I think a certain order in which files are loaded, is needed. (maybe more related to repolling files
Indeed those topics did come up. I didnāt want to put them in as requirements because they really are bugs in the current OH more than a requirement for how the future OH should do things. But indeed, any new config file system would need to not reintroduce or preserve these problems.
let me rephrase:
File/Load order is definitely an issue.
For startup time Iāve read that most of the time is the rule engine compiling rules, so this is (at least partially) out of scope.
There were already some discussions in the openhab-vscode repo (just one example: Concept on how to deal with LSP features Ā· Issue #123 Ā· openhab/openhab-vscode Ā· GitHub). Some functionality has already been implemented on the client side in an additinal āclient-sideā LSP.
One of the reasons for this implementation was performance especially if the LSP is running on small machines like a raspberry.
It would be great if we could find a way to describe the syntax in a way that allows the creation of a parser for Java (for openHAB backend) and TypeScript/JavaScript for VSCode. Maybe it is possible to replace the XTEXT based parsers e.g. by ANTLR generated parsers.