The current way openhab is kind of special and very confusing. Paper UI items and things are completely different to the config items. Since bindings use the paper ui setup, but if you want to have any non-trivial setup it’s really confusing to try and work with the two completely separate setups. The config files should mirror the paperui items, and vise versa. It took me months to understand why paperui wouldn’t let me edit some config items.
I used YAML with homeassistant and hated it. I thin the main issues was that it’s all based on whitespace which is hard coming from a c background, and the fact I didn’t have an editor which raised any issues. You also had to restart oa after any changes so it was just too much of a pain in the ass. But if there is proper vscode support for the syntax then it shouldn’t be so bad.
I use YAML for Ansible all the time and PyCharm at least manages the white space warnings but I find they are rarely needed as the proper white space gets inserted automatically on every newline. Over all, YAML is chosen because it is easier to work with than JSON but I believe whatever gets implemented will support multiple formats should someone choose to implement and add-on to support it. Of all the structured markup langauges I am familiar with (JSON, XML, and YAML), YAML is by far the easiest for a person to work with so it is probably the best choice for now.
Nowhere will we find it acceptable to have to restart OH to make a config change and that is not being proposed here. We will have an import/export function that doesn’t require a restart. So that should not be a fear.
Here is a quick mock up of one way it could look. This is just one option and it may not be the best. I split it up by strategy. One could also split it up by database or I’m sure other ways as well.
The root of the issue is there needs to be a mapping between three things:
Database that the states get saved to
Strategy used that determines when states get saved (every change, every update, periodically based on a cron expression, restore on startup). It gets complicated because the user needs to be able to define their own strategy, though we should offer the most common ones by default (everyMinute, everyHour, everyDay). That is what the cron dialog is all about.
The Items that get saved to the selected database using the selected strategy. It gets complicated with Groups because one can save all members of a Group (in the .persist file we use *), or save the state of the Group itself. This makes the selection complicated.
But as I think is pretty clear from the rest of the discussion, there doesn’t appear to be anything in the core at this time (though maybe there is given Scott’s link). But either I’m missing it or there is no REST end point to set these strategies.
I like the idea of splitting up the strategy configuration and DB/Strategy assignment, but would it be easier to just add the DB/Strategy assignment to the Item confiugration screen? That way, the persistence could be assigned at time of creation/configuration of the item (which is something of an after-thought and a dedicated secondary step in the current OH1/OH2 realm), it would be more intuitive to users that Items need direct persistence assignment (until recently, I was confused myself that the “*” symbol in .persistence files isn’t a wildcard that lets me assign the same strategy to multiple items), and we could even show that items are already persisted (as part of a group), but still allow the user to select a secondary/direct persistence strategy/DB.
As I was making this that did come to mind. My only problem with that approach is handling the “All Items” case. I’d have to manually add something to all my Items which seems excessive. Or we treat “All Items” as a special case and can select it where we create/edit the strategy. So I opted for consistency in this case. But like I said, this is one way and probably not even the best. As long as we have a way to handle the “All Items” special case I don’t really have a strong opinion either way.
I suspect the whole persistence subsystem needs a review and rework. Maybe something better will come up.
This is actually exactly why I created Design Pattern: Group Based Persistence. You then just assign the Group to the Item to choose which persistence strategy get’s used for that Item.
I personally converted dynamodb persistence to be compatible with openHAB2 but still using compat layer since the new APIs seemed “raw”… For details check the discussion in the thread
Alright that means I’ll come up with a UI and we try to extract necessary functionality out of that study to design a REST interface. That allows later on to create an API bundle. And at that point people can individually work on persistence bundles that implement that API.
For oh3 we need one working implementation of course.
Just some crazy idea: why not persist the state of all items by default onEveryChange when persistence is configured? (I understand it requires some additional logic for rrd4j, but that would be an presistence specific implementation detail)
By the time we get to OH 3, we should have a way to view and edit Item metadata. You can currently add and remove it though the REST API, but you cannot view it. It would be nice to view and edit metadata in the Item details.
There are a ton of use cases for Persistence and that “strategy” doesn’t fit with all of them. For example:
sometimes we need to save when an Item is updated, not just changed
to generate some charts for slow changing Items sometimes it is necessary to save the Items current state periodically whether or not it changed or else the charting software will not draw the line on the graph
at least one of the databases requires an every minute strategy just to work (rrd4j)
sometimes you only want to save the state after some filter is applied to the data to get rid of bad readings (we need a Rule for this)
Not to mention that saving every change of every Item in every database will be a TON of writes (sorry SD card users) and generate a huge amount of useless data stored in the database.
It really is a shame that Persistence has been ignored up to this point because it is a core feature that many of us absolutely depend upon for telemetry (i.e. charts), alerts, calculations, and to dive logic.
Here is a good concrete example for why we need more than one strategy to support. Let’s say we have a slow changing Item, maybe a Switch, that we want to chart. I do this myself to chart whether the fan or the heater is on on the same chart as my indoor temperatures.
In order for the chart to work properly, I must have data stored in the database at least every 10 minutes or so. Otherwise when I zoom in too far the line goes away because there are not enough points to generate the chart.
But, I also have some logic that needs to know when the last time the Item changed was. I can’t get that information using the every 10 minute strategy because the lastUpdate really means the timestamp of the most recent value stored in the database. With an every 10 minute strategy, that timestamp is not guaranteed to be the time when the Item changed. And we can’t use previousState(true) because that will give us the time when the Item changes to the previous state. There is no way to determine when the Item changed states to its current state. So we would need to store the state of this Item using two different strategies for the two different use cases.
This will be increasing important for the NGRE because there is no way to set global variables in it. So metadata is going to have to pick up some of that slack.
To add to the list you mentioned, even though the SD card writes is not an issue with external persistence services (e.g. Dynamodb), persisting all may have a very detrimental effect to openHAB performance.
Since persisting actually does take some time, all internal queues eventually keep on growing, making the system unoperable.
I will apologize beforehand if this was already discussed/proposed I don’t have the time currently to read the full thread.
For having a single way of storing items/things/binding configuration : cannot we simply use the current syntax (parser already exist) and store config for the different elements in file using same syntax? No need to have a new parser, easy way to go from file to paper UI and vice-versa.
And there is really a long heated discussion on textual files that resulted in this design study and I feel personally a little offended by you, that you do no take the time to read this thread (click on “summarize” and you get the picture already) and suggest to use the old syntax instead.
I thought about adding a tab to the item in grid mode. But I don’t really know how to present meta-data and what it is used for. Semantic tagging is done per meta-data isn’t it?
There is a way to use global variables. Jython 2.7.0 definitely works with global variables. In finding a fix for Jython 2.7.1, I learned more about scriptExtensions, which can be used by any language and can be used for global variables. They basically put objects into a shared context accessible by all script engines (all scripts have their own).
As I’ve mentioned elsewhere, I also have Jython and Groovy working for scripted Actions. I will submit a PR once ESH is remigrated, and workup an example for using scriptExtensions.
Metadata is the next best thing! For example, I have metadata in most of my light Items with values for a lux level and brightness setting for each Mode (what I use for TimeOfDay, since I have Party, Vacation, etc.). When the outside lux changes, I have a rule that adjusts the lights according to the metadata for each Item in an active area of the house. I have another rule to detect activity in an area, which turns on/off the lights to values based on the metadata. Previously, this data needed to be stored in Items or in some other object in a rule script. I could add listener’s too, to watch for changes in metadata, but haven’t had the need.
For UI layout, it would be a tree for each Item, with Keys at the root, and branches of name value pairs. The data should be editable, but not sure the best way to design the UI for it.
Metadata essentially provides additional data points, beyond e.g. an Item’s state. Here is a writeup that I did…
Eventually, I will get a post about how to setup Area Triggers and Actions, and the rules I use. Once I get the PR out, I’ll distribute them as rule templates! Although, I have DSL versions too.
I had a look at the REST API and I’m not happy with the design. I cannot request a list of metadata namespaces. I can only query metadata if I know the namespace beforehand.
So I can’t really integrate that into the UI yet. The core must change first and allow querying for namespaces.
Please have a look at how I see this solved. Every thing/item/rule/timer has a “storage association” (aka filename). The backup service will consider those for exporting to several storage units (aka files).
Every thing/item/rule/timer has a list, grid and textual view mode. The latter one allows find/replace, batch editing. For the format I have settled on YAML for now. Please have a look and tell me if you like that syntax. Maybe there is a better suited format that I have not yet considered.
I love the “storage association” concept, I know it from other products, and its very flexible for automation, backup and the choice between UI editing and good old “vi”
But I do hate YAML so much, but I can see that it is gaining a lot of land being implemented in many, many products. Personally I do swear much more to JSON, even though I dont admit its harder to read in an editor, but its ensy to handle, simple and well known and supported in all programming languages and tools and REST interfaces.
It is one single line of code (maybe two) to be changed in Paper UI NG to see the original json instead of YAML. I can imagine that being an option in the future.
I’m writing this now, as I don’t have much time tomorrow.
Can connect to a real openHAB instance now. If you use the hosted version in the link, your openHAB must be served via https though. (Use nginx as proxy for example on openhabian).
Login page, including “demo” data-set for experimenting.
Proper about page with open-source attributions (legally required, I think).
Things/Items page:
Added/Changed sorting options for items/things. They are also propagated to the local database already, but I haven’t decided yet what to do next.
Add item -> Although I wanted to avoid dialogs, I think it fits here
You can send item commands to your openHAB instance, yeah. But that’s it already for OH interactivity.
Shortcut for list/grid/textual mode (Alt+l, Alt+g, Alt+t in Chrome). The modifier key depends on the browser and platform. MacOS uses the command key instead for example.
Inbox:
Nice animation while performing a manual discovery
Maintenance:
Persistence page added
Service/Binding/Things:
Configuration pages are completely rendered from dynamic data now
Documentation:
I have started to document the project
Help required
Please help with …
proof reading all context help texts,
and writing the tutorial pages. We need some nice diagrams, videos, pictures and of course text to introduce newbies to the interface and openHAB in general. The text should be concise still, no complete novels
I’ll have a look and play and see what we can do. Probably won’t be until this weekend or next week. Should this be captured in the README.MD file? Or should we start a separate thread and put it into the docs directly? I’m not sure what makes the most sense. @Confectrician, do you have an opinion? Where does existing (and very thin) PaperUI docs come from?
I take this to mean if I check it out and run it from the html/static directory it will connect to my running OH? Awesome!
But OH is already served on HTTPS on port 8443. Do I need to set up nginx to support a LetsEncrypt cert or something else? Or will it work on port 8443?
Well, I got it to run on my OH install following the instructions on github but it doesn’t appear to be connected to my instance of OH. Is there a setting somewhere or am I jumping the gun?
Locally it just works. I’m talking about the “hosted” instance of paper UI NG. It is served via Https from github and mixed content protection of modern browsers prevent to connect to oh, running somewhere locally on http only.
