Next generation design: A Paper UI replacement proposal


(Hilbrand Bouwkamp) #96

Just some crazy idea: why not persist the state of all items by default onEveryChange when persistence is configured? (I understand it requires some additional logic for rrd4j, but that would be an presistence specific implementation detail)


(Scott Rushworth) #97

By the time we get to OH 3, we should have a way to view and edit Item metadata. You can currently add and remove it though the REST API, but you cannot view it. It would be nice to view and edit metadata in the Item details.


(Rich Koshak) #98

There are a ton of use cases for Persistence and that “strategy” doesn’t fit with all of them. For example:

  • sometimes we need to save when an Item is updated, not just changed
  • to generate some charts for slow changing Items sometimes it is necessary to save the Items current state periodically whether or not it changed or else the charting software will not draw the line on the graph
  • at least one of the databases requires an every minute strategy just to work (rrd4j)
  • sometimes you only want to save the state after some filter is applied to the data to get rid of bad readings (we need a Rule for this)

Not to mention that saving every change of every Item in every database will be a TON of writes (sorry SD card users) and generate a huge amount of useless data stored in the database.

It really is a shame that Persistence has been ignored up to this point because it is a core feature that many of us absolutely depend upon for telemetry (i.e. charts), alerts, calculations, and to dive logic.

Here is a good concrete example for why we need more than one strategy to support. Let’s say we have a slow changing Item, maybe a Switch, that we want to chart. I do this myself to chart whether the fan or the heater is on on the same chart as my indoor temperatures.

In order for the chart to work properly, I must have data stored in the database at least every 10 minutes or so. Otherwise when I zoom in too far the line goes away because there are not enough points to generate the chart.

But, I also have some logic that needs to know when the last time the Item changed was. I can’t get that information using the every 10 minute strategy because the lastUpdate really means the timestamp of the most recent value stored in the database. With an every 10 minute strategy, that timestamp is not guaranteed to be the time when the Item changed. And we can’t use previousState(true) because that will give us the time when the Item changes to the previous state. There is no way to determine when the Item changed states to its current state. So we would need to store the state of this Item using two different strategies for the two different use cases.

This will be increasing important for the NGRE because there is no way to set global variables in it. So metadata is going to have to pick up some of that slack.


(Ssalonen) #99

To add to the list you mentioned, even though the SD card writes is not an issue with external persistence services (e.g. Dynamodb), persisting all may have a very detrimental effect to openHAB performance.

Since persisting actually does take some time, all internal queues eventually keep on growing, making the system unoperable.


(Oblaise) #100

I will apologize beforehand if this was already discussed/proposed I don’t have the time currently to read the full thread.

For having a single way of storing items/things/binding configuration : cannot we simply use the current syntax (parser already exist) and store config for the different elements in file using same syntax? No need to have a new parser, easy way to go from file to paper UI and vice-versa.


(David Graeff) #101

Accepted.

Clearly not. See MQTT 2.4 documentation as example why the current syntax is confusing and crap.

And there is really a long heated discussion on textual files that resulted in this design study and I feel personally a little offended by you, that you do no take the time to read this thread (click on “summarize” and you get the picture already) and suggest to use the old syntax instead.

I thought about adding a tab to the item in grid mode. But I don’t really know how to present meta-data and what it is used for. Semantic tagging is done per meta-data isn’t it?

Cheers


(Scott Rushworth) #102

There is a way to use global variables. Jython 2.7.0 definitely works with global variables. In finding a fix for Jython 2.7.1, I learned more about scriptExtensions, which can be used by any language and can be used for global variables. They basically put objects into a shared context accessible by all script engines (all scripts have their own).

As I’ve mentioned elsewhere, I also have Jython and Groovy working for scripted Actions. I will submit a PR once ESH is remigrated, and workup an example for using scriptExtensions.

Metadata is the next best thing! For example, I have metadata in most of my light Items with values for a lux level and brightness setting for each Mode (what I use for TimeOfDay, since I have Party, Vacation, etc.). When the outside lux changes, I have a rule that adjusts the lights according to the metadata for each Item in an active area of the house. I have another rule to detect activity in an area, which turns on/off the lights to values based on the metadata. Previously, this data needed to be stored in Items or in some other object in a rule script. I could add listener’s too, to watch for changes in metadata, but haven’t had the need.

For UI layout, it would be a tree for each Item, with Keys at the root, and branches of name value pairs. The data should be editable, but not sure the best way to design the UI for it.

Morning
    |_ Low_Lux_Trigger: 5
    |_ Level: 100
Day
    |_ Low_Lux_Trigger: 90
    |_ Level: 50        
Evening
    |_ Low_Lux_Trigger: 30
    |_ Level: 50
Night
    |_ Low_Lux_Trigger: 30
    |_ Level: 10 
Late
    |_ Low_Lux_Trigger: 30
    |_ Level: 1
Party
    |_ Low_Lux_Trigger: 5
    |_ Level: 100

Metadata essentially provides additional data points, beyond e.g. an Item’s state. Here is a writeup that I did…

Eventually, I will get a post about how to setup Area Triggers and Actions, and the rules I use. Once I get the PR out, I’ll distribute them as rule templates! Although, I have DSL versions too.

Semantic tagging uses tags, and also uses metadata for synonyms.


(David Graeff) #103

I had a look at the REST API and I’m not happy with the design. I cannot request a list of metadata namespaces. I can only query metadata if I know the namespace beforehand.

So I can’t really integrate that into the UI yet. The core must change first and allow querying for namespaces.


(Normann P Nielsen) #104

Please have a look at how I see this solved. Every thing/item/rule/timer has a “storage association” (aka filename). The backup service will consider those for exporting to several storage units (aka files).

Every thing/item/rule/timer has a list, grid and textual view mode. The latter one allows find/replace, batch editing. For the format I have settled on YAML for now. Please have a look and tell me if you like that syntax. Maybe there is a better suited format that I have not yet considered.

I love the “storage association” concept, I know it from other products, and its very flexible for automation, backup and the choice between UI editing and good old “vi” :slight_smile:

But I do hate YAML so much, but I can see that it is gaining a lot of land being implemented in many, many products. Personally I do swear much more to JSON, even though I dont admit its harder to read in an editor, but its ensy to handle, simple and well known and supported in all programming languages and tools and REST interfaces.


(David Graeff) #105

It is one single line of code (maybe two) to be changed in Paper UI NG to see the original json instead of YAML. I can imagine that being an option in the future.


(David Graeff) #106

Friday weekly progress update

I’m writing this now, as I don’t have much time tomorrow.

  • Can connect to a real openHAB instance now. If you use the hosted version in the link, your openHAB must be served via https though. (Use nginx as proxy for example on openhabian).
  • Login page, including “demo” data-set for experimenting.
  • Proper about page with open-source attributions (legally required, I think).

Things/Items page:

  • Added/Changed sorting options for items/things. They are also propagated to the local database already, but I haven’t decided yet what to do next.
  • Add item -> Although I wanted to avoid dialogs, I think it fits here
  • You can send item commands to your openHAB instance, yeah. But that’s it already for OH interactivity.
  • Shortcut for list/grid/textual mode (Alt+l, Alt+g, Alt+t in Chrome). The modifier key depends on the browser and platform. MacOS uses the command key instead for example.

Inbox:

  • Nice animation while performing a manual discovery

Maintenance:

  • Persistence page added

Service/Binding/Things:

  • Configuration pages are completely rendered from dynamic data now

Documentation:

  • I have started to document the project

Help required

Please help with …

  • proof reading all context help texts,
  • and writing the tutorial pages. We need some nice diagrams, videos, pictures and of course text to introduce newbies to the interface and openHAB in general. The text should be concise still, no complete novels :slight_smile:

If you know html+css: Please help designing the timer-task page. See Next generation design: Paper UI design study

Next

  • Item meta-tags are missing in this design study, that need to be tackled.
  • Need to think about embedded scripts in rules.

This weeks release is uploaded and live on Friday 6pm GMT+1 as usual.

Cheers, David


(Rich Koshak) #107

Well, I’m out. :wink:

I’ll have a look and play and see what we can do. Probably won’t be until this weekend or next week. Should this be captured in the README.MD file? Or should we start a separate thread and put it into the docs directly? I’m not sure what makes the most sense. @Confectrician, do you have an opinion? Where does existing (and very thin) PaperUI docs come from?

I take this to mean if I check it out and run it from the html/static directory it will connect to my running OH? Awesome!

But OH is already served on HTTPS on port 8443. Do I need to set up nginx to support a LetsEncrypt cert or something else? Or will it work on port 8443?

Well, I got it to run on my OH install following the instructions on github but it doesn’t appear to be connected to my instance of OH. Is there a setting somewhere or am I jumping the gun?


(David Graeff) #108

:smiley:

Locally it just works. I’m talking about the “hosted” instance of paper UI NG. It is served via Https from github and mixed content protection of modern browsers prevent to connect to oh, running somewhere locally on http only.


(Jerome Luckenbach) #109

The other ui duics are managed in the openhab-docs repo currently so with the current structure Thread + direct contribution would be a valid approach.

Anyway i am not sure if this is the best approach for the future.
We are maintaining all addons docs within the addons folders to get them updated parallel to pull requests.
For me there is no valid reason to handle this different after reintegrating esh.
We now have the situation that all ui bundles are maintianed in the openhab2-addons repo.


(David Graeff) #110

Which I think is wrong because of the different technology stacks and different product cycles and different teams working on the repos, but Kai likes it this way.


(Rich Koshak) #111

That is why I asked. It makes a lot of sense to me to treat these docs the same as other add-ons. I also like to have the docs and the code together so that updates to the docs can be part of the same PR and merge as the code changes.

I’ll start a new thread and make them wikis to get something on paper. Where they go will just be a matter of where it all get’s copied to. It may be a bit. This hopefully will be a one pager. I want to play with it a bit more deliberately though before I start something though.


(Jerome Luckenbach) #112

That’s a valid point, but doesn’t affect my statement.
Even if it would be maintained in a different ui repo, i would prefer keeping the docs beneath the code for a better maintainability.

+1
I think this will be the best for now. :slight_smile:


(B K) #113

I created some diagrams and slides for the openHAB Basics videos I’ve been making on my YT channel. I was planning to share the slides one way or another after I got through the last topic in the series (Rules), likely by creating separate topics for each section here on the forums. Here is an example slide, showing the overall “model” of OH2 components in SVG format.

Concepts

If this could be useful to the next gen GUI documentation, I can put that task of sharing higher on my priority list…


(Rich Koshak) #114

:smiley: I once drew something very much like this only it was a bit more comprehensive (and incorrect in a couple of places). It was also way back during OH 1.x. But it is interesting to see how different people draw the same concepts. I wish I could find it again. Lets talk about OH 2 Drawings . Scroll down and you will see a version that looks very similar to yours.

I did make this different layered view at some point too, I can’t remember why:

It is obviously missing the concept of a Bridge. I think I made it before that was commonly used.

One thing to note, if you want to make drawings for use in the docs, it was decided way back when that draw.io is the tool to use which will provide the raw editable files so others can edit it later.

I think a drawing like that could be helpful in the Concepts section of the docs. I don’t think it belongs in the PaperUI NG docs. Those should strictly “how to use” the UI and not cover the whole OH architecture. At least that’s my opinion.


PaperUI-NG / Beginner's Tutorial
(David Graeff) #115

Yes that looks like a good overview.
I have found a graphic once that I included as a placeholder on https://davidgraeff.github.io/paperui-ng/tutorial-3.html.

That is what I’m aiming for. The texts should end up in the Paper UI NG repository. Either as markdown, processed during the build, or straight as html, like I’m doing that at the moment.