Next generation design: A Paper UI replacement proposal

(David Graeff) #121

It sounds like you want to stop people from contributing good ideas and waiting for the AC, although I think that all texts, images, tutorials made for this design study are re-usable for every other future prototype. I mean that’s the entire purpose of this design study, I guess.


(Rich Koshak) #122

Not at all, but lots of users are responding with the apparent assumption that what is presented here is what will happen. I just want to inject a little context. It’s a proposal of what could be. Not a fait accompli. Some or all of these ideas will likely become incorporated into the next version of PaperUI. But there are other ideas out there too and the process for how the ideas will be adopted is not yet defined.

What is presented here is great and I did say I don’t want to stop the work, but there are certain aspects like config file formats, Rules builder approach, etc that have a huge impact on the future direction of OH and needs to be adjudicated by the AC. There are other developers who have done some work in this area that are not part of the discussion as well (e.g. Yannick has already built a prototype Rule’s editor for the NGRE).

Perhaps unless someone comes up with something radically different that the AC agrees to. I’m aware that there is another prototype that exists that is not being discussed right now. As a for instance, I’m already looking to have to rewrite about 30% of my existing NGRE docs to meet this GUI. I’m looking at about the same amount of work to meed Yannick’s Rule builder too.

And starting to build the docs at this point generates the impression that what is presented is what will happen. And we don’t know that yet.

Finally, Kai did ask us to hold off on some of these discussions until the AC is a little more defined so we don’t rile up the users like happened on the config file threads. My comment above is in part to respect that request as well.

(Hakan Tandogan) #123

At long last… This was really necessary.

(David Graeff) #124

Wrong thread ^^

(Hakan Tandogan) #125

True ^-^

(Udo Hartmann) #126

I tried the design study, but it failed on firefox (65.0, 64bit) Finally got a glimpse of the future (very promising!) when changing to Chrome :slight_smile:

(David Graeff) #127

Oha, thanks. I usually check Firefox as well, but I missed a recent change.

I’m currently thinking about the rule editing. It’s not an easy task.

Editing is already possible, as seen in the picture, but how to present the three categories best?

Maybe having a three column design, or a tinted background behind triggers/conditions/actions respectively.
Or something like a frame, like here:

(Rich Koshak) #128

There is a lot I don’t like about IFTTT, but their recipe builder is kind of nice. Break it up into a sort of wizard flow maybe for the initial creation.

Yannick’s prototype uses a top to bottom arrangement with color coding and a more “yahoo pipes/Node Red” arrangement of elements. So all the triggers go at the top. Then all the conditions, then all the actions at the bottom. Each section is denoted by a different color and sometimes shape.

Speaking of NodeRed, they have a little more free form (I don’t know if Yannick’s is presented top to bottom of constrained to top to bottom).

In both cases I don’t know if I like the need to double click to drill down into each node for the properties. I like the way the parameters are presented right at the top level here, though users will quickly run out of space if they have more than two of any one type.

I don’t know if it needs a frame if you have some other way to show like types. Rules pretty much flow on one direction so position on the graph could do it. Color can do it. Shape can do it. Any combination of the three could do it.

One thing that would be nice though is to have a special shape or color for the “run another rule” action. In that case clicking/double clicking could take you to that other rule. Double clicking a Script Action or Script Condition should open the code editor.

The lines are visually instructive but I’m not sure they are strictly needed. The triggers are going to pass to the conditions and finally the actions. Unless there are constructs in the NGRE JSON Rules I don’t know about that’s all that is allowed.

I’ve spent some time with the current PaperUI UI for this so I don’t really know the full of what’s possible with these Rules.

(David Graeff) #129

There is this one construct that neither Paper UI nor Yannicks really good Flow Editor handle as far as I know. And that is Inputs/Outputs. That’s what you see in the above picture. I have modeled Outputs to be on the right side and Inputs on the left side of a “component”.

Those connections already bring in some “flow lines”, that’s why I struggle with the idea to add more visual connecting lines like Yannick does.

I’m also more left to right oriented in my first attempt. Top to bottom is another option.

Yannick is cheating a bit though. He is “compiling” the flow into real rules in a second step (because rules can’t handle if/else branches). I’d say the core should either support if/else for conditions or we need to limit ourselves in what is possible. But better not pretend that rules (without embedded scripts) can do something that they actually can’t.

I have closely looked at the input/output concept of NGRE and it is indeed possible that a Trigger connects to a condition and at the same time to an action (bypassing the condition). Conditions are also kind of on their own, because they are executed anyway within a rule, if they are connected to other rule components or not.

Definitely. It’s coming for the design study. Mainly to find out if a code editor dialog on top of a (complex) rule graph is doable. Paper UI does the same, and I’m not fully convinced, but we’ll find out.

(Rich Koshak) #130

What are the inputs and outputs in this case though?

For a trigger the input is “off screen” as it were, some event. The output is “rule triggered!”

The input for the condition is always “rule trigger x triggered!”. The output of a Condition is either to forward the “rule trigger x triggered!” or nothing depending on how the conditional evaluated.

The input for the Actions are “rule trigger x triggered!” and the output is, like the event, “off screen”.

Given this, is there something I don’t know or understand or does the fact that there are input/outputs on the nodes not matter that much in this context?

As long as it is consistent I don’t really have a preference. I’m sure it’s one of those areas where everyone will have their own preference.

I agree. I can’t speak for him but I suspect that was one of limitation with the JSON Rules that caused him to abandon the prototype until NGRE matured a bit. Because indeed if/else does not appear to be supported and we shouldn’t fake it.

But even with what is offered now is so much more than Rules DSL users are used to it’s going to be really powerful I think.

OK, good to know. So the lines are needed. I was starting to wonder if lines were needed at all. But there still really isn’t an output beyond executing the Action. Though if one implements a Condition Script, there is no end to the side effects they could cause from that script. :frowning:

I can already see the support posts on that one. :cry:

(David Graeff) #131

Inputs and Outputs are more like this:

An MQTT topic received trigger can have configuration (for example the topic to watch. But the user can use a wildcard topic here).

One possible output is the actual topic and another output is the received message. You can then connect those to for example the comparison condition or to a text-to-speach action.

Not many components support inputs/outputs though, because Paper UI never visualized them. But it is code wise really easy to support configuration as well as inputs.

(Yannick Schaus) #132

Not really :slight_smile: The main thing that led me to wait a bit was that NGRE rules weren’t running anymore after an openHAB restart (they were left in the “uninitialized” state because of some bug, which I believe has been fixed since). I really believed the script actions and conditions could replace the DSL in most cases - you can even import libraries!

One of the purposes of the Flows Builder was to actually test an idea - re-using the NGRE as-is and especially its “run another rule” module to build more complex flows à la Node-RED, which could include “else” branches.
Using prefixes and tags the resulting rules are hidden in Paper UI so they wouldn’t be modified or managed (or even run) anywhere except in Flows Builder.
Basically a " when T then do A, then (if X then do B else do C)" flow is implemented using 3 rules (IIRC :joy:):

  • Rule 1: when T (trigger), do A (action) then run Rule 2 and Rule 3 checking conditions
  • Rule 2: do B (action) but only if X (condition)
  • Rule 3: do C (action) but only if NOT(X) (condition) - there is no way to negate a condition in the core so the client would calculate it for known modules.

So no need to alter the “IFTTT-like” model of the NGRE, just chain simple IFTTT rules together to achieve something more complex.

I actually added support for outputs in the code editor autocompletion for scripts.
Given a N1 node which is a trigger like (N1 is also the UID of the trigger module):

you get the outputs (oldState, newState) in a subsequent script editor:


or even the outputs from another node using the ctx variable - even those from other rules since the “run another rule” action preserves the context:


Input/Outputs are hard to grasp at first, they are not documented at all…

(Rich Koshak) #133

That’s why I hesitated to speak. It’s good to understand what the lead to the dead end.

I agree the ability to import libraries and create Rule Templates will be huge!

OK, so this is a feature I’m not aware of. That does make sense for the use cases you list. But does the concept make sense for other more common use cases like “send command to a Item” or Script Action? Unless I’m wrong, and I clearly often am wrong, there is no way to get the contents of that Input inside the Script Action, for example. I worry this might cause confusion. I’ll need to think about it some.

What I’m thinking about is something like this. Let’s say that you have an MQTT Trigger, no Conditional, and a simple sendCommand Action. What’s the correct output to select? Does it matter?

(Yannick Schaus) #134

I could be wrong, but I believe modules have to be designed with input matching other modules’ outputs so the wiring is done automatically. I think it was used in commercial products based on ESH, but I haven’t encountered that in practice in openHAB. Script modules have lower-level access to the context so they’re able to see all outputs of other modules which have run before.
Those concepts were explained in detail during the Smart Home Day last year, so it’s probably time to watch the video of that talk again :wink:

(Rich Koshak) #135

That’s the part I couldn’t figure out. Which object in the context gives me that information? It doesn’t appear to be in event. It doesn’t make sense for it to be in the ir. I never could get anything useful out of rules.

I did watch the video but I had a hard time associating what he talked about to what I was seeing in PaperUI or the JSONDB files I was producing for inputs and outputs. I thought that some of those were specific to what Bosh was doing and not available to openHAB. And in the video he pretty much just says there are inputs and outputs, not that next layer down that explains what they are or what they might mean.

(David Graeff) #136

Exactly. Inputs and Outputs have a type each. A type is just a string, the convention is to use a java class name (like “org.eclipse.smarthome.core.types.event” or “java.string”). Only compatible types can connect to each other. Inheritance is not considered (so a “State” can’t connect to a “Command” although that would work with openHAB).

So there is still some work needed in openHAB core to really make inputs/outputs shine. But I imagine a rule where you just draw a connection from “MQTT topic received” to “TTS speak this”.

(Yannick Schaus) #137

Most event-based triggers will have event as an output, which is an object with several methods.
I tried that in a script:


here’s an output:

{N3.oldState=4, N3.newState=3, ruleUID=dumpctx:N3, N3.event=Scene_General changed from 4 to 3}
Scene_General changed from 4 to 3                                                                       

(David Graeff) #138

This is the UI that I’ve come up with for Item Meta-Data.

  1. You enter the namespace and select an item.
  2. You modify the meta data in json (I’m not feeling like adding a generic UI for super complex nested json objects)
  3. You click “assign” and after the syntax check passed the data is submitted.

I guess that is better than nothing. I’m not sure if the existing Meta namespaces can be queried from the storage (there is no REST interface yet).


(Rich Koshak) #139

Yep, I know about that one. So that actually is the input to the Script Action. That makes some sense. I guess I was under the impression that event would be the only input. So what we are saying is there can be other types of Inputs and there will be some other object in the context that gives us access to that information?

One reason why I’m really focused on this is because I’m spending a lot of brain cycles right now figuring out what a library of Rules would look like. For example, how would I have to write the Time of Day Design Pattern as a library of Rule Template to minimize the amount of work the users would need to do to use it.

If I can’t count on event, it makes that quite a bit more complicated. I’m already a bit daunted by the implicit variables that depend on the type of trigger. I love that we can get the Thing ID when the Item is triggered by a Channel Trigger. But I get nothing when it’s time triggered or system event triggered. So the top of all the library Rules will be a bunch of if(receivedCommand) exists use that else if(currState) exitst use that else…

It really ties the Script Action to the trigger type in a way that I was hoping we could get away from. Are we saying that we can’t get away from that?

(David Graeff) #140

I know this is no solution for OH 2.4 users. But for OH 2.5 we can aim to just add necessary information to the automation engine. As I said, nobody really has thought about the Inputs/Outputs and they are mostly not documented.

If you say what you expect as inputs, someone, maybe me (depending on available time), can just add that piece of info.