Extending Blockly with new openHAB commands

[cweitkamp (Christoph Weitkamp)]

Hi @stefan.hoehn,

the NotificationAction will be registered dynamically as ActionService once you install the openHAB Cloud Connector. You might try to import all classes from org.openhab.core.model.script.engine.action.ActionService and import them. Like this:

(function(context) {
    'use strict';
    var oh3_actions = find_services("org.openhab.core.model.script.engine.action.ActionService", null) || [];
    oh3_actions.forEach(function (item, index) {
        context[item.actionClass.simpleName] = item.actionClass.static;
    });
})(this);

I am not a JS-Rule expert but I am doing something similar with Jython.

// Edit: the find_services method can be found here:

Core/automation/lib/javascript/core/osgi.js

[ysc (Yannick Schaus)]

FYI (and a bit of a spoiler alert) I have already spent a few hours on my idea/openHAB-related project of late, that is providing the means for users to extend the Blockly toolbox themselves with YAML definitions, similarly to the UI widgets (these definitions follow and same structure are stored in the JSON DB as “UI Components” in the same manner as the widgets).

Here’s an example:

This would make Blockly infinitely extensible, which regarding its appeal could be a game changer.
You could define or import blocks to tackle with domain-specific interests, import utility functions, and wouldn’t solely rely on what’s on offer in the built-in toolbox.

Because in fact the situation is pretty similar to the UI Widgets actually: no matter how many built-in widgets and options you add, you will never be “exhaustive”. There will always be another need that you didn’t anticipate and now if there’s no extensibility, the pressure is on you to deliver a solution.

So instead there is a way to define custom blocks that get added at runtime to the Blockly editor toolbox, so the users themselves are in control; they can reuse self-contained YAML definitions from others that include everything, share their own… including in the new Marketplace! Since these definitions just follow the UI Components structure, like widgets, they can be easily be offered on the marketplace as well.

The best part of it is that users could choose what to install to their toolbox based on their needs - you just browse the add-on store, see what’s available and install what you need and only that! A set of blocks to deal with GPS positions, another set to deal with date & time, or a block set to deal with specific hardware or services, you name it.

Note that not only new block types could be shared (that involves defining a way to derive JS code from declarative properties with placeholders, that part is not yet done), but also pre-assembled blocks of already-existing types - for instance check the Library > Randomize category on https://blockly-demo.appspot.com/static/demos/toolbox/index.html

In this example you can drag and drop a whole routine block, but contrary to the individual block types that you can define and generate the code of, it’s not a single block: the user in this case can import the block assembly as an example and inspiration, but alter the logic entirely.

So to be completely honest, while these screenshots are real, there is much to do (the code generation part will be the trickiest), and I’ve been figuring out things as a go to make the final spec in my head - but don’t worry it’s coming along nicely.

As a conclusion, @stefan.hoehn don’t take all this all wrong, because I support your work and believe a comprehensive built-in “system library” of blocks is very much needed and there is no conflict, neither in my intent nor in my code; my current work doesn’t really overlaps with yours, and I believe I will merge your PR before I can submit this new code; but in the end I believe Blockly in OH can have a two-tier approach - a comprenhensive, well-tested built-in set of blocks (in the openHAB category and children); and the user/runtime-defined Library.

[stefan.hoehn (Stefan Höhn)]

I guess I completely get it right and I think it is just another great idea and improvement for OpenHAB. I would also allow me to provide good samples, I guess, for all the blocks I am currently working on to have kind of a demo.

Back to my implementation:

I only now found out that there is a linting that should be run before pushing: npm run lint

This led to quite some (mostly formatting errors)

• formatting: Is there a way similar to “mvn spotless:apply” in the core to automatically apply the formatting? (edit: I fixed them now manually)

• Please have a look at the audiosink-component here . I have noticed that the following popup hadn’t been implemented (even the import file doesn’t exist). Taking it out did not have, afaik, any side effect but now lint complains. How can we solve that? What does the popup need to be implemented?

edit: for the time being I readded it, so the build runs through and can be reviewed but still we should for sure talk about that.

TIA,
Stefan

[stefan.hoehn (Stefan Höhn)]

Finally I was able to provide the first drop of the new blockly blocks and the build has run through on the Pull Request, so it is available for reviewing, further testing and documentation (@Andrew_Rowe).

The Pull Request

• has refactored and cleaned up the code
• lots of code documentation
• tooltips and help urls were added on the blocks to point to openHAB documentation pages
• introduced lots of new blocks including voice, multimedia streaming, calendar (ephemeris), span timer based execution
• the blocks have been put into openHAB subgroups

Here are the final subgroups

Items & Things

Timers and Delays

Ephemeris

Voice/Multimedia

Logging/Output

For example something like this can be easily done noew:

I hope that this be a leap in using openhab blocklies.

Stefan

PS: there are more ideas to come

• variable persistence (lets you reuse when triggering rule multiple times)
• http requests
• notifications (see discussion above)
• Execution of code (from /usr/bin/command) and scripts, though I am not sure if this is actually a good idea.

PPS: I finally found out who was the originally contributor which I based my work on even though I was’t able to get in touch with him. His name is Brian Homeyer and it is only fair to appreciate his ground work on that.

[rlkoshak (Rich Koshak)]

@stefan.hoehn

Execution of code (from /usr/bin/command) and scripts, though I am not sure if this is actually a good idea.

This is part of a bigger two part concern: Actions.

executeCommandLine is one of the core OH Actions similar to isWeekend or say. I think we should endeavor to support all the core Actions.

However, there are also binding provided Actions (e.g. publishMQTT). I don’t think there is an API end point nor any other way to get the list of those Actions though so I’m not sure how we can solve that problem.

I’ll have a go at testing this out probably tomorrow (I’ve a full day today). Thanks a bunch for picking this up and moving it forward. I’m very excited about the potential!

[stefan.hoehn (Stefan Höhn)]

Rich, I created some blocks that would generate the following code

logger.info('Starting Loop')
logger.info(itemRegistry.getItem('plannedTime').members)
var theItem_list = itemRegistry.getItem('plannedTime').members;
logger.info(theItem_list)
logger.info(theItem_list.length)
for (var theItem_index in theItem_list) {
  theItem = theItem_list[theItem_index];
  logger.info(theItem)
}

Both logs in line 2 and 4 print out the following test items of my server

plannedTime_down_rs_az (Type=StringItem, State=NULL, Label=AZ Rollladen Schließzeit, Category=time, Groups=[plannedTime]), rollershutter1_time (Type=StringItem, State=NULL, Label=Rollershutter 1 Zeit, Category=, Tags=[Blinds], Groups=[plannedTime])]

line 5 prints 2 as the length

But the for-loop doesn’t print out anything which confuses me. What am I missing?

[rlkoshak (Rich Koshak)]

The best I can guess is the for loop isn’t executing so I’d focus on that. One thing to be careful of is that theItem_list is a Java List<Item>, not a JavaScript array, I would not expect JavaScript operations like in to work with a Java List<Item>.

Now that I’m not trying to run out the door I can post an example.

I tend to stick to Java ways when working with Java Objects which is one of my controversial positions. Given that the purpose of rules is to interact with OH and OH is Java and requires Java Objects most of the time I’m not a big fan of converting stuff to JavaScript only to have to convert it back again.

A native Java way I think to do the for loop would be something like:

var forEach = Array.prototype.forEach;
forEach.call(theItem_list, function(theItem) {
  logger.info(theItem);
});

I found this on a StackOverflow posting and cannot say whether it works or not but it’s worth a try.

Using the Java Stream API it would be something like:

theItem_list.stream()
            .forEach(function(theItem) {
              logger.info(theItem);
            });

Note, the order is not always guaranteed with stream operations because they are optimized for parallel processing.

To use pure JavaScript you need to convert the Java List<Item> to a JavaScript Array which I think can be done using

var theItem_array = Java.from(theItem_list);

[stefan.hoehn (Stefan Höhn)]

“You’re simply the best, better than all the rest”

That exactly did the trick!

… works as expected. The is a proof-of-concept and I am sure you know where I am heading to:

• adding a filter
• accessing individual attributes of the item.

Which then will look like this:

Do you know by chance how I would access for example the groups attribute from plannedTime_down_rs_az

plannedTime_down_rs_az (Type=StringItem, State=NULL, Label=AZ Rollladen Schließzeit, Category=time, Groups=[plannedTime])

with respect to

var theItem_list = Java.from(itemRegistry.getItem('plannedTime').members);
for (var theItem_index in theItem_list) {
  theItem = theItem_list[theItem_index];
  logger.info(theItem)
  logger.info(theItem.getGroups)  // doesn't work
  logger.info(theItem.Groups) // doesn't work
}

[rlkoshak (Rich Koshak)]

@stefan.hoehn

theItem.getGroups

OK, here’s one of the things I don’t like about Rules DSL.

Xtend and therefore Rules DSL provides some features we call “syntatic sugar”. These are little shortcuts that let you use short hand for certain things.

In this case, let’s say have a method on an Object named getFoo() that returns a foo.

Rules DSL has to bits of syntactic sugar that can come into play.

1. When calling a method that doesn’t take arguments, Rules DSL lets you omit the parens. So MyObject.getFoo() and MyObject.getFoo are equivalent.

2. When one has a getter method (named “getX()” where X is the thing you are getting) you can call that method as if you were accessing the value directly. For example MyObject.getFoo() and MyObject.foo are the same.

JavaScript supports none of that. If it’s a method you have to call it as a method. JavaScript only allows MyObject.getFoo().

So in this case you need theItem.getGroups().

As you proceed, I suspect you’ll find more stuff like this you’ll need to negotiate. Using the openHAB JavaDocs will be invaluable. In this case, GenericItem (openHAB Core 5.1.0-SNAPSHOT API) is a great place to start to find what’s available on an Item.

[stefan.hoehn (Stefan Höhn)]

getGroups()
doesn’t work either

However, with looking now at the GenericItem (thanks for pointing to it - I actually have the core source even cloned) It shows the toString()-method is kind of hinting to the wrong method name for the getter which is actually

theItem.getGroupNames()

which finally works. So I have to map the name to the right function.

So this will be available soon as well which definitely opens many more possibilities with blockly as it allows iterations over a number of items:

Edit: I played around with it even more and it is actually pretty cool what we can do now:

Of course you could now check the State of the item and turn it on if it is off or vice versa.

Here are the items of the group firstFloor - I intentionally created the 4th one by not categorizing as a light:

LightLeft (Type=SwitchItem, State=NULL, Label=Light Left, Category=light, Tags=[Point], Groups=[firstFloor])
LightRight (Type=SwitchItem, State=NULL, Label=Light Right, Category=colorlight, Tags=[Point], Groups=[firstFloor])
MainLight (Type=SwitchItem, State=NULL, Label=Main Light, Category=light, Tags=[Point], Groups=[firstFloor])
SocketSwitch (Type=SwitchItem, State=NULL, Label=Socket Switch, Category=poweroutlet, Tags=[Point], Groups=[firstFloor])

Note how I used the “create text with” block. Only by chance I noticed that it actually has a very clever implementation that applies a .toString() method or in case of several string a .join(). This this is neat trick which allows to scan a collection whether it contains any string. Of course this could be improved here by making the comparison case-insensitive which I left out here for brevity.

1. I am wondering if we should a block that allows us to retrieve the state-description as well?

StateDescription [minimum=null, maximum=null, step=null, pattern=null, readOnly=false, channelStateOptions=[StateOption [value=ON, label=EIN], StateOption [value=OFF, label=AUS]]]

2. Do you by chance know how to access the displayState as it is not part of the GenericItem so I am not currently able to support this in the block?

[stefan.hoehn (Stefan Höhn)]

On another topic,

@ysc asked me to put all timers into a grouping object named “timers”. I implemented it as follows. Do you mind taking a look if this is the right way of implementing it and making it safe?

would result into

var scriptExecution = Java.type("org.openhab.core.model.script.actions.ScriptExecution")
var zonedDateTime = Java.type("java.time.ZonedDateTime")
var logger = Java.type('org.slf4j.LoggerFactory').getLogger('org.openhab.rule.' + ctx.ruleUID)

if(this.timers === undefined){
	 this.timers = new Object()
}
if(this.timers.MyTimer1 === undefined){
	 this.timers.MyTimer1 = null
}
if(this.timers.MyTimer2 === undefined){
	 this.timers.MyTimer2 = null
}

if (this.timers.MyTimer1 === null) {
	this.timers.MyTimer1 = scriptExecution.createTimer(zonedDateTime.now().plusSeconds(2), function(){
	  logger.error('abc')
})} else {
		this.timers.MyTimer1.reschedule(zonedDateTime.now().plusSeconds(2))
}
if (this.timers.MyTimer2 === null) {
	this.timers.MyTimer2 = scriptExecution.createTimer(zonedDateTime.now().plusSeconds(2), function(){
	  logger.error('abc')
})} else {
		this.timers.MyTimer2.reschedule(zonedDateTime.now().plusSeconds(2))
}

[rlkoshak (Rich Koshak)]

1. If you build it someone will find a use for it I’m sure. But I wouldn’t implement that as a first priority thing to do.

2. displayState is, as far as I know, only something that exists in the UI. But it’s not exactly clear to me if that is actually the case because StateDescription gets used all over the place in core beyond just UI stuff so maybe the equivalent of displayState is available some other way.

@stefan.hoehn

would result into

That approach looks reasonable and straight forward for the case where the timer name is hard coded. Is there a way to dynamically set the name of the Timer? I’m thinking of the use case where one potentially has one timer per triggering Item. In that case, the name of the Timer needs to be somehow derived from the Item’s name.

I’m a little uncomfortable about automatically rescheduling the Timer if it’s already running. That is not reflected in the blocks at all and that only covers one use case. The other use cases could be to cancel the timer instead of rescheduling it or simply to do nothing. I think Blockly needs to handle all three cases.

[stefan.hoehn (Stefan Höhn)]

1. If you build it someone will find a use for it I’m sure. But I wouldn’t implement that as a first priority thing to do.

Already fully implemented

2. displayState is, as far as I know, only something that exists in the UI

Ok, then let’s forget about it.

3. Is there a way to dynamically set the name of the Timer?

This would be something like this where you would create a timer based on the items name

However, this would result into nonsense code:

I may even have to prevent for the time being that a variable can be put into that place…
This could maybe be solved by using timers as an associative array and not creating property fields for that which makes the implementation even more complex …!?

4. Regarding the rescheduling

if (this.timers.MyTimer2 === null) {
	this.timers.MyTimer2 = scriptExecution.createTimer(zonedDateTime.now().plusSeconds(2), function(){
	  logger.error('abc')
})} else {
		this.timers.MyTimer2.reschedule(zonedDateTime.now().plusSeconds(2))
}

The idea wasn’t originally from but I think it actually makes sense:

• Let’s say you trigger the rule for the first time. Then the code makes sure the timer is created.
• Now you trigger the rule for the second time. You most like do not want the timer to be recreated but rather only rescheduled.

I am with you that this might not be directly obvious, though if we didn’t do this, I think it would become much worse in that every retriggering of the rule would create a new timer which I think would be worse, wouldn’t it?
An alternative could be that we would have to then cancel the original timer and by assigning the new timer it should then free up the reference of the old one and be garbage collected.

[rlkoshak (Rich Koshak)]

@stefan.hoehn

However, this would result into nonsense code:

Yes, I foresaw that and that’s my concern. We would need to be able to do something like “get name from item theItem” + “_Timer” or something like that. so that it evaluates.

I bet we could use the variable block perhaps to build the timer name and use the variable in the Thing’s name slot.

@stefan.hoehn

that which makes the implementation even more complex …!?

Perhaps but it also might cover more use cases. And it’s only the behind the scenes code that becomes more complex. The user facing blocks would remain relatively the same complexty.

@stefan.hoehn

Now you trigger the rule for the second time. You most like do not want the timer to be recreated but rather only rescheduled.

It’s that last sentence though. Instead you might want the timer to be cancelled instead of rescheduled. Or you might want to just ignore the event and do nothing at all. Rescheduling is only one of the possible ways one may want to handle this situation.

So break it down:

What to do	Use cases
Reschedule	When you want to delay execution until the rule stops triggering for a time: e.g. debouncing, dead man’s switch, etc.
Cancel	Do something only if a second event doesn’t follow the first event fast enough: e.g. detecting the end of double button double presses
Do nothing	Use the timer as a flag to indicate how long ago the rule last triggered: e.g. rule latching

@stefan.hoehn

I am with you that this might not be directly obvious, though if we didn’t do this, I think it would become much worse in that every retriggering of the rule would create a new timer which I think would be worse, wouldn’t it?

Not if we have the option to choose what happens. That’s what I’m arguing for. Blindly rescheduling the timer every time only covers one of the three use cases for timers. And really we do have four:

1. reschedule the timer
2. cancel the timer
3. do nothing at all
4. indeed, create another timer even if the old one is still running (I can’t think of a reason why one would want to do that but just because I can’t think of it doesn’t mean users shouldn’t have that option).

We need to support all of these options that the user can choose from for what happens if the rule triggers again and the timer already exists.

[stefan.hoehn (Stefan Höhn)]

re3 ) Regarding the storage of the timers, let me work on it by putting it into an array instead.
Maybe from that we can later find a solution for the dynamic timers.

re 4) Regarding how would you envision to block type to look like

(don’t worry about the graphical design - just a quick idea for you to think of)

[rlkoshak (Rich Koshak)]

That looks OK. Though it raises another use case. Timers are complicated.

Consider this case. I want to set a timer to do one thing after 5 seconds. However, if the rule triggers again for that Item and a timer already exists, not only do I want to cancel the original timer, I want to do something else.

In that case we’d have two “do” clauses, one that executes when the timer expires and the other one to run if the timer already exists.

And of course we have all the variations on that too. Maybe we want to reschedule the timer as well as execute the “already exists” lambda.

There might be some concepts for use here that we can borrow from my Timer Manager library. See https://github.com/rkoshak/openhab-rules-tools/tree/main/timer_mgr.

The main function is “check”. It takes up to five arguments, only two of which are required.

Argument	Optional	Behavior
key		Unique identifier for the timer. Timer Manager is designed mainly to support the case where one has one timer per Item that triggers the rule.
when		Indicates when the timer should expire. Supports all sorts of ways to define that but for our purposes it the “10 seconds” part of the blocks above.
function	X	The function to call when the timer expires. Notice this is optional. Maybe we only care about the existence of the timer and don’t need it to actually do anything.
reschedule	X	By default it doesn’t reschedule. When true the timer will be rescheduled if it already exists when check is called again.
flapping_function	X	A function to call if check is called and the timer already exists. This function is called whether or not there is a function or reschedule is set to true.

So with this one function we can support all of the following behaviors:

Call	What it does
this.tm.check(event.itemName, "5m", runme);	Call runme after five minutes. If called and the timer exists cancel the timer
this.tm.check(event.itemName, "5m", runme, true);	Call runme after five minutes. If called and the timer exists, reschedule it for five minutes
this.tm.check(event.itemName, "5m", null, false, flapping);	Create a timer for five minutes. Do nothing if the timer expires. If called again when the timer already exists call flapping. Do not reschedule
this.tm.check(event.itemName, "5m", null, true, flapping);	Create a timer for five minutes. Do nothing if the timer expires. If called again when the timer already exists, call flapping. Reschedule the timer for five minutes.
this.tm.check(event.itemName, "5m", runme, false, flapping);	Create a timer for five minutes. Call runme if the timer expires. If called again when the timer already exists, call flapping. The timer will be cancelled and runme will not be called.
this.tm.check(event.itemName, "5m", runme, true, flapping);	Create a timer for five minutes. Call runme if the timer expires. If called again when the timer already exists, call flapping. Reschedule the timer for five minutes.

Th behavior can be quite complex but the interface and set of options is pretty simple. I don’t know if this offers any inspiration or not. Maybe just food for thought.

[stefan.hoehn (Stefan Höhn)]

My feeling is that this is getting too complex and I wouldn’t want to include external libraries. Especially if it is going into the direction of having to many arguments in the blocks. Then people won’t be using it as becomes too hard to understand. Blocks are different from APIs. They should be straight forward to be used from my perspective. Maybe we cannot cover all circumstances but at least most of them.

Keep also in mind that you cannot use parameterized functions like in “normal” languages. Basically if you use the timer with the retrigger = none parameter you may be able to do most of the things by using the individual blocks (there are blocks for any individual method). If used, they can simplify most of the stuff.

@Andrew_Rowe We should make sure that the different use cases are well documented with examples when to use what.

So for the time being I would now try to implement the associative array persistence as well as the retrigger parameter with the four options.

@ysc please jump in now (and not only when I have finally implemented) ok, so I can save finally get this stuff done for the first drop of functionality.

thanks for all your ideas and thinkings as these block should really make a difference in the future and therefore should be well-designed.

Stefan

[rlkoshak (Rich Koshak)]

@stefan.hoehn

My feeling is that this is getting too complex and I wouldn’t want to include external libraries.

I didn’t post the above to for us to use that library. I posted it as a way to think about all the different ways a timer can be used in rules in the hope that it might help us come up with a Blockly interface that is relatively simple yet supports all the use cases.

Almost all the possible use cases for timer use is covered with one function with up to five arguments in my library. I’m not saying we need to have one block with five arguments in Blockly, but we need to have a way to cover all of those use cases somehow. The library is just an example showing that it is possible.

I really don’t see how only supporting some of the timer use cases will ever be satisfactory and eventually all of them will need to be added. Better to think about that now and design a good interface rather than let the arguments and features accrete over time in a hap-hazard way. If we can come up with a way to support all of those cases under “what it does” I’ll be satisfied. It doesn’t have to be all handled by the timer block itself too, though the more we can put into that one block the easier it will be for users I think.

[stefan.hoehn (Stefan Höhn)]

Don’t get me wrong, Rich. I definitely appreciate all of your thoughts but currently it seems 1) it is only us two thinking about it and secondly I have the problem that I finally need to implement that and fear that I am running out of time (and maybe the skill) getting all of it done in the first shot. I have already refactored quite a lot of code and I even refactored quite a lot because Yannick had some more requests and I am happy to do that but it is getting more and more complex to even generate the code behind it.

Let’s shot for a first version but I am sure we can add more later on.

[Andrew_Rowe]

@stefan.hoehn

We should make sure that the different use cases are well documented with examples when to use what.

That is a great idea. I will try to make the limitations clear in the documentation