When is the right time to write a binding?

[dotorg (G H)]

I’m new to OpenHAB. I’ve been investigating using it to replace the Homeseer system I’ve got at my house, mostly because HS3 is such a cobbled together mess of badly-designed VB code, with a worse UI and even worse mobile apps.

I’ve got a half dozen custom plugins I’ve written for HS3, either for 3rd party hardware there weren’t plugins for, or custom hardware for various uses around the house.

So far I had pretty good luck in converting one of them to hand-crafted rules, items and a sitemap. That one was an interface to the Monoprice six-zone whole-house audio amplifier. I found an example someone had for it on here, and while it was a bit of a learning curve in Xtend, I was able to refactor the example to run more efficiently and reliably. What I found, though, was that there was a really shocking amount of manual code needed to make it work. On the “receive” side of things, where the ASCII data from the receiver (in this case, over TCP) has the context of what its talking about, I was able to use groups, the group member list and a structured naming for the items to find where to stuff current state for the amplifier. (Its six zones, and about a dozen items per, so it was a lot of items to create, but was manageable, if oddly inefficient.) On the “send” side of things, though, it ended up being almost a thousand lines of rules for something that should’ve been maybe five or six. Every one of those items needed its own rule to handle state change, because the Xtend code for the rule doesn’t seem to be given the actual item that generated the event. (Its literally the only event-driven system I’ve ever seen that doesn’t provide it!?) So it was a lot of cut-n-paste-n-change, but it works. I think its at the tattered edge of what is “sane” to maintain by hand, though.

In looking at how bindings work, one big benefit is the fact that they have channels, and items can route commands back to them. So even if you have to manually create your items (yuck), at least you don’t need thousands of lines of repetitive rules for them.

One of the plugins I need to replace is a good bit more complicated. It provides a linkage to a combination IR blaster and environmental sensor suite. IR interactivity (because its time-sensitive) happens over TCP sockets. Environmental data, general device metadata and things like that are via a REST API. They’re published onto the network via SSDP/uPNP, so autodiscovery would work fine to find them. In theory, the IR interface could work exactly the same as the Monoprice stuff. The underlying protocol is pretty similar. (Actually, it’d probably work as-is with the GlobalCache binding.) These would be two pretty simple items per device on the network. Losing auto-discovery wouldn’t be ideal, but it’d be easy to manage.

The second piece the existing plugin provides is state management and both encoding and decoding of IR messages to and from a bunch of HVAC units. This starts to get a lot more complicated to do as rules – it’d be quite a few items per unit, all that need to respond and generate encoded IR from the other i/o items, as well as polled data from the HTTP endpoints, per unit. Even with liberal use of lambdas, I think I’d end up with quite a few thousand lines of rules to manage it all, even though the actual logic for what to do with almost everything is identical. Managing it long-term would be a nightmare.

It seems, from reading the forum and the docs, that there’s a pretty hardcore “bindings are for physical interfaces” mindset with OpenHAB and ESH, but the fact that you can’t (unless I’ve totally missed it) define, and implement, the thing->channel->item pattern with text-file based config and .rules files means for anything but the simplest of integrations, a custom Binding is essentially a requirement. (I’m guessing that’s why 75% of the OH2 bindings are simple TCP interfaces, and aren’t written as rules!)

Is there some detail I’m missing here? I don’t want to write a full binding if I don’t need to, particularly given its not the simplest of things to properly write a thread-safe, properly reliable bidirectional TCP client in Java. But I’m just not seeing a practical way to do this without a massive amount of manual work without one. (I’d probably write an abstract TCP binding, or at least generic connection, that I can extend and reuse with other devices. Half the OH2 plugins have the same thing implemented over, and over, and over again.)

[rlkoshak (Rich Koshak)]

@dotorg

That one was an interface to the Monoprice six-zone whole-house audio amplifier.

I bet the community would love to see what you’ve done.

@dotorg

What I found, though, was that there was a really shocking amount of manual code needed to make it work.

That is usually a sign that:

• one is trying to code in the Rules DSL as if it were a standard programming language with faked-up data structures in HashMaps and Lists

• one isn’t using Groups, lambdas, or other approaches to make more general purpose Rules

• it is a problem simply not suited for Rules DSL and it might be better to write in a JSR223 language like Jython or implemented as a binding

If you post your code I could take a look and see if I have any suggestions.

@dotorg

I was able to use groups, the group member list and a structured naming for the items to find where to stuff current state for the amplifier.

You’ve been reading the Desing Pattern postings.

@dotorg

so it was a lot of items to create, but was manageable, if oddly inefficient.

That’s OH in a nutshell really. No matter your approach, you will have more Items than you think you should because an Item represents only one piece of information or one way to activate something. You may end up with a dozen Items to represent a single device if there is a lot of sensors or actuators on it. But Items are the coin of the realm: everything in OH works with Items.

@dotorg

Every one of those items needed its own rule to handle state change, because the Xtend code for the rule doesn’t seem to be given the actual item that generated the event.

Depending on the timing of the events (i.e. they don’t come in too fast) there is a hack that one can determine the triggering Item using lastUpdate from persistence. If you use JSR223 rules you can get the triggering Item. I think the new Experimental Rules Engine also provides the triggering Item.

Beyond that, you should be able to use a lambda so at least you don’t have duplicated code in each of those Rules. That should help some I would think, not having seen your rules.

@dotorg

In looking at how bindings work, one big benefit is the fact that they have channels, and items can route commands back to them.

While correct, I want to make the concept of Things and Channels a little more clear. A Thing represents a Device. A Device may have zero or more sensors or actuators. Each sensor or actuator is represented by a Channel. A Channel gets linked to an Item. Everything else in OH sees the Item.

So it isn’t so much that the Item routes changes to a Channel as the Item actually represents that Channel to the rest of your OH system (Rules, Persistence, Sitemap, et al).

Not seeing your code but based on your descriptions I bet you could reduce that thousand lines of code down to 200 or so, but given the communucation is over TCP, a binding might be a better approach regardless. And then hopefully you can then contribute it and the rest of the community can use Monoprice six zone audio amplifiers as well. If not a binding, I’m certain you would be more comfortable writing in a more typical language like Jython, JavaScript, or Groovy so the JSR223 add-on should be seriously considered.

Obviously, I think you should go the binding route and contribute it to the project (or host it yourself and distribute it through the IoT Marketplace) since the rest of the community would benefit.

@dotorg

So even if you have to manually create your items (yuck)

In simple mode OH will automatically create an Item for each Channel on each Thing. However, most users quickly abandon that as they want to be able to provide a bit more customization to their Items and/or prefer to use more human -friendly names (e.g. LivingroomLamp verses zwave_device_dongle_node2_switch_binary).

@dotorg

One of the plugins I need to replace is a good bit more complicated.

That too sounds like a good candidate for a binding.

@dotorg

encoding and decoding of IR messages to and from a bunch of HVAC units.

Are you aware of the liric binding?

Also probably a good candidate for a binding.

@dotorg

It seems, from reading the forum and the docs, that there’s a pretty hardcore “bindings are for physical interfaces” mindset

I’m not sure why you get this impression. Some of the most used bindings interact with REST APIs or are purely software including off the top of my head:

• Astro - pure software
• Network - pure software
• Weather, Yahoo Weather, Wunderground - interacts with various weather service REST APIs to pull down weather info
• MQTT - messaging
• Mindcraft - pure software

and there are dozens more.

And even so, what you are proposing are bindings for devices so I don’t know the problem.

@dotorg

(I’m guessing that’s why 75% of the OH2 bindings are simple TCP interfaces, and aren’t written as rules!)

The reason why is that the Rules are intended to orchestrate all your actuators based on the values from your sensors. Rules are not intended to be used to interface with new devices, APIs, or Technologies. There are a lot of bindings and other features that let you do that from Rules but that really isn’t what Rules are intended to provide. This is one of the reasons why it fights against you when you try to code in it as if it were Python or Java or some other more generic language. You have a fallback set of tools you can use from Rules to interact with a complex API or device but ideally a binding would be created for all but the simplest of these interactions.

Often there is a transition from Rules to a Binding. This is what happened with the iCloud integration and hopefully is happend with Roku integration. They were fully implemented in Rules first and then transitioned to a binding.

@dotorg

but the fact that you can’t (unless I’ve totally missed it) define, and implement, the thing->channel->item pattern with text-file based config and .rules files

This is correct. You need a Binding to have the Things and Channels. Things and Channels are the interface between Items and the Binding. They don’t provide anything for you in and of themselves.

@dotorg

(I’d probably write an abstract TCP binding, or at least generic connection, that I can extend and reuse with other devices. Half the OH2 plugins have the same thing implemented over, and over, and over again.)

I think there are already providers for this sort of thing implemented in ESH. You will have to check but over the past six months or so a lot of generic proviers for things like this have been implemented and released in the SNAPSHOT. I know this is the case for HTTP and Location, if I haven’t misunderstood what I’ve read on the forum.

But, given that almost every binding is written by completely different developers and written at different times this is not surprising.

So the tl;dr is I think all three of your use cases would make good bindings. If you do create bindings, I hope you will consider contributing them to the community.

[dotorg (G H)]

Thanks for all the feedback and clarifications. They’re all super useful.

Interestingly, I had gone to look at adding the item that received the command or event into the xtext context, and discovered someone added it to ESH three days ago. That’ll make a huge difference in rules complexity. While I think there’s some benefit to a domain-specific language like the OpenHAB rules, I think there’s a risk of getting too “there’s design patterns for using this data to do what you need”. The existing patterns, both in procedural and functional languages have evolved over decades because they work.

The big issue I’ve got with rules in OpenHAB right now is the fact that they’re running in an intrinsically asynchronous environment, and those design patterns people are using (like looking at the last updated item, etc) are impossible to use in a thread-safe manner. You’re stuck either accepting unreliability in rules, or having to do hacky things like slow down execution to get updates to happen reliably.

Event-driven interfaces always include the source of the event for that very reason… You end up having to have massive code duplication, or have race conditions in asynchronous environments otherwise. Locking constructs help with shared resources, as long as you have a single point of interaction with it.

With the addition of the triggeringItem, rules become vastly more powerful. The other thing I was thinking of adding to the core was an implicit “allItems” group, so you can arbitrarily look up items without having to manually add a group to them. All ESH would need at that point is the concept of parent items, and you’d be able to do some pretty flexible containerization of rules for reuse, and make it a lot easier to implement multiple instances of a given set of behaviors. (A second set of implicit groups would really help with that… if you took names like "Module_{Instance}Item" and split on the underscore and created groups automatically for Module and each Module{Instance}, you could write code like Module_Instance1.members.findFirst(item | item.name.endsWith(“Connected”) and generically handle things. (Although you can manually do it by hand, of course).

I intend to clean up the Monoprice code and post it back, as its probably pretty useful right now, if at a minimum as an example of how to interact with remote hardware.

For the other stuff, I may eventually make the bindings for them available, but because they’re custom hardware and custom firmware, its a lot of time on my end to not only write and sufficiently document the bindings, but also create clear schematics and board layouts for the hardware, specify parts lists, host the firmware source and the STL files for the 3D printed enclosures. Its something I’d love to find time to do, but there’s never enough of it! Either way, they’d be too narrow-focused to warrant inclusion in the main OH repository, IMO.

Its clear that bindings are the way to go for these devices, though. While the addition of triggeringItem would help with rules, it won’t really make that an appropriate path for implementing them.

Thanks again!

[gitMiguel (Miika Jukka)]

The right time is now

I had my binding developement waiting for months just because the idea that Thing has to present a device. But then i thought that this is valid only for official release. I can have anything I want in my system. And eventually publishing it to the community I hope I gave something back. And as a little surprise to me bunch of people found it usefull.

[dotorg (G H)]

I’m workin’ on them.

This IR-controller one is probably not super useful broadly-speaking given there’s a lot of moving parts with the hardware and the firmware is a festering pile of crap I wouldn’t want to subject any other human beings into having to look at, but I’ve got other custom hardware combinations that may make sense to get together sooner. (Like a wifi-enabled controller for NeoPixel/WS2812b LED lighting strips.)

Just need to to get over the hump in the learning curve on ESH. The “how do do what needs to be done once I know I need to do it” part of the plugin is easy, The “how to interface with ESH” part is somewhat less so. Still, having suffered through writing plugins for Homeseer, its a breath of fresh air.

[rlkoshak (Rich Koshak)]

@dotorg

Interestingly, I had gone to look at adding the item that received the command or event into the xtext context, and discovered someone added it to ESH three days ago.

That has been a much requested outstanding issue since at least OH 1.7. I’m glad to see someone is taking a crack at it.

@dotorg

The existing patterns, both in procedural and functional languages have evolved over decades because they work.

But the existing languages have inherently more complexity and essentially require one to be a programmer to be successful. The goal of OH and the original Rule’s DSL and the new Experimental Rules Engine is to provide an environment where users do not have to be programmers ot be successful.

@dotorg

The big issue I’ve got with rules in OpenHAB right now is the fact that they’re running in an intrinsically asynchronous environment, and those design patterns people are using (like looking at the last updated item, etc) are impossible to use in a thread-safe manner. You’re stuck either accepting unreliability in rules, or having to do hacky things like slow down execution to get updates to happen reliably.

All true statements, yet for many if not most home automation use cases it works. No one is recommending OH to be used for industrial automation, though some do I’m sorry to say. As with the NoSQL, relazing one requirement (consistency in that case) one is able to gain in other areas (redundancy, load balancing, etc).

In this case, since Rules are really designed to orchestrate between Items and are not really intended to be used to implement a bidirectional communications protocol over two separate transports, they are adequate most of the time. Obviously they will be much better when we can get the triggering Item. I’m hesistant to jump on board with the synchronization issues though because that is a big can of worms.

@dotorg

With the addition of the triggeringItem, rules become vastly more powerful. The other thing I was thinking of adding to the core was an implicit “allItems” group

Before going too far down this path make sure you are aware of what is being done with the Experimental Rules Engine. That will eventually replace the Rules Engine as the default way to implement rules. It makes no sense to introduce a lot of these sorts of changes if they don’t make it into the new engine, where applicable.