Flexibility of framework supplied types

Development
,
1

Last year I made a small experiment with resulted in a post on with same title on Eclipse SmartHome forums. Since we’re now, openhab-core took place of ESH framework part, and OH3 is around the corner, I would like to bring the issue back to you and revisit it.

Making long story short - over Christmas 2018 I did an exercise with a non-core Type necessary for one of “my” bindings. Because BACnet is quite extensive on the type and command layer and some structures can not be mapped to openHAB I wanted to see if I will be able to supply a List or Calendar object which exists there. It is currently not possible due to reason listed below.
On the command level - BACnet supports prioritization of commands meaning that beside ON or OFF value there is an additional piece of information (1..16). Obviously I could try to create 15 item types with additional property on the each and every commandable channel for every BACnet point but as you can imagine, this would result in a mess of unbelievable scale. This also falls close to a toggle command and other binding-specific commands reached in Profile types topic which eventually reached also to the command kinds.

I narrowed the issue to one element which currently locks extensibility of that part of framework and it is the TypeParser.
Currently org.openhab.core.types.TypeParser is a static helper class with documentation stating that “This is a helper class that helps parsing a string into an openHAB type”. I traced usage of it and currently there are ~60 calls of it, mainly from openhab-core and several bindings (modbus, mqtt, oceanic, smartmeter, neeo, openhabcloud, rules, persistence, rest).
It is not a huge amount of calls to be fair and it is manageable to clarify these. It is also possible to at least provide non-invasive way for provisioning of new types by keeping signature of TypeParser but delegating actual parsing logic to OSGi service layer.

The big question is, what is the @architecture-council look on that. There is limited use of new types, yet currently framework part is locked down. As per Kai answer in origin thread:

When the type system had been introduced 9 years ago, its originally idea was to be extensible.
But due to many different reasons, this never worked out and the type system since then became a static part of the core framework and is not meant to be extended.

Sadly I need this part in order to get proper support for BACnet. I failed to get it onto openHAB 2.x a while ago, the 1.x binding is stuck on basic elements (binary, analog, multistate inputs and outputs) while I been asked several times about support for complex objects.
I know that many things rely on existing types and I acknowledge that a new third party type won’t be handled equally as a core one. At the same time I don’t want to open a Pandora box where every binding brings its own types.

I appreciate your feedback and input to further discussion.

2 Likes
2

Is it possible to extend an existing type with tags and/or metadata to achieve some of what you are missing? That send to me, as a mere user, one way to handle the scenario you describe. I don’t code in core so I cannot comment on the rest.

3

This is a problem I also encounter with both ZigBee and ZWave bindings and I fully agree that this is something that should be addressed as we move toward OH3.

My proposal was to supplement the primary Item type with additional “attributes” to allow rules etc to have access to this data. This allows (as an example) to switch a light On or Off, but say “I want this to transition over a 5 second period” - or whatever. Currently OH is very limited, and you either end up with multiple items (as you suggest) or having to use String items and encode them with json (which is not nice, but is what I was recommended, so it’s used a lot for alarms in ZWave where multiple bits of information is available).

I made a proposal in the core repo for this, and I think a discussion on how to manage this is needed for OH3.

1 Like
4

It is possible to do some workarounds, but these are still remaining workarounds. Metadata is handy to place additional information, but it still remains fairly static once it is made. I believe metadata is very good for making better UI/tool integration, but ultimately it doesn’t fly back to underlying bindings which is used to feed the item. If you take a look on where metadata is currently used - it is in Alexa, Homekit and UIs, but not in the bindings.
Tags, groups, categories are helpful when we rely on rules, yet again they seem to be static and more useful when it comes to automation, but not pushing data back to binding.

The necessity for additional types comes also from more advanced controllers who are actually able to provided complex structures. I will be able to receive them on the binding side but I will have no chances getting them back to UI as some of them are dynamic, see below.

We can always turn something into String or Map, but I keep asking myself how reliable such solution will be? I believe it solves the problem in some way, but not entirely clear. Our problematic when we take into consideration a BACnet schedule. This is a complex object which can be mapped to a separate Thing, yet some of its properties are are variable size Array() or an overlay type which turns to be an Array under the hood.
So prioritization is one thing the Schedule as a Type or variable length Type accepting something like a tupler/pair (time + command), is a second one.
Turning complex Schedule/Calendar objects into String will result in fairly long values which are difficult to manage. There is a nice example of how to do it with Habpanel made by @nepotu.

Making some attempt to clarify these parts and bring 3rd party type might be beneficiary for multiple cases as it will lead to further unification between UIs and bindings.

5

I don’t think I said it needs to be a string - it should be an object (a Map<String, Object> so that you can add attributes of any type). The example I provided here was a simple example only, and also does not use a string in any case.

ZWave and ZigBee have similar concepts that I am also interested in accomodating. Currently Kai has said that these should be mapped to a json string, and this is what ZWave does (not for schedules, but for other complex information) and this is not very nice (IMHO of course).

Again, I do not propose this at all. A json String is what we currently have and this is what I want to avoid - I proposed that we can add attributes - these attributes may be a simple String, or Integer, or they can be any other Object type.

1 Like
6

It’s also being used pretty heavily in Scripted Automation. More of the community library submissions rely on metadata than not. And with scripted automation with the helper libraries it’s really easy to access and change the metadata. Since OH 3 will become the default in OH 3 I don’t think it would be out of bounds to rely on it if it solves a problem for a binding. I know it’s not used much in bindings but it’s not clear to me if that’s because of technical reasons or because it’s not the way it’s been done in the past. Bindings do have access to metadata in some ways as that’s how Alexa and Google Assistant (coming soon) works, and they are add-ons.

Don’t get me wrong, I’m not arguing against new types, though I am a little wary about a proliferation of new types. People already get confused by the difference between Contact and Switch. I recognize that the current set of types may be insufficient but I’m concerned about a proliferation of types that are only slightly different. Nuance is difficult for far too many users.

I’m not a developer so I don’t really have any real insight, but I thought I’d raise it as an option in case it was useful.

7

IMO Contact should be removed.

1 Like
8

I agree. The current types are a bit of a mix of specific and abstract and contact does tend to cause confusion.

I don’t want to change the topic, but I think it’s related so I’ll add it anyway :wink: …

I think that for OH3 we should have a bit of a rethink about types in general - the issues raised here by @splatch, and myself here and on GitHub are part of the wider issue (IMHO) - we also have no concept of groups that bindings have visibility of, and this severely limits how many bindings can work (it means that we cannot take advantage of broadcast and multicast techniques since the binding receives commands for each member of the group, and not the group itself).

I would advocate a rethink for OH3 - earlier rather than later so that it can be properly thought through and planned, and binding developers can also have visibility of what will be implemented so they can do their own planning. I think that we should be able to come up with something that does not significantly alter the current system, but enhances the current implementation to support technologies and systems that need it. I think we all appreciate that a “free for all” would be a mess, but I think it’s also clear that the current system falls woefully short of meeting the needs of modern systems.

We all want OH3 to grow, and I believe that it should improve on OH1 and OH2 and not just be OH2 with a new UI. It needs to meet the needs of the future (or at least the present!), while building on the past - and not just be stuck in the past.

@rlkoshak thanks for engaging here. I’m not sure who is on the “architecture council” but I really think that we need this engagement. We need to consider the needs, and we need to have a plan for OH3 that contains enough detail, and is publicly visible so that people can see how great it will be, and developers can start to plan themselves for the future.

(sorry - I’ll get off my soapbox now :sunglasses: ).

9

I will start from last point - both Alexa, Google Assistant are javascript integrations which do not even function as an binding even if they could. The Homekit is the only one integration working within OH2 bounds and it falls into IO category, again it is not a binding. Homekit integration is twisting items again into another form of Things required by Apple ecosystem. It does not bring additional commands, it does not have direct contact to devices.

Getting back to the point. You can do a lot of different things with scripting, however we need to clearly distinguish where scripting is used as workaround for framework shortages, where it is useful or where it is failing into law of the instrument.
If I noticed properly @Kai mentioned that openHAB have around 250 bindings available. Obviously we can’t make OH core polluted by one and we need to carefully look for reoccurring patterns, see if there are commonalities and prepare valid abstractions or ways to handle them in the framework.

I believe that one of such commonalities is a Schedule representation which can be passed between UI and binding. Obviously making it present in framework is a lot of work yet lack of it brings more troubles than benefits and make life of our users much harder.
When I install for example ebus integration for my heating system binding is able to fetch daily schedule, but these are separate channels. More over these channels most likely end up as Strings (CSV, JSON, whatever) as there is no other way to represent variable length inputs in OH. I can see information fetched from controller but it is represented in worse form than on 2" LCD of controller itself! As an end user I am forced to invest lots of additional time to learn how to parse, display and control something which is relatively easy to change at the controller. Is this is really smart/connected home future we’re fighting for?

We’re looking forward to have OH3, improve the UI situation. I went to @ysc new UI developed as part of effort. I use later code examples only to show shortages of openhab-core. I appreciate Yannick work and commitment in moving user experience forward.
The nice widget which we have there is populated by traversing rules, parsing cron expressions and time of the day triggers.
It does mean that none of the bindings will be ever able to fill this widget in unless it will contribute rules to runtime and later monitor their configuration to map it back to device. This does not improve user experience by single penny, it does not improve binding and UI developer experience at all too.

Lack of decision to address schedule type will clearly indicate that we as a community (or architecture council as subset) are still subject of the instrument law brought above. Whether tool is xtext based, javascript or NGRE it is still a hammer; devastating tool. I apologize for the harshness of the statement, but sometimes level of faith in this tool reminds me fetishism.

3 Likes
10

Here is an working (I believe) experiment with custom type:

I modified two core classes in order to provide bunch of new types. First of all there is a TimeType which represents time of the day compared to DateTimeType which covers everything at once. I agree, name is rather unfortunate, but I wanted it only to represent just a time.
Most of HVAC controllers works with very basic schedule - time to turn on, time to turn off, sometimes allowing having several windows or providing additional mode switch. My modbus based controller for ventilation is like this, boiler is the same.

I leave this here for review to show how little (in theory) is necessary to move things forward. Modification to ItemEventFactory is few lines long. I suspect that I ignored some important aspect there causing lack of direct usage of TypeParser but we’re here to discuss things.

11

I’m not totally certain this rises to the AC level just yet. If there is a disagreement on how to precede among the affected maintainers , than I think it’s supposed to be brought before the AC to adjudicate. There AC is supposed to clear roadblocks, not dictate from in high.

But, given this could impact every single repo from the phone apps to core to the cloud server and bindings, maybe it does need to be brought up before that. Though I would like to see some sort of commentary from the core maintainers on that issue before I being it up to the AC. It’d be good to see some comments from the add-on maintainers and UI maintainers as well.

I’ll watch for a week or so and if nothing happens I’ll raise it, though it will probably be decided as a “wait and see”. If a discussion gets going, I’ll see how that goes and at heart then we’ll have something to point at for why it’s being raised.

There is little in the core and probably little in the bindings, but how much impact is there in the rest of OH? BasicUI/ClassicUI, Android and iPhone apps, HABPanel, HABApp, Cloud Server, docs? Even if the changes are minimal at each repo, it’s a significant amount of coordination. We’ve done it before though so there is precident for how to propigate new types.

12

Thanks Rich.

Ok, fair point, however I would still expect some input on direction that the framework is to take. Many of us raise issues, but there is often little engagement and feedback from those who will ultimately decide if it’s something that can be included.

I would really appreciate some form of plan - some direction for the framework for users and developers alike. I’ve said it a few times here and other places recently, but I don’t see any clear plan for OH3, other than the immediate need to refactor the namespace and consolidate the UIs. I really think that there should be some clear objectives and it’s unclear to me who agrees that if it’s not the architecture council?

Somehow, there needs to be engagement on peoples suggestions - some process for deciding what to take forward and implement in OH3. I don’t want to be sitting here in 12 months time having gone no-where - I want to see OH improving and moving forward, and capable of working with the newer technologies and concepts rather than sticking to OH1. Maybe it’s the core maintainers who should be involved in these discussions if it’s not the architecture council?

I apologise if I’m seemingly being pushy here - it’s out of a little frustration at not seeing any planning for the future, and a huge desire to see OH3 become the best HA framework around and I’m happy to help where I can.

3 Likes
13

At least to date, the maintainers for the various repos have been in there driving seat. There really isn’t a central driver. And that is perhaps a problem. But I’m not sure what the AC can do about it. The AC has no real power. It can dictate the future direction of oh all it wants but unless the maintainers and devs not only agree but actively work towards that direction, it’s nothing but a lot of hot air.

The group in the best position to be driving this capability is the core developers. They are the ones who will, at a minimum, approve these sorts of changes.

The nature and roles and responsibilities of the AC may change over time, but right now they really don’t have much power.

14

I know I’m late to the party but can I offer my two cents on this. If we allow a bunch of new types, it makes certain types of things very difficult to maintain (certain affects any UI based things but would also affect any ‘meta’ type binding as well).

Consider the NEEO remote binding - allows me to bind any binding’s channels to physical buttons on a remote or to display items on the remote. Currently I have to handle the ‘complex’ items uniquely (like HSB). If we start adding new types - makes these types of bindings almost impossible to maintain (and future ones - like the YIO remote) because I have to start coding for those types of things.

Can I suggest the following:

  1. We have a well defined base types - string, number, boolean, undefined, etc.
  2. We have ones that extend the base types to add context to that base type - time (extends number), date (extends number), etc
  3. We have complex types that contain other types and add context to that type - HSB (contains three number types), etc

Number 2 and 3 can be dynamic in nature (ie added by a binding) as long as they use the well defined base types.

The important thing is - that a UI or meta binding can work with an API to properly display/manipulate those types.

Example: a complex type should implements some interface that allows you to discover/traverse the underlying types that make up the complex type and probably have some type of method to make changes (either in whole or to specific parts of the type). An extension should have some interface that allows specific formatting/parsing - maybe even encoding/decoding of a string, etc.

I’m tired and am sure I didn’t cover this correctly - but hopefully you get the idea…

1 Like