OH3 thoughts and musings

I had a few other things going on over the last couple months so I’m a little late to the OH3 party, but I’ve finally had a chance to sit down and kick the tires and thought I’d share some of the things I’ve seen and the thoughts I’ve had.

My OH2 was a native install on my linux server and grew up quite haphazardly over the years. There’s a lot to like about the native install once it’s up and running, but, I took the upgrade to OH3 as a good chance to modernize. I switched the OH install over to docker. Spinning up the OH container and getting it running was, of course, trivial and my few very minor issues are not OH related but issues with mDNS on docker (I’ll figure it out eventually, I’m sure).

I then undertook the largish task of eliminating years of cruft by rebuilding my config from the ground up. Because I started from scratch, in some ways my experience parallels a beginners experience and some of the following comments come from that perspective.

Main UI
Bravo devs! :clap: There are of course any number of minor decisions that I personally wouldn’t have made (but that’s understandable and possibly even for the better overall; in the immortal words of Neal Stephenson, “I am a market segment of one”). Previously, my configuration was 98% in text files, but to put the UI through it’s paces I wanted to try, at least initially, to do the majority of the item building with the interface. After a few starts and stops (see Semantic Model below), I was very impressed and decided to follow all the way through, completing all item (and a large chunk of the rule/script) config with the UI. The giant leap in administration and usability is, I suspect, the reason that my rebuild took me a few days and not a few weeks or more.

One question I would have for any dev on the UI is about custom metadata on items. If you add data to one of the built-in metadata namespaces (an awesome feature!), then that namespace is actually listed
in the metadata box on the item page. If, however, you create a custom namespace for an item, that namespace is not listed and you have to go through typing the namespace into the Add Custom Metadata dialog every time. Was there a specific reason behind this choice? I certainly can’t say for sure, but I would think if the metadata box can be populated by the built-in type then it shouldn’t be difficult to have it also populated by the custom type as well. This would be enormously helpful for people who use custom metadata both to control rule function and as a result of rules. Then, at a glance. it would be possible to see if metadata had been properly created by a rule or if that item is one you’ve already set the metadata up for without opening up the json file and searching or painstakingly entering each namespace into the dialog one at a time. If there is some reason this is prohibitively difficult or has been considered and rejected then I won’t worry about it, but if it’s feasible, then I’d be happy to put in a request for it.

Semantic Model
Much of the cruft in my original config came from several different attempts I made over the years to manually implement something similar to the semantic model. In the end, I only ever maintained this for lights because I found that the effort wasn’t worth the meager or non-existent benefits for most other object types.

It actually took me a few tries to find the right fit between my digital layout, smarthome pieces, and actual house. But, in 3.0 the full semantic model is not only worth the effort, it’s not really much effort at all with the clever interface. I have enthusiastically embraced incorporating all but a very select few of my items into the model.

I did run into a few places where I’m still not satisfied with the mapping of the model. For example, all my z-wave wall switches that are connected to lights are taqgged with wallswitch and the points that control the brightness or toggle all have the light property. But what about the switches connected to my ceiling fans? Ceiling fan is an equipment type, not a property, so I am forced to choose between staying consistent with the rest of my model and leave them as wallswitch or setting them as ceiling fan to get them appropriately isolated in the equipment tab on the over view. In an extreme example, my son’s ceiling fan has never been directly wired to a wall switch, so now I’ve got it connected to an embedded z-wave switch and that switch is controlled by a separate z-wave wall switch so from an equipment standpoint it goes wallswitch -> switch -> ceiling fan for a single control point. These are the exceptions and not the rule, the model is still a huge boon both for beginners and at least in my case, slightly more advanced users.

I do have a few thoughts on the model that I feel it’s worth putting out to the community. I know Yannick has clearly outlined why the model should remain general and there’s not a lot of push to add extra locations, and I agree with all of his points. There are, however, a handful of important concepts that I feel are worth at least some consideration as candidates for adding to the model. I realize that the model is simply groups and tags and that most of the functions can be replicated by manually setting these for items, but each of the following, in my opinion, would see wide enough use and be common enough for beginner users to warrant being added formally to model:

Locations

  1. Pool - There are a few specific outdoor locations in the model (e.g., garden) but a pool area is one that many users have automation or sensors for and is rather distinct from both a general outdoor space and the other specific outdoor tags.
  2. Suite - The only “collection of rooms” level in the model is really the floor (you could argue apartment as well, but, to me at least, that implies a fairly separate isolated area), but I think one more level available between floor and room is very useful. In my specific case I have many items and rules that are not just for the master bedroom but the master bedroom + walk-in closet + master bath. Right now my master suite is just a generic room with sub-rooms, but that doesn’t really feel very satisfactory. Yannick’s argument, if I recall, for the specific locations that have already been included in the model is that these are room types that tend to have distinct functions and that multi- or general-purpose rooms didn’t need to all be represented by specific tags. I would argue that a suite of rooms, from an automation standpoint, has a pretty distinct set of functions: averaging temps in the different sub-spaces, aggregating light and presence data, etc.
  3. Individual/Person - I think it would make quite bit of sense to extend the concept of “location” in the model to include the inhabitants. Many if not most of us have equipment, items, and rules specific to distinct people in our schemes. Smartphone is even already an equipment type, but it doesn’t really make sense to anchor that equipment to a physical location in the model. This idea, however, extends beyond just personal electronics to items that hold preferences and settings for individuals in a home or even beyond. For example, a few years ago, I had a full digital sticker reward chart for my younger son built-in that would fit quite naturally into a person level category but not really any “location” within the house.

Properties

  1. OnlineStatus - One of my primary use cases for OH is a centralized location for tracking the connectivity of not only the smarthome equipment but various other devices in and around the house. From the forums it seems that this is fairly common; Rich has certainly put up several wonderful posts about the various rules and widgets he uses for tracking all of his components and many commenters have asked about linking to OH instances at remotes site for this very purpose. I think Online or Connectivity status property would be very useful both for beginners to be able to see such aggregate data in a properties card and for advanced users for to create general rules for handling offline alerts.

Other

  1. Multi-location equipment - The way the model page and the auto-populated cards work, equipment cannot be anchored to two different locations. There are, however, some fairly basic and common reasons why someone might need to be able to do. 3-ways lights are the most obvious. I have a 3-way switch circuit (as most of do, or at least should) that controls the light on my stairs. One of those is a z-wave wallswitch registered in my model, but now it would be sensible to say that wallswitch equipment is both in the main floor hallway and in the upstairs hallway. I can, as I said, manually add the equipment to that second group and that will be effective for any rules or other such things that I custom build, but that secondary location is not reflected on the model page or the location cards. It is a minor point, but again, I’d be interested in the devs’ opinion on how much that would wreck vs the sensible benefits of it.

Rules
Back somewhere in the 2.5’s I converted all my critical rules from DSL to jython. I saw the various discussions and solutions for updating to 3.0 with jython rules, but decided instead to switch them all over to js because…well, mostly because I don’t know js nearly as well as python and I like learning new things (I obviously used the js helper libraries for this as I’m not a masochist). I also spent a little time playing with the blocky option primarily just to play but also to occasionally get an autogenerated hint about how to do something new in js. Also, yet another huge shoutout to @rlkoshak (Is there a “Shoutout to Rich” tag for posts? There should be.). Although I didn’t actually use any of the helper scripts he’s made available, I did peruse them early and often in my learning process for js timers, and his posts on context and the this object are invaluable.

The rules interface is also really well done and added exactly 0 difficulty to my process of working up my rules from scratch. The ease of moving between, or combining different actions is incredible. The linking up of the item triggers and condition to the state description metadata is a fabulous touch.

Amusingly enough, the one thing I couldn’t figure out was in blocky. Maybe it’s there and I just didn’t poke around enough, but I couldn’t figure out how to compare a number item state to a number. There doesn’t appear to be anyway to cast item states in blocky and the if block’s type checking won’t accept an item state block if the other block in it is a number. If this is actually an omission then it should probably get a request as I can see this being fairly important from a beginner user-friendliness perspective.

Controversial Opinion
I am assuming that none of my other musings are going to set off any significant issues. I do, however, have one opinion that might be polarizing. I think item uids should be mutable (this does not apply nearly as much to thing or rule uids). I know the arguments against it; I’ve even seen a few very recent posts going over them, and I understand the potential for downstream breaking changes in a config if an item uid is changed. For those reasons I don’t think it should be as easy as any of the other item editing that can be done in the UI, but protected behind a few extra clicks and confirmations or even only enabled as an advanced option in the system settings. Nonetheless, I think it’s a good idea specifically for those users who have gotten past the very most beginning steps and are on their way to becoming more advanced users (i.e., users who also likely understand the potential consequences of changing a uid but can see the utility behind improving the readability of their config).

It seems with the changes to 3.0 that OH development is emphasizing attracting new users with less experience and coding know-how/desire. But part of the growth that those users need to eventually be able to have is to build a config that is structured in a usable and sensible manner. One of the most key components to that, in my experience and opinion, is a consistent and understandable item naming convention. Some of the requirement for this had been removed because Main UI, in all its item dialogs, shows the human readable name as well as the the item uid. That doesn’t hold, however, the moment the user begins to explore scripting and rules writing.

As I was initially re-building my config in the semantic model, I actually hit a few dead-ends and restarts in terms of developing a naming convention that was sufficiently global, useful, and sensible with the model’s location->equipment->point structure. I’m comfortable enough to dig into the jsondb files and reconfig that way but I don’t think that’s something that newer users should be encouraged to do and the process of deleting, recreating, and re-linking items just to rename them is simply not a bulk solution.

3 Likes

The REST API does not support querying for all the metadata on a given Item. You have to know the namespace for the metadata ahead of time in order to pull it through the REST API. And if the REST API doesn’t support it, the UI can’t show it.

I think there is already an issue open but you should go search and see. If there is one add a comment. If not go ahead and create one.

It should indeed be a relatively trivial endpoint to add. I don’t know though why it wasn’t added already so there may be some technical difficulty preventing it.

This is an important point. Not everything needs to be nor should it be part of the model.

Create an Equipment Group tagged with Ceiling Fan. Then add the Point Item with control as the Point and something appropriate as the property (I’d probably choose Power).

Then I’d make an Equipment Group tagged with ceiling fan and add both the ceiling fan’s switch (as above) and the zwave switch as members of that Group. Or perhaps leave the ceiling fan’s switch outside of the model entirely and only include the zwave switch.

It is expected that an Equipment might be made up of Items linked to Things from different bindings. The simple example would be a Status Power tagged Item linked to the Network Binding to show the online status of a WiFi devices.

Personally I think there are already too many location types in the model. Really, what is the difference between a bedroom and a family room beyond the name? I’d much rather see fewer locations and either an adoption of synonyms like HABot uses to further define the type/name of the room. The more specific stuff like this that is added to the model the more work it is to maintain (each of those needs to be translated into a few dozen languages for example).

But having said that there is an issue open in openhab-webuis to add stuff like this to the model. I think pool was already added

However, note that I only really track in OH the status of those devices that are relevant to the home automation. For example, if a sensor goes offline I can issue an alert and change the behavior of rules. openHAB itself is really kind of a poor general IT monitoring system and I strongly recommend using Zabbix or Nagios or the like for that purpose. It’ll be way more capable and way less work.

I use Status/Power as the tags for now. If there were a Status/Network I might use that instead.

The switches and the lights are for all intents and purposes one Equipment. So I would put them all into one Equipment and put that Equipment in the location where the actual Light is. After all, it’s the Light that you really care about. The switches are just ways to control the light. And from the model/OH point of view does it really matter that the switches themselves are in other rooms? All you’ll really care about is controlling the actual light.

All I can really say is at this time Blockly is unfinished. If there is anything you can think of that I’ve missed, I’ve opened an issue on openhab-webuis listing all the things I think are missing.

Anyway, that is a bug I think an issue was filed for already. To work around it, assign the Item state to a variable first and then use the variable in the if condition block.

As OH exists today it is simply not technically feasible. Links will become orphaned. Metadata won’t follow to the new Item name and will become lost. Rules will break. UIs will break. all the Persistence data will be lost.

From an internal perspective, the Item name is the UID used as the key for the database. You cannot change a UID in any database Iim familiar with as a matter of design.

The only way to “change” an Item name is to delete the old one and create a new one and then clean up/recreate everything that used the old Item. So be careful with your Item names is the best that can be offered.

But it’s worth noting that rarely does the Item name even matter any more, particularly if you are doing everything in the UI. You hardly ever see or use the actual Item name. Almost all of the time you will be seeing the Item Label, not the name.

But it applies there too. When setting triggers you select the Item from a list that shows the labels, not the Item names. And when referring to Items inside the rule, the developer sidebar provides a handy way to look up an Item’s name if you can’t remember it off the top of your head. In some circumstances the autocomplete will also fill in the Item name for you.

Can more be done? I’m sure the answer is yes. But because of some decisions made half a decade ago in how the JSONDB was structured I don’t think it is ever going to be technically feasible to rename an Item, at least through the UI. I’ve documented a procedure one can use to edit the JSONDB manually to change the name of an Item. Doing that will only really lose the persistence data instead of breaking everything.

2 Likes

I did the same as you. Fully nuked and started fresh in OH3. Recreated all my things, items, and js rules in the UI with no text files. I think the only text file in use is the persistence file for influxDB.

I love text files. But I love being able to modify configurations from somewhere other than sitting at my computer at home. I can be 1000 miles away and make config changes now very easily. I miss the text files because I could make changes really fast rather than clicking a bunch of menus. But the pros vs cons win for the main UI config.

I was never happy with habPanel. Just didn’t like the process for designing it and didn’t like interacting with it. It just rubbed me the wrong way. So I used sitemaps and basicui. This of course was very… basic… and had it’s very hard limits. I am really digging the OH3 main UI. Copy/pasting text YAML to keep a consistent layout probably helps with my like of it too. I’m making pages for wall mounted tablets, pages for my phone, pages for desktop, etc. If nothing else, it’s kept me occupied. I am no longer using sitemaps or basicui. I’ve been able to do everything I need in the Main UI, and I’m doing it way better than I was before. I am quite pleased.

I also found that numerous zwave devices that sucked in OH2.5 now work flawlessly in OH3. And these weren’t new devices either. My Linear garage door opener has been around for a while and sucked. Now everything works as it should in OH. I was pleasantly surprised by that little gift.

My only complaint / problem:
There remains to be a major performance problem with Rest API Auth in OpenHabian on Ras Pi devices. This makes logging in remotely through NGINX Basic Auth impossible, at least with NGINX in the recommended configuration. Multiple people have reported the problem with heaps of data and logs in forum posts and git issues.

The maintainer for OpenHabian says there is way to make it work fine, but is actually refusing to share it out of some strange spite. If I acted like this in the org where I am a maintainer, it would be completely unacceptable, and it would have been the end of my role as maintainer for sure. I’d report the post, but he’s the forum moderator too. If you’re suffering from this as many people are, you have my sympathy and I’ve tried.

Another community member wrote a fix for it and opened a PR, but it is not gaining any traction with the development team. As I’m a volunteer maintainer in another large org too, I get the volunteer thing and it’s not a criticism. It just is what it is when there isn’t a paid staff of developers waiting to fix every problem. Unfortunately, that means this problem is staying a problem for foreseeable future until something changes.

For the above reasons, I have abandoned OpenHabian on my pi and moved it to a Windows machine. This was surprisingly simple. Backup on the pi, restore backup on windows, like nothing happened. I moved influxDB and Grafana to Windows as well, also without a hitch. The rest auth performance issue does not have negative impacts on a hefty machine like this, so it is working much better. I still have issues where I need to clear cookies to log in, but at least it works.

That’s an outright false accusation.
I disliked your aggressive tone and post and still do. It is absolutely not OH style.
Reading several of your posts made me wonder if I should do right you a favour and after some thinking I as the moderator that I also am decided to make a public statement that this sort of behavior is unacceptable (your wording) by not helping you in person.
Anyone else of course is welcome to ask and I think I made that quite clear with my response you linked to (if not I think now I did).

The issue itself (granted, IMHO) is of lesser importance as OH is designed to run locally in the first place so no need for administration from the outside. Despite of your claim, there wasn’t anybody else asking but you. Granted I don’t read every thread so yes it may have slipped my attention but for sure there will not have been many. There’s also many potential solutions to work around this such as to use a different proxy software or the old (user) UI.
So the issue is nowhere in terms of importance near where what you would like to make it sound.
I just happened that you have it. Ok, bad luck. But now, instead of helping you with your “perfectly valid request”, some voluntary responder dared to point out the inappropriateness of your wording.
Now that ! A maintainer even ! Scandal, call a moderator ! (or as Greta said: How you dare !)
Strange ? Maybe. Spite ? There’s multiple translations but they’re all insults. No comment.
A statement ? Yes.

For what’s it worth, I made the nginx config I use myself part of openHABian well before your post so it has been available to anybody and it still is available today.
I don’t know if it’s a solution to your specific problem because I have never encountered that myself and might as well have totally misunderstood what that is about. But then again I never claimed I do.

For completeness’ sake, and hopefully as a solution also to you, see OH3 with NGINX Reverse Proxy and Authentication - #44 by ysc which is what I have been using and implemented and shared in openHABian.

@mstormi , @Pedals2Paddles It’s probably best to just let this go for now or move it to PMs. This discussion is not productive.

Well my tone and posts were not “aggressive” and I see no reason for you to even mistakenly take them that way. I’m sorry but you’re incorrect. Perhaps you’re misinterpreting my eagerness to see a problem get fixed, and the internet does not convey tone well. One way or another, your assumption is wrong sir. And your response is incredibly inappropriate and does a disservice to the community.

Also false. There are multiple long running threads on this forum. Multiple issues in git. A PR in git with comments. Claiming it is just me is 100% wrong sir.

This occurs using your configuration from the OpenHabian config tool, which I actually pointed out to you in this thread, along with others who were experiencing the same problem. I don’t expect you or any other volunteer to be perfectly on top of every post and issue all day, as I am can’t be either where I have a similar role. But if you bring it up, I will point it out.

I don’t remember if this is how it is but I will check when I get home and test. If it works, fantastic. But I’m guessing since it was not working as it was straight out of config, it wont do it. We’ll see.

Not false, but I should have clarified that I meant “after you made that post”.
I obviously don’t know your issue’s history so it’s as-obvious that I’m not referring to that.
Either way, there has not been any single openHABian issue opened.
Which would have been the first place to put it when the solution is in nginx because nginx config is part of openHABian but not of openHAB.

I’ve been a proponent of the idea that OH would have beginner, intermediate, and advanced modes, so that new users would have to intentionally choose to step up a level. However, it would be difficult to determine what should be turned on/off at the lower levels, so I don’t see this actually happening.

As I’ve thought about it more, a user-security model would make more sense so that users have to intentionally give themselves access to features with higher degrees of difficulty. It’s an extra step, but I feel like it would help reinforce when things are going beyond “basic”.

I feel like OH3 is already moving in this direction with the functions placed in the “developers tools”. I can definitely see where it would be worth pushing for a more comprehensive list of features that would make sense behind an “advanced” label.

Although, as I said and @rlkoshak probably said better, OH3 does take some really good steps toward smoothing over the some of the old more technical issues that might separate a beginner from a more advanced user such as the ubiquity of access to item via their more human readable labels. So perhaps with just a little more growth OH3 will further pare down the list of specific advanced features.

1 Like

Yeah, that’s something I should have emphasized more. I actually expected to give up on using the UI for building out my items after flirting with it for a little while because I, too, really felt like the text files made sense, made things simpler in many ways, and were more expedient. I didn’t even have issues with remote admin because I had ways of accomplishing that in the cases where it was a true necessity. (In my OH2, the only items I had that were not in text files were those that needed persistent but variable custom metadata.)

In the end though, as you say, there were so many pros to the UI item config system that I didn’t ever even hit a moment where I had to stop and think about whether or not to continue or switch back to text files. It just worked and worked smoothly.

Of course. I should have caught that; that’s a great point. It would have to start with a new endpoint for the API, but I can also think of a couple other use cases I’ve had over the years where getting a list of an item’s namespaces could have streamlined my scripting (which could be more of a comment about my scripting than the utility of the API). So, maybe it is worth opening up or continuing such a discussion. I’ll check and see if there’s a relevant issue.

Definitely. Many of my Equipments have points from multiple things. I guess my point with the ceiling fans wasn’t that there isn’t a way to shoehorn these particular situations into the model (and, in fact, your solution is pretty close to what I settled on) but that for so much that I did, the model works so elegantly it’s somewhat jarring in these situations where that elegance breaks down and you’re forced to settle.

For the most part I would agree with that, and am not opposed to some of the calls to try and make the tagging system more general but with user defined synonyms. For some more advanced features, however, (off the top of my head, I’d say the item registry’s getItemsByTag method), specificity in the semantic tags is extraordinarily useful. I certainly have general rules for bedrooms about lighting in the evening that don’t apply to other living spaces. There are other ways of getting to the exact same list of items, sure: my own tags or running rules on my own group members instead, but those are extra layers of complexity to track that don’t have to be there if the already existing model is grown out just a little.

That said, I agree mission creep is real and a thing to be feared. Knowing where the threshold is between adding enough to help and adding too much to bog down is something I leave to those more experienced and knowledgeable than I.

I’ve learned this the hard way it feels like about a thousand times…I am delighted to see that OH3 does in deed take some of this onus away, especially for anyone who isn’t doing a lot of scripting or widget creation.

Your points on the technical feasibility are well taken. It would be a huge effort in many respects and probably not worth it even in major updates of OH3. That said, I still don’t think it’s a completely valueless thought experiment.

This is perhaps the most critical point. I did not discover the full utility of the developer sidebar soon enough in my initial forays and probably should have put some more in my op how useful that feature is.

2 Likes

If you’ve spent any time working around ontologies like this, you will know this is a common problem. The ontology has to take some firm assumptions and positions and when you approach the edges it tends to break down a little bit. I’m not at all surprised that the OH model falls down a little when you move away from the edge cases. But I find if one is careful with how the equipment is defined even those edge cases can be handled.

Indeed. If you will be doing almost anything in the UI, the developer sidebar should be a major part of your workflow. For those unfamiliar with it, to open it hit alt-shit-d or open it from the Developer Tools. Once open you can:

Pinned Objects:

  • search for and see the current state of Items
  • depending on the Item’s card widget you can send commands to the Items
  • once found you can pin them so you can have a list of all the Items you are working with pinned to the side for reference
  • search for and see the current state of Things
  • search for and see the current state of rules
  • pin rules and run them/disable them

tl;dr: You can pin almost anything to see it’s status and interact with it on the side as you build a rule/widget/model/etc.

Event Monitor

  • shows a stream of all events which includes
    • Item updates
    • Item commands
    • Item changes
    • Thing status changes
    • Rule status changes (can be very handy when debugging rules)
  • Can be filtered so you only see the events that are relevant to you at the time
  • Is more complete than events.log

Code Tools

  • A place to test widget expressions, invaluable for testing and debugging custom UI widgets
  • a button to open -Scratchpad- a special Script rule where designated as a place to experiment or create events for testing

Create Shortcuts

  • A list of shortcuts to create various entities in openHAB (Items, Rules, Things, Widgets, etc) that takes you straight to editing the new entity so you don’t have to navigate through the settings

The Developer Sidebar is key to achieving a high degree of productivity when working through the UI.