Roadmap to Happiness - What is missing in the core framework

Here are two more things that have been rolling around in my head, but I do not believe there are associated ESH issues for them (yet)ā€¦

  1. For the Basic and Clasic UIs, it would be really nice to be able to use Item StateOptions, without having to link the Item to a Channel with a DynamicStateDescriptionProvider. A simple use case is to select from a list of radio stationsā€¦ but to really be useful, youā€™d need the ability to update the list programmatically.

  2. A user interface for viewing and editing Item metadata. Iā€™m close to sitting down to build one, since I am using Item metadata in my JSR223 rules and would like an easy way to modify it. The REST API lets you add/remove metadata, but does not allow for reading it. The use of Item metadata has so many useful possibilities! I expect the lack of the ability to create/edit it in a UI is simply due to how new it is.

1 Like

@mstormi

I have 200 rules, i want some rules to run before the others - priority ā€¦

I know that NULL is the default, but why not to have the ability to have different default? Writing a rule is overkill just for default values. If the item gets the value from persistence, the default value will not have any affect.

Then itā€™s not a default anymore.
There are plenty of examples on how to deal with Null values at start up. Changing it will only change the value not the process

Have you opened an issue on Github? It would be even more awesome if you indeed could supply a pull request to add this missing piece.

Yes please, but with grep like filtering. :smiley:

Iā€™d be very interested in SSL certificates with the embedded MQTT broker.

Yes please.

:+1:

This one is huge. I canā€™t tell you how much time I and others spend on this one issue on the forum. I just spent 10 minutes helping someone with this very issue. The big deal is rules start running before Items are done being loaded and restoreOnStartup causing havoc.

I think Davidā€™s comment means that GUI based Rules editing is already in work. We donā€™t need to open new issues for stuff that is already being worked. I donā€™t think anyone is saying that the ERE is done and usable yet. As you mentioned, the documentation isnā€™t there yet.

The ERE will make this very easy FWIW. You can do this right now with just a few clicks.

:+1:

NOTE: depending on what you are using these for, there might be other options. I documented a few in Design Pattern: Encoding and Accessing Values in Rules

The problem with this is Groups, despite how many people use them, are not strictly hierarchical and Items can belong to any number of Groups. Iā€™m not sure using a Group name would work in this context.

Rules are event driven and are processed fifo. An event occurs and Rules configured to trigger based on those events. And no two events ever really occur at the same time. And unless there is something wrong with your Rules there should always be a thread available to start processing the event.

Given this is how it works today, how would you define the priority and what would you do in all the various cases? For example, I have a Rule that is already running at a lower priority, do I kill it if a Rule at a higher priority comes in?

Iā€™m not arguing against the idea, just trying to understand how it fits.

Then one would argue you need to make sure the events occur in the order you want them to run. Short of a complete rewrite for how pretty much all of ESH/OH works I donā€™t see how priority can come into it. Maybe instead of fifo a priority queue could be used to assign a Rule thread to an event, but unless you have long running Rules, which is a problem unto itself, there should be no wait. No wait, no priority.

From an architectural purity perspective I agree with you. However, I have helped many users over the years with use cases that became somewhat needlessly complex because of this.

I think the sticky point is if one wants to display the lastUpdate on the sitemap there is a need to create separate Items and Rules to populate those Items. Iā€™m personally kind of meh about the idea since it is so easy to solve with Design Pattern: Associated Items but I do understand the annoyance.

My list:

Generic Transformation Service

Can I expand nested transformations to pulling transformations out of the bindings entirely into a generic transformation service? Why is it I can only use a transformation with certain bindings (e.g. HTTP, Exec, MQTT, etc) but not others (e.g. Zwave, Zigbee, etc)? Why are the bindings involved at all? Iā€™d like to see a way to link a Channel to an Item and apply a transform to the Channel at the Item, not on the Thing.

For a relatively trivial use case, letā€™s say I have a bunch of Contact door sensors. Contact is the appropriate Item type for this type of sensor. But all the Zwave door sensors require the use of a Switch. Now I either need to create a bunch of proxy Items and Rules to convert the Switches to Contacts so I can now use a Group:Contact for all of them or I have to deal with a Group:Switch and Group:Contact and duplicate the logic.

For another simple use case, I sometimes want to use a Number Item to store the result from an Exec script, but the output channel only lets me store the result in a String. Maybe a generic transformation service would let us write transformations that frees us to use the appropriate Item type for our home automation model instead of what the binding developers decided was appropriate.

Higher Default Number of Rules Execution Threads, Dynamically Sized Pools, Error Logs When Threads Are All in Use

Iā€™m hesitant to request this as at best if masks a problem in peopleā€™s configurations. Five should be enough, but for some power users it seems not to be.

And 2 is definitely not enough for the Quartz thread pool. Especially with the natural human propensity to schedule events to happen at midnight or on the hour.

Very simple changes but I donā€™t know the overall implication.

Even better would be a pool that grows to some maximum and then starts logging errors when we get to the maximum.

At a minimum, it would be nice to get some logs stating that we have X rules awaiting an execution thread. Right now it kind of fails silently.

Sitemap Widgets for Filling out DateTime Items

Itā€™s pretty self explanatory but one just needs to look at the massive complexity and pain involved in all the Alarm Clock examples to see how nice this would be.

It would be nice if it handled full DateTime as well as just a Time.

This may not fit with the core framework but it is a prerequisite forā€¦

Dynamic Scheduler

Related to the previous, a built in scheduler. Lots of people go to great lengths to be able to dynamically set up a schedule for things to happen at certain times on certain days of the week on a repeating basis. For now all we can do is rely on something like the CalDav binding, which doesnā€™t work well for a lot of users or the creation of dozens of Items and complicated ugly sitemaps.

This is a major weakness in OH right now.

What people need is a way to say from their sitemap/HABPanel ā€œSend MyItem the ON command every weekday at 8:30amā€ and create a Rule that triggers when MyItem receives the ON command. The Scheduler is smart enough to interpret the Itemā€™s state and schedule the events accordingly.

Maybe this belongs in itā€™s own binding, but Iā€™m pretty sure there is a need for at least one new Item type to make this work which does impact the core.

Add/Edit Tags in PaperUI

This one is a real challenge for less technical users who want to use Alexa/Google Assistant/Hue Emulation.

Elimination of Contact or Creation of A Standard that Gets Enforced

Contacts are supposed to be used for sensors, Switches are supposed to be used for actuators. But this convention has not been followed by the binding developers. In fact, rarely do I see Contact used at all. While there will be much wailing and gnashing of teeth, Iā€™d like to see either the binding developers follow the standard convention or the elimination of Contact as an Item type.

Status Update on the ERE

This probably doesnā€™t deserve a an issue and may not even belong here, but whatā€™s the status of the ERE? I know simple Rules can be set up but it is hard to judge itā€™s maturity. Can we access Actions (I know executeCommandLine is not available yet based on a recent thread)? Logging? Timers?

Iā€™m willing to take a shot at starting the documentation for it but donā€™t want to start unless it is nearing completeness.

Iā€™m sure Iā€™ll think of more in the shower tomorrow morning. :wink:

Improve Performance of Rules DSL Parsing

There might not be much that can be done with this but users are reporting start up times of 10-40 minutes on RPi 3s due to Rules parsing. I suspect the main focus is on the ERE right now, but the Rules DSL will be around for a bit and we shouldnā€™t ignore this pain point.

3 Likes

Create a new bug report for every single wrong usage youā€™ll find :slight_smile:

1 Like

No, not yet, but I plan toā€¦ unless someone else gets to it first!

Yes, it would, but very unlikely :wink:. Iā€™ve done some other simple pages that used the REST API, so I figured Iā€™d throw one together in the interim, if making changes without a UI bugged me enoughā€¦ but without the ability to read metadata through the REST API, I donā€™t believe this would be possible. I donā€™t have the ESH knowledge to develop the Paper UI, and there would be deeper work needed to be done first. With the recent work in Semantic Tagging, I also expect there isnā€™t a very firm codebase to work on. I think this is one that would need @Kaiā€™s lead, if he doesnā€™t kibosh it all together.

Rich also mentioned editting Tags in a UI, which is closely related to this.

This is a PaperUI, ClassicUI shortage and need to be reported as an issue, I strongly agree. There are so many basic widgets missing actually. Like Push Buttons and Multi-State Buttons.

Yes (executeCommandLine works fineā€¦ which thread?), yes, and yes. :smiley:

@chris is going to hate meā€¦

He doesnā€™t agree (we had a thread on this a year or so ago) IIRC.

Iā€™m talking ERE, not JSR223. Specifically editing the JavaScript in the browser.

2018-10-23 15:51:04.416 [ERROR] [internal.handler.ScriptActionHandler] - Script execution failed: ReferenceError: "executeCommandLine" is not defined in <eval> at line number 1

Iā€™ve tried several formats and ways to define the script, mostly blind since I donā€™t know what is appropriate here but I always get something indicating that my JavaScript is completely malformed or it doesnā€™t know what executeCommandLine is. But based on the JSR223 Actions.js examples that line should work.

I know there is some overlap between the ERE and JSR223 but they clearly are not the same and features that are available in JSR223 are not in native ERE. And since the vast majority of ERE users will be writing the scripts like the above that is what Iā€™m looking for status on.

Jython and Groovy are not options under ā€œthe scripting language usedā€.

tl;dr, just because it works in JSR223 doesnā€™t mean it works in ERE, yet.

I donā€™t remember disagreeing with this ;). I donā€™t mind people opening up multiple reports unless they are clearly linked and weā€™re trying to solve a specific issue. One thing I do dislike is people assuming that two issues are the same when often they arenā€™tā€¦

I have one: The ability to tell if a device was changed from outside of openHAB (for example a user manually turned on a light) or from within such as a rule. Even something as simple as the triggering source property being either something like ā€œsystemā€ or ā€œbindingā€ would be useful I think.

  1. Id like to see the Private cloud of Openhab Cloud being intergrated, ie a component fully installed as part of the install. You either select internal or external. Not that the external cloud is unreliable but I prefer all on premise

  2. Echo the remote access restrictions, there needs to be more granular control over security for other users as Vincent as mentioned. Allowing certain users to see/do certain things would be great

  3. Graphical interface of the debug system with the ability to easy filter based on binding etc

Hey

Thatā€™s a great question thanks for asking! So here are just a few ideas for issues i have had:

  • Built in user authentication (Instead of using nginx proxy)
  • Built in SSL generation using Lets encrypt (Instead of manual generation) or manual addition using Paper UI (this would also solve the need for nginx)
  • Built in configurable proxy (give the ability to access things like camera images and video streams without opening individual ports for each camera on the router)
  • Built in Terminal client in Paper UI for CLI access instead of using third party tools
  • Built in DynDNS or other DDNS reporting (auto update dynamic dns)
  • Built in UPNP client to open router ports (auto open router port for access)
  • Separate log/register for device state changes (Instead of scrolling through miles of log files)
  • Tagging support in paper UI (instead of having to manually add them through the console, useful for Alexa integration )
  • Ability to trigger zwave inclusion and exclusion from sitemap items
  • Ability to have code generated sitemap items or dynamic drop down lists (Listing radio stations, songs, etc)
  • Ability to have multi-line sitemap items
  • Built-in GPS tracking and reporting in the mobile app (This might be a bit out of the core scope)
  • Built-in Text input fields in the mobile and basic UI (This might be a bit out of the core scope)

I know there was a couple more things that would be nice to see but I cannot remember them right now, so I hope this list helps.

Thanks!
Sebastian

2 Likes
  1. Schedulingā€¦Schedulingā€¦Scheduling
  2. A method to see the status of timersā€¦how much time is left

It is not. Depending on the item, you may want to reset it to default or keep the persisted value, so you need a rule in any case. You can deploy a single rule to trigger on ā€˜System startedā€™ to set all default values in one go, and unlike with a simple item property, you can even apply conditions here.
Iā€™d even go further and say that the ability to set it via config will result in a number of issues where people set defaults and forget about when programming their rules. From someoneā€™s point of view to spend a lot of time on supporting other people debug their rules Iā€™d therefore even vote against having this.
Yes the need to check for NULL is annoying but thatā€™s integral part of proper programming anyway, and the main root problem is the unreliable startup order so rules sometimes get processed while items arenā€™t available yet. But that issue already is on the list.

Either way, this thread is meant to consolidate needs that several people have in the core framework so developers know what they should focus upon, and my point was that none of your points require any changes to the core but can be solved differently without too much hassle really.

On Smart Home Day, @henning reported plans to replace the current scheduler.
I agree it would be great to also allow for user-provided scheduled actions/rules to be managed by the framework, maybe thereā€™s a chance to combine this ?

@kai stated that essentially ESH automation engine is to replace the ā€˜oldā€™ engine in the foreseeable future (hereā€™s the recording).
But I think thereā€™s still many details that need to be sorted out.

I like this point. An automatically updated list of running timers (more than 2 seconds timeout time) would be amazing.