Making OH Rules Easier for Everyone

Creating a new topic for the conversation around rules and making OH easier. I would like to see this continue, hopefully providing a diverse view, for how to make writing rules easier.

I’ve pasted in where the conversation started drifting off topic, here is a link to the original topic: Could not cast 7.7 to ...DecimalType in rule

rlkoshakRich KoshakFoundation member

2d

Could the OP’s rule have failed due to an object having (or seeing, for lack of words) two states and the OH rule being confused?

No. An Item can only have one State at a time. It is certainly possible that an Item could change state while a Rule is running and that can cause some problems when people do not plan for that or expect it. It can also take some time for an Item to reach a state after it has been commanded or updated which causes some people problems (e.g. sendCommand to an Item then immediately log our the state of that Item and see in the log that the Item is still in the old state because the Item hasn’t finished processing the command yet).

But none of those cases are present in OP’s Rule and the error he saw had to do with type.

The reason it was confusing was because it says “Could not cast 7.7 to DecimalType” meaning that the 7.7 is getting to the Rule somehow which should eliminate the possibility that the Item is NULL. And a number Item can only have a DecimalType or NULL as its state so the error really doesn’t make sense. Something else weird is going on, perhaps having to do with the calls to doubleValue, but there again I would have expected the error to be different.

Now the problem that I discussed in that thread you linked to is different. In Object Oriented Programming, There are two relationships: “is a” and “has a”. What that post is showing the “is a” relationships for the DecimalType class. The tl;dr is that DecimalType is also a Number. The one Object (instance of a Class) is both types at the same time.

Given that, let’s say we have the following sendCommand methods on the NumberItem class:

  • sendCommand(String cmd)
  • sendCommand(DecimalType cmd)
  • sendCommand(Number cmd)

Now lets say I call sendCommand with a DecimalType. The Rules Engine gets confused because DecimalType is both a DecimalType and a Number. Consequently two of the three sendCommand methods are valid and it doesn’t have any other information to go on to tell it which one is the one it should call. Both are equally valid. So it generates an error; a different error from here that says something like “Ambiguous method call blah blah blah”.

This is why I recommend never casting to DecimalType in Rules and always casting to Number. In your Rule you want to treat the State as a Number anyway and it avoids this specific error.

But, like I said, that error is not what OP experienced above. Why there was an error and why just casting to Number instead of DecimalType fixed it.

I’ve been toying with the idea of a “just enough programming tutorial to write OH Rules” but it’s a really big problem. How much is just enough? How much is too much? Is it really appropriate to create and maintain as part of the OH docs themselves? Should I write a book?

H102CM6.5 H102

1d

I’ve been toying with the idea of a “just enough programming tutorial to write OH Rules” but it’s a really big problem. How much is just enough? How much is too much? Is it really appropriate to create and maintain as part of the OH docs themselves? Should I write a book?

I’ve been thinking about this and agree it’s a really big problem. Most non programmers search the site for rules, to copy and paste, without really understanding how the code works. This works for the most part but if adjustments are needed for extra functionality then a post for help usually results in a solution and everyone is happy.

So, the question is, how many people are willing to read a book and learn vs posting a problem and receiving a solution? You may be able to answer that question by looking at the number of post, for help with a rule, and the amount of info that is given by the poster. How many times have you seen a reply, to the OP, that ask to provide more info than just the error message? If there wasn’t a description of what to post when creating a new topic you could simply say they didn’t know any better.

Personally, I would love to see you make a tutorial or write a book about OH rules. Maybe start with a good foundation, what is an Object, Function, Class, etc… and how they work together. I’m not a programmer, so this may be similar to someone trying to do calculus or trig without learning algebra first. In this case your book may require a few volumes. :thinking:

On the other hand, if you do just enough, how often would you have to revise your tutorial. I’ve read several post about OH updates and rules not working afterwards. The solution often involves something to do with how the rule was coded. This also brings up the question, how many version of OH would a just enough tutorial cover?

I’ll stop my rambling here because I suspect you’ve already thought thru everything above, plus some.:grinning: My opinion would be to write the tutorial/book, start with a good foundation and understanding, that allows us to adapt to future changes and possibly even community contributions.

diyhaDiy HomeAuto

1d

These were exactly the issues I was frustrated with Xtend. Lets try Kotlin, I need to work hard. Kotlin does automatic type casting e.g String 7.7 to target Double 7.7.
Rich, don’t write your book yet man, wait for Kotlin support! :slight_smile:

H102CM6.5 H102

1d

I see you just replied to a post with: “Can you please paste full rule, or at least a stripped down version that reproduces your issue?”:smirk:

Xtend, Kotlin, JavaScript, Python, etc… doesn’t make a difference when your having to ask for info that should have been provided as part of a new topic. I can’t see an agenda, at lease collectively, for a tutorial or book helping when people are already doing less than minimum.

Yes, I want a book but I won’t be selfish. There are many others that would benefit but the big question remains. With all things considered, is the time and effort really worth it?

diyhaDiy HomeAuto

5

1d

is the time and effort really worth it?

Yes, till we monopolize the automation market. :wink: Our rivals (in my opinion) : Samsung SmartThings, Home Assistant, Google Home, Amazon (must be planing big), Belkin Wemo, Nest Labs, Wink, Apple HomeKit and various “hubs” like Vera that are out there without any strong community support or open source code, but they may got money power, they can buy a lot of support staff.

Its a transitional phase, that experienced guys like Rich, you, @5iver have to spend their energy on little issues, we can cover most of that, hopefully, with modern languages like Scala/Kotlin. UI based rules, will cover, maybe basic stuff. If newbie’s are having issues and we don’t respond, we lose users, and eventually the project will be irrelevant.

But if we monopolize, the army of app developer community will help users with common+complex stuff, we just need to give devs a solid platform.

Bear in mind Microsoft had an army of support staff to help customers, to ensure their monopoly.

rlkoshakRich KoshakFoundation member

1d

On the other hand, if you do just enough, how often would you have to revise your tutorial.

So when I say just enough programming I mean programming concepts. The sorts of things that exist in almost every programming language, presented with the Rules DSL as the exemplar.

From that perspective it would some never have to change because the concepts never change and for the most part they way they are implemented in the Rules DSL are not something that will change because they are fundamental to how the language works.

Three posts and changes you are referring to are caused by the parse getting a little more strict and providing better error reporting as a result (i.e. some rules with errors used to work but now the errors can’t be ignored) or they’re are changes to how Rules are defined which is not a general programming subject but something very specific to Rules DSL.

And notice I keep saying Rules DSL instead of Xtend. Rules DSL != Xtend. There is some overlap but they are not the same.

This also brings up the question, how many version of OH would a just enough tutorial cover?

All of them. Anything more specific belongs in the Rules docs, not in a separate tutorial or book.

Rich, don’t write your book yet man, wait for Kotlin support

Even if Kotlin were available tomorrow, the Rules DSL would remain the default rules engine for a good long time. No book I wore would become OBE anything soon.

for a tutorial or book helping when people are already doing less than minimum.

No but it sure saves us a hell of a lot of time to be able to respond with a link to an answer instead of rewriting the same answer over and over.

I’d like to be able to respond with a link to this for some of the parts for help that show a complete lack of knowledge of how programming works (variables, if statements, etc.) The same way I can respond with links to the Beginner’s Tutorial and concepts section of the docs when users misunderstand let OH concepts.

hopefully, with modern languages like Scala/Kotlin.

Don’t hold your breath. Even using Scratch isn’t going to help self proclaimed noon technical people understand how to program rules. One of the reasons I actually like the Rules DSL is because it purposefully removed whole features of the Xtend language to provide a simpler programming environment for new users.

Having access to JSR223 languages, Kotlin, Scala et Al it’s not going to do anything to help those users. Experimental Rules Engine and perhaps if Chris resurects the rule builder in Habmin will. All these more robust languages are great for those of us who are already developers but they don’t help the non-technical folks at all. In fact they raise the already high barrior to entry even higher for these types of users.

Helping theses users to “think like a programmer” would be my goal. The rest is just learning syntax and libraries. And the Rules DSL doesn’t have too many of the latter and the former isn’t really all that bad.

Adding Kotlin or any of the other languages (you might consider Processing as well) are great for OH developers and great for those who are already programmers or inclined to learn. There are IMHO a step backwards for supporting the users who have never programers before and who just want to turn on their fan when the temp goes above a threshold.

diyhaDiy HomeAuto

1

23h

All these more robust languages are great for those of us who are already developers

Exactly, I want to attract more of those, but first we need to have a “decent+modern” language, IDE support and tool set and integration with ESH core in place. Then those devs will do custom, region specific apps on those foundations. I never meant non-tech users to digest our scripting paradigms. UI rule engine should stretch itself on those lines.

H102CM6.5 H102

23h

Helping theses users to “think like a programmer” would be my goal

YES, this would be very beneficial to the community, regardless of the language now or in the future.

Book title, “Thinking like a Programmer an OpenHab Foundation” ?:crossed_fingers:

diyhaDiy HomeAuto

1

23h

I agree with this thought. Every kid should be taut how to think like a programmer. Its gonna save species in long run when we become multi-planetary. There won’t be any community helping those on distant planets to fix their issues with spaceships or smart homes in those environments.

H102CM6.5 H102

23h

I agree with this thought. Every kid should be taut how to think like a programmer. Its gonna save species in long run when we become multi-planetary. There won’t be any community helping those on distant planets to fix their issues with spaceships.

I’m certain I wont be around to worry about multi-planetary issues.:exploding_head: But I do agree with kids being taught to think like programmers. :exploding_head: More importantly I think all kids should serve their country, if only for 1 to 2 years . Learn to take orders, discipline, respect, pride, honor. Values IMHO that families now days lack. As a disabled vet and parent, I can say this with truth…somebody needs to get the belt out the closet or programming will be the least of future issues.

My last reply as this is getting off topic (and maybe a bit personal):roll_eyes:

Thanks to all.

diyhaDiy HomeAuto

22h

I think all kids should serve their country, if only for 1 to 2 years . Learn to take orders, discipline, respect, pride, honor.

Thats a great thing you said. We humans are better than other species because we co-operate and collaborate. The concepts like nation, religion, tawn, city, family that we get attached to are mere manifestations of that core-dna. We like our buddies and we sacrifice our own life to save others. The alpha-males in (most) other mammal species don’t do that.
I think its about time we think like species rather than countries, skin colors, regions or religions.

Off topic!!

rlkoshakRich KoshakFoundation member

1

19h

Exactly, I want to attract more of those, but first we need to have a “decent+modern” language, IDE support and tool set and integration with ESH core in place.

I am the Lorax. I speak for the trees, er, the non-technical users. They are my primary, first, and highest priority (on this forum at least). All of my responses and opinions expressed on this forum are from this perspective.

Not that I don’t think it’s important, but I could not care less about any changes or improvements to OH except in so far as they impact the users, in particular the non-technical users. These users consume the majority of my time so it is a position of self preservation.

This might help explain some of the postings I make. I’m very critical when it comes to changes that impacts the end users. I always ask “is this going to make it easier for the not-technical users?” If the answer is no I push back. If the answer is yes I help out as much as I can. If the answer is maybe, I need the proposer to prove to me how it makes things better for the non-technical users k or at least how it doesn’t make things worse.

lipp_markus

38m

HI all, thanks much for this discussion, very interesting.
Just a preamble: I am certainly a tech enthusiast, have limited programming background (stopped around the point when Turbo Pascal incorporated object-oriented programming)…and love home automation when it just does things…i.e., not in my face, it simply works, as one would expect it for any given situation. Now, the last is the tallest of all orders of course. In the context of this discussion, I would just add the following remarks from a user perspective.

Maybe my biggest problems in following along and learning a new skill is a combination of overhead and short-hand writing style. Keep in mind that my day job has nothing to do with computers more than email and MS-Office…and I am not every night fiddling with OH or even thinking about it. Just to make it clear, I understand that somethings are simply difficult, no matter what, and I will not even start talking about date/time issues; while very relevant to home automation, I understand why this has a mind-boggling complexity.

A little more down to earth, for example, the syntax of timers, or lambdas is still a mystery to me…what exactly needs to be before the “|” and what after and why? and do I need to declare a new variable for this, is it a function, etc? And this is despite the efforts of many to explain this to newbies over and over on this forum…I read many of these posts…but, for me there is nothing that makes much sense.

Reading up on kotlin a little (thanks @diyha) intrigued me immediately as it makes intuitively more sense to me than say, Java. Reading design patterns (thanks @rlkoshak, you just will be my hero forever) are critical, as abstract concepts come to life.

At a higher level, I understand that OHs DSL is here to stay for a while…so be it. But at least it will not and does not help me a lot. It is easy to write simply rules in it (but then simple is always easy), but it gets quite more complicated quickly; and the only real source is the forum…but try as I might, I am still missing posts that are relevant to my problems, just because, I am traveling, cannot read every day, etc.

Sometimes I feel, that the biggest hurdle and the biggest challenge is not to write a comprehensive description on how a different ecosystem works; it is translating how this makes sense. For me, one of the entrance hurdles is alway the amount of overhead, while it is boiler-plate and trivial…I am struggling a lot before I am recognizing it as such, and then cannot help the feeling of “why do I need to type this if it is all boilerplate and standard”, because at the end of the day, I still have to learn every line…

So, I guess what I want to say is: computer languages are always logic…duh…and always make perfect sense…double duh…but few are ever intuitive, or have just a too high barrier of entrance to read up on call definition boilerplate language, on syntax that is close to Xtend or Java (because it can use all the libraries), but then not because…what as if in my case, I don’t know Java or Xtend…then I have to read up on both.

This was not a rant: home automation is not easy and I will be forever grateful about the time that all the knowledgable people put in to help guys like me understand better…and there are many, many on this forum…really thank you so much. I learned so much from all of you, and I still have so much to learn.

diyhaDiy HomeAuto

27m

We are getting better, one step at a time, my friend. Thanks for sharing your heart.

diyhaDiy HomeAuto

42m

We are getting better, one step at a time, my friend. Thanks for sharing your heart.

diyhaDiy HomeAuto

rlkoshak

3m

Rich, you fight your battle, I’ll fight mine. :+1:
From my perspective, more devs means more life energy directed to this project. And devs can make a difference. Windows/Apples/Androids are not successful because “users” could program for it, its devs.
No offense, the Xtend and UI based rule engine are both pathetic (in my opinion), one targets devs and other users, they both fail at that. We can’t get hung up on those forever. We need to move on.

Thanks @lipp_markus for taking the time to share your perspective.

Um …
Geez that thread derailed quickly, from smarthome rules to multiplanetary kids, the universe and military
Tl;dr I skipped to the end and i have to agree, xtend rules DSL needs to be thrown into a fire. Use jsr223. Thanks.

Rules DSL, jsr223, xtend, kotlin, etc… from what I’ve read, have there advantages and disadvantages, regarding OH. The idea, as Rich mentioned before is how to help users think like a programmer. Guy’s like you, Rich, diyhaDiy, have this skill and if OH changed, to whatever language, it wouldn’t affect your ability to continue writing your own rules. Everyone else (non-programmers) would just scratch their head and continue asking for help.

Perhaps a tutorial to help users understand programming and writing rules in OH would open an avenue to more advanced languages, cleaner rules, stronger community. Maybe even adopt a few multiplanetary kids.:smile:

@illnesse BTW I like your Hacking BasicUI project. Is there a guide for how to install on an existing OH setup? Keep up the great work!
Thanks

1 Like

Agreed. However, my intention with Kotlin is we create re-usable rules / apps developed by programmers. And also a modern sandbox test framework. They will have their own site-specific config. Rich has a library of patterns, those will actually become part of OH-kotlin extension pack.

Kotlin based Apps, their architectural components(scheduler, timer, async work manager), constraints, user permissions to app, etc. is large topic and needs discussion. Maybe we can get some inspiration from Android community. First 21 mins 40 sec are not relevant to our project.

More pointers are maintained here, in the first post

But we have to keep in mind, Android doesn’t run mission critical systems. We do. And ESH/OH is basically a background OS, so no UI state machines.

It is a hard concept to grasp so don’t be hard on yourself.

Every time you see [ ] in Rules DSL you are looking at a special type of function called a lambda. What makes it special is that it is also an Object. This means we can assign it to a variable (var, val), pass it to other functions as an argument, put them into a collection, etc.

But the key take away is [ | ] defines a function/method (some languages call this a function, some a method, I’m not sure what Rules DSL calls it so I usually use lambda). The Rules DSL is somewhat unique in that the only way to define your own function/method is through a lambda.

Whenever you call a function or method in any language you pass it zero or more arguments. For example createTimer takes two arguments, a DateTime and a lambda. The stuff before the | is defining the names and sometimes the types of the arguments.

So, given

val myLambda = [ String foo, Number bar |
    logInfo("test", "Foo = " + foo)
    bar + 1
]

we have defined a lambda with two arguments, a String named foo and a Number named bar. Inside the body of the function we can reference the arguments and do work with them. In this case we log out foo and return the result of adding 1 to bar.

When we call the lambda using apply, we need to pass in those two arguments:

val result = myLambda.apply("Foo", 5)

result will be set to 6 since the lambda returns the result of 5 + 1.

If we try to call

val result = myLamnbda.apply(5, “Foo”)

we will get an error because the first argument must be a String and the second must be a Number.

Here’s another example:

MyGroup.members.forEach[ i | logInfo("test", i.name + " = " + i.state) ]

In this case we defined an argument to the lambda named i that forEach will call with each element of the list of members. This is one area where there is some magic going on. What type is i? The Rules DSL is usually pretty good at figuring this out which is why we didn’t give it a type. In this case we know members is a List of Items so i will be of type Item. If we know the Group is all Switch Items, and it makes a difference we could use:

MyGroup.members.forEach[ SwitchItem i | logInfo("test", i.name + " = " + i.state) ]

Most of the time it doesn’t matter so you will almost always see the type left off.

Behind the scenes, the forEach is looping through each of the members and calling .apply(member) for each of them.

If there is no argument for the lambda, you don’t need to put anything before the |. You see this in calls to createTimer since rarely do we need to pass anything to the Timer body. That is why you often see

createTimer(now.plusMinutes(1), [ | 
    // do stuff
])

I think that in this case the | may even be optional. But I try to be explicit and consistent and always include the | when I post to the forum because it helps avoid the “it’s magic” feel that beginners can experience when the language allows shortcuts like that (I should probably always supply the type in forEach as well for the same reason).

This is also why I always put the lambda passed into createTimer inside the parens even though the language allows one to put the lambda outside. The following is equivalent to the above:

createTimer(now.plusMinutes(1)) [ |
    // do stuff
]

Now for the after the |. A function/method/lambda needs to do something. The code representing what the lambda does goes between the | and the ]. So when we call val result = myLambda.apply("Foo", 5) the code in the body of the lambda (between the | and the ]) executes with the values passed in as arguments.

I’m not convinced there is any programming language that will solve this problem. Ultimately no matter what the language, writing Rules is programming and once you get beyond a certain level of complexity you will either need to become fluent in whatever programming language you are writing in or resort to StackOverflow and this forum for help.

One can debate whether the Rules DSL was successful (I’m not sure it was), but one of the goals in removing a lot of features from the Xtend language was to minimize how much one needs to learn to become fluent in the language.

One advantage of less esoteric languages like Kotlin, Jython, et. al. have over the Rules DSL though is that even though there is more to learn to become fluent in the language, there are far more resources on the Internet at large to help you learn it. I’m encouraged by the Experimental Rule’s Engine use of JavaScript for this reason.

There is a perennial fight between boiler plate and required structure and terseness in languages. Compared to many languages (e.g. Java) Rules DSL has less boiler plate. But less boiler plate doesn’t mean easier to write, read, or understand. Anyone who has coded in Lisp can tell you that. There are only about a dozen commands in Lisp.

The Rules DSL has less boiler place than many languages and from what I’ve seen it has maybe a little more than Python, about the same as JavaScript, and far less than Java.

And you don’t have to type out most of the Rules DSL boiler plate anyway. If you are using VSCode, kuba added a lot of code snippets (including a lot of the Design Patterns) so you don’t have to type them.

But ultimately a computer programming language is always going to be less intuitive compared to human language because there can be no room for ambiguity. Computers can only do what they are told and they have no common sense nor the ability to handle nuance. All that boiler plate is there to eliminate ambiguity. the computer never has to guess what you mean because there is only one thing you can mean.

The boiler plate may seem pointless but every symbol written in code is in fact a command sent to the computer. Every { is actually telling the computer to do something. So you can’t just skip it (unless you are writing in a language that uses some other way to send that command to the computer, e.g. Python uses a : and indentation).

This is one of the places that the Rules DSL fails. Should Kotlin ever become a Rules programming language it will face the same problem because like the Rules DSL, it will rely upon Java libraries for some features.

Ultimately most of your complaints and problems could be transferred to ANY programming environment. Defining Rules will always be a programming exercise. And if you are not a programmer then you will struggle more than you would otherwise. I don’t think there is any language in the world that will be able to fix that. The developers can try, have tried, and will continue to try to make it better. But we will never get away from the fact that you are programming.

This is why I’m pondering the book discussed earlier in the thread. It doesn’t really belong in the OH docs themselves but it might help users like you get some foundational knowledge that hopefully would help the non-programmer users become successful more quickly, no matter what language Rules end up being written in.

@lipp_markus, I’d love to hear some more specifics and what other parts of the Rules DSL you find confusing, frustrating, or hard to understand. Perhaps I can address some of them here.

If that were the only goal, I would think you would pick a language somewhere in the top 20 TIOBE Index.

Kotlin is 43. Other candidate languages with more popularity (I’m using the August list) include:

  • 4 Python
  • 8 JavaScript
  • 21 Scratch
  • 27 Scala
  • 30 Lua

(Java is number 1).

No. They BOTH target users.

I’ve said it several times. Rules DSL != Xtend. DSL stands for “Domain Specific Language”. It is based on Xtend but the difference are large and Rules DSL is basically a language unto itself.

Currently, the only language in OH that targets Devs is Java. You are working on adding Kotlin and that would be pretty cool for developers. But you’ve made it clear that you are not adding Kotlin as a Rules Engine. Which is fine. But it doesn’t do anything to address the problems or limitation of the “pathetic” Rules DSL or Experimental Rules Engine.

“If you build it they will come” is not a very convincing argument for how it will get better for users like Lipp.

But in OH the “users” have to be devs too. That isn’t the case for Windows/Apple/Android.

That’s the problem that needs to be solved. How do we present a Rules environment that lets non-programmers develop and code their own Rules?

You are presenting a solution to maybe get more developers involved with developing more add-ons to OH but have thus far proposed nothing that solves this problem.

Kotlin doesn’t solve this problem. JSR223 doesn’t solve this problem. At least the “pathetic” Rules DSL and Experimental Rules Engine are trying to solve this problem.

I will. But you might understand why I get annoyed when you tell me to not write anything to help non-coders be successful in Rules DSL (or any other language) because you are working on adding Kotlin, only you are focusing on doing so to make it easier for coders to write add-ons. That has nothing to do with making it easier for users to write Rules.

If everyone who has said this would contribute just one or two articles to the OH docs, maybe that would be possible. Until then this is basically a “go f$%^ yourself” to any user like Lipp.

That is the goal of my idea (minus the kids part). :wink:

Assuming they apply. I’m under no illusion. Many of them are ways to work with the Rules DSL rather than against it. They may not be needed in some other language.

We better not be. OH is not and cannot be a real time system. There is no guarantee in the order of processing of events. There are no transactions. OH is not written to handle these sorts of constraints which, at least in my world, are required to build a mission critical system. Though in my world mission critical means people may die if the system fails.

And EMACS is an operating system masquerading as a text editor. :wink:

8 Likes

Wish I could double-heart your post here. @rlkoshak thanks so very very much. I just simply have to say this here: you are a role-model for me on how to explain stuff to people. I am working in a highly technical and rather abstract job myself and I take cues from your explanations on how to build the narrative for those who are less experienced or simply do not worry on a daily basis about the same things I do. Your explanations always rock!! Thank you!

Just for the record, actually there are quite a few things to like about the rule DSL of OH2. It is one of the more intuitive choices at least compared to what I understood JSR223 and one of the many pythin dialects would entail. I am simply not willing to spend that much time.

My post was not intended as a complaint, but I thought I add some perspective from the uninitiated. I mean, I can help a few people on the forum (I believe), but then I read posts from the experts and I know how much higher up the bar hangs. Just observing myself, I noticed that I am not spending time as regularly with the rule DSL to learn it correctly. In the end, a language is a language, computer language or otherwise, and I am even notice that I am unlearning my mother tongue just because of a lack of use. So the old adage use it or lose it applies to all languages.

Well you answered my issues with the lambdas extremely well, thank you again. The other higher level issue, I am experiencing: I work best in a top-down approach and I have not found a systematic overview of all the methods (or objects) and how they are similar or different in rule DSL. And as the forum seems to grow in activity over the years, I am having more and more trouble retrieving information that I know I read on the forum.

To give a more concrete example MyGroup.members.forEach looks a little like daisy chaining objects or methods (not even sure anymore). I know I can look it up in VSCode, but often it feels like a mushrooming universe and if my (often ill-guided) intuition sends me down the wrong path, it will take a while to find the right approach. You and others have written about this here in the forum, but then retrieval is difficult. But on a positive note, your DPs are just an extremely helpful anchor to orient myself.

Where it gets extremely confusing is everything related to date and time, but I am not sure what would even be helpful. The recent changes with regard to getZonedDateTime and the various DateTime objects in JodaTime and elsewhere, are shall we say…confusing to the uninitiated. Just for an example copied from elsewhere here on the forum:
Number time = dateTimeESH.getZonedDateTime().toInstant().toEpochMilli(); (too much ‘daisy chaining’ of methods that I will need to look up each and everyone…)
But then, I realize that this is something I will just have to accept, as the way we use date and time is rather difficult in programmatic terms…nothing we can do about this.

Maybe a simpler topic of my confusion is the advice/change to not write imports using wildcards. While innocuous in and by itself for my as the casual user, I can remember import java.util.* but without wildcards I am not even sure which libraries are available and which one I would need. Now VSCode gives help, I realize this, and error messages sometimes give pointers too. I still would not know how to look it up and I guess my google skills are bad enough for me not to have found anything helpful.

Hope that gives a few more details. I will gladly read whatever I can, but the more I am trying to express myself, the more I am liking your book idea. :slight_smile: I would gladly pay for it.

3 Likes

Let me respond to that in a greater detail. I do not agree with most of it.

Well @lipp_markus we are very flattered to hear your thoughts. You are already happy with whats there and some people are supporting you, then you shouldn’t be on this thread. Am I right?

Agreed. We purge everything thats already covered by basic Kotlin infrastructure and our extensions to it.

Thats needs to change. We gotta get to a modern language or we die.

Kotlin integrates well with Java libs, I hope more and more new code is written in Konlin, but there is nothing that restricts integration with Java.

Don’t we all use Androids/iOS devices? Didn’t we get our problems solved by dev community? Most of the time, heck, yes. When it didn’t, did we ever complain to those communities, that I couldn’t write a custom app? No.

No, I would pick something that has teams behind it. Kotlin’s got JetBrains and Google Androind community behind it.
And FYI Kotlin is official Android modern language, you can see all presentations on Google IO 2017/18, the code is in Kotlin. And you can verify it through various other posts. The point is is we can leverage Kotlin skills of Android devs.
I am not saying there are other great languages ahead of Kotlin. What matters is how much life energy is behind a given project. We got billions of beautiful programming languages over past 60 years. Doen’t mean we have to embrace them all, forever.

To be continued…

If an environment (Xtend) is pathetic to a dev, imagine how would a user do anything it?

Thats something I agree on. I could point to my second year engineering books that could cover this. I learned those like 15 years back. And I am sure those courses are there in every Compute/IT engineering syllabus.

Thats your current assumption. The current community members are techies, that doesn’t mean we target the platform to techies only. We want this to be used by billions (of users) and developed by millions (of devs).

To be continued…

I didn’t say use it to control nuclear reactor, I meant its more important than your teenage daughter’s phone being hang up.

Don’t be so funny about OH being an OS. Android today is known by many users as OS, they don’t know anything about Linux. OH is an OS and we have to present it so.

TBD…

No problem. I admittedly have a smart five-year-old, but I always try to think about how I would explain something like this to him which forces me to consider what I can rely upon as the base knowledge. Since he’s 5, there isn’t much so I have to work backwards until I find a foundation upon which to build. I’m glad to hear my approach is successful. I always fear I’m coming across as patronizing or pedantic.

I got to where I am by helping people on the forum. It’s the best way to learn. :slight_smile: And it is the best way to keep the skills current. Even if you aren’t working on your own Rules. Most of the Design Patterns came from helping people on the forum. I use maybe half of them in my own setup.

OK, so just about everything in the Rules DSL is an Object. Objects are defined by Classes and sometimes people mistakenly use the two words interchangeably. But think of the Class as the blueprint for a car and the Object as an actual car created using that blueprint.

Objects have data members and methods (I’ll use Java nomenclature, methods are Java’s name for functions).

To access data members or call methods on an Object use ..

So, to put MyGroup.members.forEach into English we have:

Call the forEach method on the members data member of the MyGroup object. You see, MyGroup is an Object. One of it’s data members is called members. This is also an Object and it has a forEach method. And to complete the line, the forEach method takes a lambda as its one argument.

Sometimes splitting a line like that into multiple lines can make it easier to understand

val List<GenericItem> members = MyGroup.members
members.forEach[ i | 
    // some code 
]

Note I added the Type to members. That isn’ t necessary but it helps to make what is going on more clear.

When we daisy chain the calls like in the original what we are doing is calling a method on the result of the call immediately to the left of the .. That saves us some lines and saves us from littering our code with a bunch of variables that only get used once.

But, as you hint at, any time you have any sort of graph the range of valid possibilities grows tremendously. Unfortunately I don’t have a good solution to this. I recommend just following VSCode completions until you find what you are looking for. Alternatively you can look at the JavaDocs for the Classes involved. The ESH javadocs are here. I don’t normally point non-programmers at the JavaDocs because they take some base knowledge to understand. But the JavaDocs show all the data members and all the methods on all the Classes. But browsing around there might help you understand things.

For example, the way to search though them to find out about the line above you would go to GroupItem, find the method “getMembers” on that page, and see that it returns an Object of type Set (oops, I thought it was a List). You should be able to click on the “Set” and it would take you to the JavaDocs for it but they must not have set up their javadocs right.

So the next easiest thing to do is search google for “Java 1.8 Set”. The first hit is the JavaDoc for Set. But actually here is where searching the JavaDocs breaks down because the forEach we are calling comes from the common based with the Xtend language and not from base Java.

And so on and so on. You can see why I don’t recommend digging that deep into the JavaDocs. It can be helpful but it requires a good deal of knowledge of what is going on one layer below the Rules to navigate.

Date and time is one area where Rules DSL fails big time.

I no longer recommend that approach. Instead use:

val joda = new DateTime(dateTimeType.toString)

No more daisy chaining. And once you have a Joda DateTime all the same methods as you have with now are available and you can use it to schedule timers and such. It might be a tad more expensive on the CPU but probably not enough to measure.

See above or a cleaner way. But this one case is a place where cargo cult coding is probably OK. Were I to be able to go back in time, I would have tried to figure out a way to make DateTimeType be a Joda DateTime as well. Then there wouldn’t be the disconnect between the two. But I’m sure there were some technical reasons why that couldn’t be the case and Joda is far to embedded in the Rules DSL right now to willy nilly take it out (i.e. it would break everyone’s existing Rules) so we are kind of stuck.

Why this is the case is because when you use java.util.* it imports that entire library which could be quite sizable and consume a significant amount of RAM. Instead you should only import just what you are using from each library. That has always been a best practice and that really is what Warnings are all about. They are not problems that break your code, they are just violations of best practice. So you could ignore them if you wanted to.

In Rules you have access to:

  • most of the ESH and OH classes but you don’t need imports to use them
  • all of the installed Actions, and again you don’t need to import them
  • I think all of org.joda.time.* and you don’t need to import them
  • everything in java.lang and you don’t need to import them
  • everything in the standard Java library and they need to be imported one by one as used
  • Sometimes you will see imports from xtend for Functions or Procedures, those imports are not required if you define lambdas like I illustrated in my previous reply.

Note, if you don’t explicitly refer to a type you don’t need to import it. For example, MyGroup.members returns a Set<Item> but we don’t need to import java.util.Set to use it. We would only need to do so if we used var Set<GenericItem> members = MyGroup.members

In practice I’ve only seen three imports used commonly:

  • java.util.Map
  • java.util.List
  • java.util.concurrent.locks.ReentrantLock

Rarely I’ll see imports from java.net or java.io when people are making HTTP calls to web services for which the SendHttp*Request Actions are not sufficient or when they want to read/write to files directly from Rules.

I’m convinced it would be helpful. Now to find the time.

I wouldn’t say that. He is clearly struggling and expressing what is causing him problems with writing Rules. Lipp and users like him and those even less experienced are the users for which we need to tailor at least one of the Rules Engines. OH is not an environment strictly for experienced programmers (though that is an accusation constantly thrown at it). It is an environment for home automation enthusiasts. And if we leave users like Lipp behind OH can’t not become successful.

On the other hand, if we neuter OH to such an extent that all the non-technical users have is a bunch of check boxes and selections from a set of pre-written Rules (like pretty much all of the commercially offered hubs) and if that isn’t good enough for you you had better start up OpenCourseware or Corsera classes on programming and learn to become proficient in Java/Kotlin/Python/etc then we are also failing our users.

We need a way to allow users to write their own Rules but in such a way that they don’t have to become fully proficient in a general purpose programming language, which in my experience can take months for some non-technical users.

That isn’t to say there isn’t a place for JSR223 languages or Kotlin in Rules development. Of course there is. We need to support our more advanced users as well . I’m not arguing it’s an either or situation. But we need to have a middle ground between point and click configuration of prebuilt Rules and proficiency in a general purpose programming language or else OH is sunk.

But the fact remains that for users like Lipp, understanding what is Java and what is Kotlin and what the differences between the two are and such is a burden and it raises the barrier to entry when it comes to users writing their own Rules. I don’t see that Kotlin would do anything to address that problem which is what my statement meant. We still have a base language that imports libraries from a slightly different language meaning slightly different ways of using it and different places to look for how to use it.

But every home automation system IS a custom app. No to home automation systems are the same. Each is a custom combination of bindings and Rules and UIs. It sounds like you are proposing an “only experienced developers write Rules and non-technical users can take it or leave it.” That eliminates one of the key differentiators between the open source HA hubs and the commercial hubs, and it is the reason many give for abandoning the commercial hubs.

Saying OH user’s can’t write their own custom app is the same as saying they can’t create a custom home automation system for themselves.

Users want to and need to be able to write their own Rules and we need to have a way (not the only way to write Rules, but a way) to aid the non-technical users so that they can write their own Rules.

And the comparison to Operating Systems like Android and iOS isn’t really that helpful. Those systems are built to allow a myriad of apps to run alongside each other with a few very carefully defined ways for apps to interact with each other.

OH solves a different problem. It interfaces with a myriad of technologies and APIs and presents them to its core in a standardized and normalized manner so it doesn’t matter how different they are. OH is all about making disparate things work together, not along side each other.

I’ll repeat what I said to you in the PM.

  1. Python is what the majority of reverse engineered APIs is written in.
  2. Python is what the majority of web API libraries are written in.
  3. Python is already 50% or more integrated with and capable of becoming a plug-in development language already through JSR223, assuming someone can figure out the distribution mechanism (i.e. download install and configure add-ons through PaperUI/REST API).
  4. If the TIOBE Index is any indication, there are far more Python developers out there right now.

Almost every other language on the list also have teams behind them so I don’t find that a compelling argument. Hell, even Java has a strong team behind it.

Honestly, 1, and 2 are the most compelling. The rest are not all that compelling, though 3 means it would be potentially less work to get to something usable, but they do add weight. I doubt too many Android developers are reverse engineering the RS232 interface to some alarm system in Kotlin but more often then not someone has already written a Python library to do it. That would let OH take advantage of network externalities.

Based on what I’ve seen on this forum, the biggest use of the Exec binding is to call a Python program that interfaces with some obscure piece of hardware. The demand is there. Wouldn’t it be nice if all we had to do was import that library with an OH add-on wrapper?

I don’t have to imagine anything. I have nearly 4 years of experience on this forum (and the Google Groups forum before this one) working with, helping, and building home automation system with non-technical users using the Rules DSL.

I also have some experience trying to teach non-technical users Python and Java and Processing. While I had some success with Processing, the Rules DSL is by far the easier to help non-technical users up to a point where they can start writing their own Rules without help in my personal experience.

No, that’s my conclusion based on my experience with my own home automation and helping users build their own home automation and based on the feedback from users fleeing the more contained commercial hubs that do not have as robust of a Rules development environment.

That’s an assumption not born out by my experience on this forum either. In my experience a sizable percentage of OH users are not techies.

That’s my point exactly. You are presenting a solution amounting to “Add Kotlin for add-on development, let the professional developers create Rules, the rest can take it or leave it.” But since every home automation system is bepoke, that approach only gets people so far. At some point they will run into something they want to do that requires some coding.

Without something between configuring prewritten Rules and full Kotlin (or your generic programming language of choice) development that chasm is too great and same types of people who flee Smart Things, Wink and Vera will flee OH for Home Assistant and the like.

One of the key differentiators of OH is that, despite it’s many flaws, the Rules DSL drastically lowers the barrier to entry for writing custom Rules.

Using the definitions of the words to mean what they commonly and technically mean, OH is absolutely not an OS. It is not a flavor of Linux (like Android). It is not a flavor of BSD like OSX. It does almost none of the tasks that operating systems perform. You cannot run it on its own.

It is a Java application. No more, no less. It’s a pretty modular and capable application, but it is most definitely not an OS.

Now I will be the first to admit it would be awesome if someone where to go through the long term effort to create a software appliance similar to pfSense or OpenMediaVault. That would go a long way to hiding the OS from the users and providing a truly turn-key solution. openHABian takes up the first baby step in that direction. But even openHABian can’t really be considered it’s own Linux distro. It’s just a bunch of script to configure any Debian based distro by installing OH and dependencies and adding some convenience features. But even as a software appliance OH wouldn’t be an OS. OH would still be just an application running on top of an OS.

To say anything otherwise is to redefine the words “operating system” to the point that they no longer have a useful meaning.

2 Likes

I got that emotion covered here on the community already. Integration with Home Assistant. That integration would be much more cheaper than users having to write their own exec binding based configs in OH. I do intend to integrate with Home Assistant. There should be no one master system for a diy enthusiast. They should co-exist.

TBD…

I strongly disagree with that. An OS is a set of tools, programmer APIs, coding practices, components, libs, and “lastly” a kernel, => targeted to solve problems of given domain in efficient ways. Android, even though Linux, solves a problem very nicely and competently that Linux/Ubuntu themselves couldn’t solve or “imagine” for years and still can’t do.

OH is an OS. If we target it as app on other platforms then we are doomed to failure.

We too do not have any robust rule programming environment to offer as of yet. If you are saying so, then its false advertising.
The personal support you give to some users cannot be counted as strong evolutionary factor here, even if there are 10 like you. Things (Xtend) will die if they are not upto the task. I am not saying they have to be killed explicitly. :slight_smile:

Farming out “icky” Python to some other home automation hub feels a whole lot like just giving up to me. Just because it CAN be done through Home Assistant doesn’t mean that OH shouldn’t bother.

I’ve personally no interest in writing some add-on for some other home automation service when I can use the Exec binding to run some Python code. A whole lot less complexity and a whole lot less overhead. But it would be so much better to be able to run it natively as an OH binding.

For example, awhile back I had an RFM69HW network that I was running some sensors off of. It would have been nice to have a binding to interface with that transceiver but all the libraries were written in Python. So I wrote my own external service that reported the sensor readings to OH over MQTT. If OH had Python binding support I could have created a binding and the whole community could have benefited and it would have taken me hours since the library is already written.

If it came down to the point where I had no other choice I’d end up just switching to Home Assistant before I would run them both. The HA integration would certainly easy the transition though. Each hub has enough complexity to handle on their own. Needing to understand both just doesn’t provide value to me. It increases work. And I doubt I’m unique in this opinion.

And it does nothing to help make OH better.

An operating system mediates among application programs, utilities, and users, on the one hand, and computer systems hardware on the other. " Operating Systems: Internals and Design Principles Fifth Edition Stallings

Technopedia

An operating system (OS), in its most general sense, is software that allows a user to run other applications on a computing device. While it is possible for a software application to interface directly with hardware, the vast majority of applications are written for an OS, which allows them to take advantage of common libraries and not worry about specific hardware details.

The operating system manages a computer’s hardware resources, including:

  • Input devices such as a keyboard and mouse
  • Output devices such as display monitors, printers and scanners
  • Network devices such as modems, routers and network connections
  • Storage devices such as internal and external drives

The OS also provides services to facilitate the efficient execution and management of, and memory allocations for, any additional installed software application programs.

Wikipedia

An operating system ( OS ) is system software that manages computer hardware and software resources and provides common services for computer programs.

For hardware functions such as input and output and memory allocation, the operating system acts as an intermediary between programs and the computer hardware,[1][2] although the application code is usually executed directly by the hardware and frequently makes system calls to an OS function or is interrupted by it. Operating systems are found on many devices that contain a computer – from cellular phones and video game consoles to web servers and supercomputers.

Webster

software that controls the operation of a computer and directs the processing of programs (as by assigning storage space in memory and controlling input and output functions)

The common thread is that the operating system mediates between the software running on the system and the hardware that makes up the computer. OH does not directly talk to ANY hardware directly. It talks to the underlying operating system which mediates access to the hardware on its behalf.

Just because a program supports plug-ins does not make it an operating system.

By your definition pretty much any application that supports plug-ins is an operating system. Is Firefox an OS? Is MS Word an OS? Is Eclipse an OS? Is Apache and OS? All of them meet your definition. None of them would be considered an operating system in any computer science or engineering college I’m aware of nor are they considered OSs by the common populace.

It’s not a matter of targeting. It’s a matter of what the words mean. There is no commonly accepted definition of OS that would allow OH to be called an OS. And I’m not up for redefining the words. No one listens to me anyway.

But its just semantics really. And it really is besides the point. No matter what you call it OH is what it is.

At this point its not me you need to be arguing with or convincing. Its the developers and maintainers of the core. Even if I put my foot down and say “absolutely not” it doesn’t mean anything. I’m just a user. The real decisions about what is and is not acceptable will be decided by them. My role in discussions like these is to represent the non-technical users. I think I’ve said my peace and been understood.

Kai is BDFL for OH so it’s him you need to convince. I’m not sure what the governing structure of ESH is (Kai is not BDFL there) so I don’t know who you need to convince over there. If it were me I would open up issues over at ESH and start trying to convince those maintainers that what you are proposing is where they want ESH to go. If it’s something that isn’t too disruptive to what they are working on or working towards I’ve no doubt it will be well received. If it is a “throw away what your doing and lets rebuild in Kotlin” I doubt you will get very far.

Short term I still think you will get farther faster and have something of more utility to OH over all with Python but but like I said, I’m just a user.

And of course forking and build your own new HA is always an option, though certainly not one I would encourage.

There’s that word “robust.” I don’t want a robust programming environment for the non-technical users.

I want a simple programming environment.

I want a programming environment that holds the user’s hands as they take their baby steps.

I want a programming environment that helps the non-technical users learn as they go.

I want a programming environment that gets the non-technical users productively writing rules as quickly as possible.

For all of its faults, the Rules DSL meets those requirements better than any of the general purpose solutions proposed thus far. And the Experimental Rules Engine meets those requirements even better.

I don’t want YOU to have to program in either of these environments. I don’t want people writing bindings in either of these environments. I want there to be options like Kotlin and JSR223 for users like you and me who don’t have to learn programming from the ground up.

But there has to be something to support those users who need more than point and click Rules configuration and “go learn Kotlin”.

Everyone wants to kill the Rules DSL. Fine. Bring me an alternative that is as simple to teach and learn for non-technical users and I’ll jump right on that band wagon. Or commit to writing all the docs necessary and provide the support on this forum to help the non-technical users come up to the bare minimum to write a Rule.

So far every proposal I’ve seen (not just from you, this topic comes up several times a year) only scratches the developer’s itches, never the non-technical user’s itches.

I’m fine with improvements to the developer experience. I encourage them. I’m not fine with improving the developer’s experience at the experience of the non-technical user’s experience. If someone can come up with a simplified subset of these general purpose languages that minimizes how much of the language the users have to learn of the language to start programming Rules I could be convinced. But large teams and companies behind a language and IDEs and libraries and stuff just turns into more that the non-technical users have to learn before they cna write Rules on their own.

Everyone thinks I’m in love with Xtend. I’m not. It’s a horrible language in many ways. But the Rules DSL IS NOT Xtend. And in the OH context, the Rules DSL comes closer to meeting my requirements above than anything I’ve seen proposed thus far. Until something as good or better gets proposed, I will defend the Rules DSL.

And let’s be honest, it shouldn’t be all that hard to create something as good or better. Heck, chris did it awhile back with a Scratch like interface Rules builder in Habmin. Sadly he hasn’t had time to keep it up and it doesn’t work for OH 2. The bar is pretty low.

But so far every proposal except for the Experimental Rules Engine increases the amount the non-technical users need to learn, not reduces. The arguments in the past was users are coding anyway so let’s just throw them in the deep end to teach them to swim. Those that survive will be stronger at Rules development in the end.

Unless I’m misunderstanding your argument, you are arguing that non-technical users should not be coding Rules at all and should just be configuring parameters. If the existing parameters don’t work then they need to go learn to become a programmer. As I’ve said, that destroys one of the key differentiators of OH.

And I know fully well that the Rules DSL is just coasting along until something better gets implemented. I’ll be as happy as anyone when we can retire the Rules DSL for good. But not if it leaves behind whole groups of users.

Yes, thats exactly OH asks its users “now”. I am not putting any additional burden on dev or user. They will both get benefited by Kotlin IDE support, its integration with ESH and test framework and much more in terms of architectural components that allow writing re-usable apps and rules rather than just site-specific rules that current rule DSL allows.
How may of you had timer and scheduling issues and discussions? That could be solved nicely in Kotlin and Android way of looking at Kotlin integration. Given you guys are even interested to listen.
Otherwise I can direct my energy to something else, perhaps Android Kotlin community itself.
If everything is in perfect shape then I wouldn’t have anything to offer to the community anymore. I must leave.

I’ve said multiple times. You don’t have to convince me. You have to convince the maintainers. Especially if you are proposing changing programming languages and core architecture.

As a user and self proclaimed speaker for new users I’m not convinced. It seems a little like 1. Kotlin 2. ? 3. Profit.

But my opinion counts for next to nothing for something like this. I’ve given it. You disagree. That’s fine. If you feel strongly about it, go and open the issue on ESH and get buy in from the people whose opinion actually matters.

Or move on to something else. Something this big and daring is going to take a lot of work and a lot of cooperation from a lot of people, many of whole have strong opinions about OH and where it is going. An add-on you can do on your own. be Fundamental changes but to how OH is built and how it works is going to take the work of many. If you are dissuaded by the opinion of two users who have no influence over the development of OH in the first place then I question whether you would have the commitment and fortitude to see this through to the end.

You have managed to tell several of the developers of OH that their baby is ugly in this thread. Can you stand up to similar criticism from them?

No, I don’t think everything is fine. I’m also not dissuading you from trying. But I’m not yet convinced this would be a viable replacement for the Rules DSL. I could be wrong. And if I am I’ll readily admit it.