Openhabian on Pi 4 running openHAB 3.2.0 Build #2454
Just finished reading through various comments and docs on accessing personal modules from scripts … and still wondering if I am going down the right path …
quick background on what I am doing:
All scripts are written in jython
I have a room script for each room in the house where rules for that room are defined
I also have control scripts for things like TimeOfDay changes, etc.
So wondering about a good approach for say, “turning off lights in the MasterBedRoom”. The MasterBedRoom.py script will contain the functions and rules but now I want to execute the control function from say TimeOfDayModes.py … other than putting the room.py scripts into lib/personal I dont see a way to access the room.py control function.
I am also looking at using metadata to control a generic room module in which case that would be a candidate to be in lib/personal … but for now …
So, an alternative (which is where I am heading), is to create a String Item called MasterBedRoom_Control, and now in my MasterBedRoom.py script I can have a rule which can be accessed from any other script through events.sendCommand()
Is there another approach for this particular problem that I am missing?
That’s correct. Libraries need to go into the automation/lib/python set of folders. automation/lib/python/personal is where you normally put your own libraries. Then you need to import them the functions/classes into your rule’s .py files like any other library, e.g. the Helper Libraries.
The terminology is “scripts” and “modules”. “Scripts” are rules and go in automation/jsr223/python. Modules are libraries and go in automation/lib/python. Modules can not be executed on their own. They must be imported into a script and run from a script.
You cannot put a Rule into a module (with some caveats because you can create rules from a library function, but it takes a script to load and execute that module).
What is the concern about putting those functions into a module? That is how it’s supposed to work after all. Common code like that is usually put into a module so it can be used across your scripts. If it’s just about having access to events you can:
pass events as an argument to the function
import events by including the following at the top of your module
from core.jsr223.scope import events
Other alternative approaches:
Put all the rules that depend on that function into the same .py file and declare the function in that file.
// Run another rule
var FrameworkUtil = Java.type("org.osgi.framework.FrameworkUtil");
var _bundle = FrameworkUtil.getBundle(scriptExtension.class);
var bundle_context = _bundle.getBundleContext()
var classname = "org.openhab.core.automation.RuleManager"
var RuleManager_Ref = bundle_context.getServiceReference(classname);
var RuleManager = bundle_context.getService(RuleManager_Ref);
var map = new java.util.HashMap();
map.put("test_data", "Passed data to called function!")
RuleManager.runNow("tocall", true, map); // second argument is whether to consider the conditions, third is a Map<String, Object> (way to pass data)
But you need to know or search for the rule ID ahead of time. And if I recall the UIDs for .py rules are randomly generated. So you’ll have to tag the rule and search the RuleRegistry for the right UID.
But for what you have described, the best approach is to use a module. That’s what they are there for.
Don’t disagree but having a MasterBedRoom function in a library module didn’t really strike me as a good approach at first … I suppose having a “control module” in libs/personal that handles any control functions for the various rooms would be fine … then any of the rules can access those from the scripts. The downside of my approach of creating a String Item and using sendCommand is overhead, which probably doesn’t matter that much.
I’ll mull it around as I play around Thanks Rich for talking this through!
In my OH rules setup, I have a notification script which takes care to use the “best available channels in a room” (eg OSD in Kodi, push to phone, TTS speech, colored light bulb…) to give the appropriate feedback to users.
With DSL-rules, I used to trigger the notification by sending a command to a String item which received the several parameters of the command (message category, message text, priority, notification method, destination room,…) separated by semi-colon:
That import is importing something that is part of the Helper Library so if it’s not working look to see if something has changed in the Helper Libraries concerning events. If you don’t have the Helper Libraries installed that’s the root cause of the problem.
NOTE: You can’t import a module written in one language into a script/rule written in another. So since your module is in Jython your code that imports it also must be Jython.
Yes, I have the Helper Libraries installed. But the setup was so painful: it took me days to get it working in OH 2 and weeks to get it back in OH3! Best seems not to touch anything. I will keep the events as a parameter of my function. This is ugly, but ok…
I am definitely exhausted to spend 90% of my time to troubleshoot that clumsy JSR scripting language (not really Python, no longer Java) instead of working on improving my home automation scripts! I am resigned to accept that is the downside of the robustness and versatility of OH… but sometimes it drives me totally crazy
It did not appear clearly in my post, but of course all scripts were migrated from DSL to Jython
Since OH 3 came out, for a number of reasons that don’t have anything to do with the usability or quality of them, I’ve not been using the Helper Libraries. I’ve also moved all my rules to the UI.
My observations are:
The need for the Helper Library is greatly reduced when writing rules in the UI because you don’t have to define the rule or triggers in code at all. You just need to write the function.
The opportunity to install and use rule templates will hopefully eliminate most of the complexity of rule writing entirely for a lot of cases. See Rule Templates - openHAB Community for a growing collection of rules you can just install and configure like an add-on through MainUI.
There is ongoing work to make sure the helper libraries that make writing rules reasonable be a part of the automation add-on itself.
If you don’t want to fight between what’s Python and what’s Java, there are two options for writing rules in pure Java available on the marketplace. JavaRule Java Scripting (SmartHome/J version) (works to write rules in Java in the UI, should be included in the next release of OH according to the comments) and JRule - openHAB Rules using Java (has a few more limitations including Unix only and can’t be used in the UI).
Thank you for your feedback.
We already discussed it: UI does not match my needs. It would be great if OH scripting could stick to one core and robust scripting language but your writing tends to confirm the opposite.
Don’t tell me that after my efforts to move from DSL to Jython I will have to move again to JavaRule/JRule in the future
Well, Jython is kind of dead, not because of OH but because the upstream project that implements it is just short of abandoned. It implements Python 2.7 which has been end of life since January of 2020 and the Python 3 implementation of Jython is nowhere in sight.
In all likelihood I’d expect to see the Jython add-on no longer being supported in OH 4. But I have no say in that decision, it’s just a prediction.
It’s possible someone might implement a GraalVM Python add-on which would be great! But again, it won’t be a straight forward conversion because that will necessarily be Python 3 and there are some significant differences between the two. But, if they follow the approach being developed for GraalVM JS one won’t need to go through all the pain to get the Helper Libraries.
HABAp is also a good choice for Python users should a GraalVM Python add-on not appear or it doesn’t work as one wants. But that’s a rules engine that runs outside of OH and isn’t really a part of the OH project. It’s also limited in that it can only do things that can be done through OH’s REST API.
From where I sit, the only way to ride out these upcoming changes with the least amount of rework will be to take advantage of rule templates from the marketplace as much as possible and by through UI rules where the impacts of the changes will be much less (though it’s going to be ugly for Jython users no matter what). If the advantages don’t meet your needs, well that’s your choice but at least you know the cost of that choice. There is going to be a huge bit of pain coming up in the next couple of years.
Since you complained about dealing with the fact that all your interactions with OH are using Java Object and Classes, choosing one of the Java add-ons is going to be pretty much the only choice that is going completely solve that issue. That or moving to HABAp which is pure Python 3. At the end of the day, openHAB is written in Java. Rules, no matter the language, are running on Java. So Java is going to be involved no matter what and if you are using a rules language other than Java you’ll have to deal with the differences.
If there were a candidate for one core scripting language right now, it would be GraalVM JS. A lot of work is being done with it to make it easier to use, make the helper libraries associated with it become built in. It will be the language backing the Blockly implementation I think and it will likely be included by default. Given that it will be included by default, that means most of the Rule Templates will likely be written in it as well. But GraalVM JS isn’t quite ready for prime time.
Hi. Just tried it and it keeps telling me events is not defined.
But don’t bother any longer. From what I last read from @rlkoshak about moving back again to a Java-like language soon, it confirms me the endless back-and forth direction of openHAB in term of scripting backend.
I feel like a pilot who wants to improve his driving to achieve the best performance on a track and who can only discuss with people putting all their energy to redesign again and again the technical systems of the car he is driving.
When both processes are conducted unsynchronized and undocumented, it just does not work, except if the pilot lot is also a specialist in car engineering.
My 15 years in developing custom manufacturing and supply chain softwares for industry are clearly not a sufficient background to cope with the top-of-the-art moving complexity on which openHAB relies on. I had in mind to contribute to the emergence of missing home automation architectures and models by joining the OH community. OH has for sure the most robust architecture to connect and control the hardware but probably the weakest scripting model, unable to provide neither the bricks to implement easily common behaviors for home automation, or to create custom and reusable logical scripting components from templates instead of relying on an outdated and poor if-then-else scripting model.
Some members like @rlkoshak are making a lot to promote and document scripting practices and provide a helpful support. And the OH clearly uses the most compatible and versatile architecture.
But from my point of view, development of home automation should be house and user-driven, starting from the standard functions one can expect to support homes: rooms, presence and access management, users notification (TTS, OSD, light, network, push…), lighting and appliance control (groups and scenarios), energy management (consumption and performance), door /shutter control, multimedia interfacing, air/heating control, watering, home appliance control, security (alarms, users and system safety, monitoring/audit of systems), mobile and cloud interoperability (apps, links with calendars, contacts and mobile devices) and privacy management (how the data collected by persistence is accessed by individuals respecting their privacy) and interfaces used to interact with the system (apps, voice, switches, sensors, panels…).
Would these standard bricks exist (even in a basic fashion), the setup of OH would become straight forward and user friendly and teams could work in refining and improving the functionalities of these bricks to cover most common use cases. And the bricks could be freely customized and reused by users.
Does anyone seriously believe that home automation enthusiasts and users need to understand concepts like persistence, programming contexts, or classes to organize their home? What makes software robust and popular is to have both a robust and scalable backend model and to hide the complexity to the user.
I have decided to stop wasting time in troubleshooting the scripts at every update of OH as I have been doing for the last 5 years while waiting for a stable solution from OH: initially expected for OH1.8, then promised for OH2, OH2.2, OH2.4, and announced as a breakthrough in OH3 but quick upgrade to 3.1 as JSR did no longer work… Come on… .
My plan is now to take my scripting model out of openHAB and make it run on any stable platform (Python, Basic C, or any old fashioned but robust programming layer) with a simple communication link (Mqtt or API) with OH which will handle the hardware model (things / model / items).
Would I have more time left and better programming skills, I would get involved to promote the architecture discussed. Unfortunately I just have enough neurons to maintain my home system but I would love to see such a scripting architecture enter OH development pipe.
But’s that’s all about laying choices before you, not project wavering in direction.
There are a number of statements here that need clarification.
That is not what I said. You complained that it was challenging to deal with the problems that arise when using the mix of Java and Python in rules. Because OH is written in Java, the only way to avoid that is to write your rules in Java, or use some external rules engine that interacts with OH through the REST API.
I fully expect the two Java rule automation add-ons to remain relatively niche. But I suggested it because it solves one of the major problems you complained about.
But if one is looking for stable and always supported Rules DSL has always been there and as long as the upstream Xtend project is maintained it will always remain.
In addition, the problem caused by the rules languages alone are not very useful and require some helper libraries to be installed is also being addressed but including the helper library in the add-on. So that means no more need to clone git repos and copying files around.
This is a challenging problem that is highly hardware dependent. But see Generic Presence Detection which uses the Debounce rule template which you can just install and use, no coding required.
OH 3 support any number of users which can be one of two different roles: user or admin. Furthermore, while not a security mechanism, widgets on the UI can be hidden from users based on their role which can do a little to approach access control.
It depends on what you mean but openHAB is a self hosted system that stores the persistence data locally. As the administrator of the data you necessarily have access to all the data stored on your system. openHAB has no control over privacy concerns for data collected by cloud services it might integrate with but you have the choice not to use those services.
Note, everything I linked to above: the add-ons, the rule templates, and the UI widgets are all installable through MainUI and require no manual editing of any code to use. And I did not provide an exhaustive list.
My point is:
OH does already have the concept of standard bricks one can just plug together without code
there already is a library of such bricks
the library is only going to grow larger.
All of these are also customizable by users, especially the UI Widgets and the rule templates. Once added the user can modify the code to their heart’s content to meet their needs.
If they are happy with the simpler capabilities offered by commercial offerings like Alexa, HomeKit, SmartThings or Google then no they don’t. However, if they want to flexibility to do exactly what they want the way they want to then yes, they do. At some point customization becomes coding and you need to understand that stuff to code.
The amount of complexity that OH already hides from the user is huge! But as is often the case with these sorts of posts, OH doesn’t get credit for the stuff it already does for the user nor for the stuff that is in work to help the user. It only gets dinged because it’s not enough.
Rules DSL was the stable solution for OH 1.8. JSR223 was niche at best used by only a handful of users.
There were breaking changes to all the rules scripts in OH 3. But the breaking changes were relatively minor for Rules DSL which continues to remain the stable default language for rules development. If you want stable, Rules DSL is where it’s at.
I know that OH is Java so that Java should obviously be preferred. So why not sticking to developing one single scripting model with Java instead of providing plenty of languages? I started with DSL and the XTend stuff. I coded all in DSL and it was a real pain for me as I had no Java background. Facing debugging issues, I was encouraged to migrate to JSR223 (remember the nextGen engine speech) so I hardly moved all to JSR223 and now everyone says JSR223 is dead arguing the EOL of Python2 that was already there in 2008. So if OH scripting is not going back and forth, at least we can say the plan is quite erratic.
I had not seen it and it is a good start.
Here you are giving the [great] list of bindings available in OH to interact with the hardware. But we miss bricks to model standard behaviors between them, built over them. The semantic model has the abstraction level of the bricks I am talking about even if it currently misses a packaging to make development user-friendly
To clarify, let me list the expected behaviors of each brick that one would expect to find to have a “ready to deploy” user experience:
attach devices to rooms,
map rooms to map the house (and use it in the UIs),
locate users based on available technologies (wallswitches, cameras, sensors, wearables, BLE and network positioning, appliance usage, predictive users displacements),
provide functionalities per room that can be reused by other bricks.
For example, is there a TTS device available, multimedia devices (music, TV, radio), dimming or scenario lighting, sensors, cameras… So other bricks can interact with the rooms in a standard way.
Access management (the tricky topic):
methods to locate user (mainly GPS and networks detection, RFID),
standard rules to welcome users (when far, in approach, arriving and at home), method to welcome users (what are the “access doors” that trigger welcoming),
types of welcoming per user profile (supervisor of the system does not get the same welcome information as children),
content of welcoming (data sources, topics, preferences).
Presence management: at home, nightly mode, absence and associated scenarios.
Scenarios when leaving home (turns lights, close doors…).
Handling guest access (concierge and guest at home (access to networks, allowed functions, tutorials for guest at home).
Adaptative behaviour at home: for example, when at home alone, the system could push mobile notifications to home notification system (see below).
Expose available notification channels in a room,
Take into location of the users, and number of users at home (or outside),
Take into account ongoing activity (for example watching a movie)
Connect with cloud brick to get reminders from calendars, Google lists, Alexa tasks and many more, and
Determines automatically the smartest method to notify users
Support light blincking or color, TTS, OSD to Kodi, toast to PC, multimedia features (TV, audio), push to mobile, email, alarm bells,
Prioritize and plan notifications to make them smart.
Light actuators or with timelined lighting scenarios,
Automatic lighting (when door open, at night),
Autoturning off light,
Expose color bulbs available to the notifications brick and provide interface for notification brick
Energy management: the Energy Meter binding measures consumption. This is a start, but the brick should be capable of optimizing energy with following behaviors :
handle opening/closing of blinds depending on sun exposure, heat transfer of windows or any other relevant parameter
improve start/stop of heating and cooling devices based on data collected by sensors (temperature slopes).
Handle groups of devices that are turned off automatically depending on user presence reported by the access management brick.
Links and control audio/video devices all together to provide a simple control over the systems (no more remote like at home)
Expose only activities available per room and have OH to interact with them (multiroom, TTS, OSD, centralize multimedia library)
Integrate and interoperate heating control systems (Tado, Nest…).
For example, I could use Tado to control the boiler and combine it with a home-made controller (sensors + valves) to regulate some rooms. At the end, this brick would both integrate the “standard commercial systems” and custom heating control brick and expose this in standard way.
Provide venting regulation algorithms for humid rooms and algorithms for monitoring heating
Interface with energy management brick
scheduling of watering, zones,
suspend watering when in the garden, optimize watering hours to reduce evaporation
Adapt watering quantity to weather conditions and forecast
Suspend robot-mower when watering
Home appliance control:
t* his brick can handle plenty of functionalities. To name most common:
notify when washing-machine has finished (either with a Wifi smart appliance or with a power consumption plug,
notify when fridge doors were left open,
notify when oven has finished.
Start devices on schedule : for example coffee machine right before the wake-up alarm, start vacuum cleaner robot when not at home
alarm covers access alarm, fire alarm (based on available sensors), flooding alarms or any type of alarm depending on sensors installed.
take into accound user presence to adapt automatically the alarm status (using the access brick to authentify users),
notify users using the notification brick
Mobile and cloud interoperability:
grab, use and expose user data (calendars, contacts, lists, weather…) at the right moment and at the right place.
For example, remind me the shopping list, the traffic warnings and trash schedule when I am leaving home, use my contacts to push a phone call to the OSD using the notification brick…
Privacy: you are talking about privacy of OH user. I am talking about a logic to ensure privacy control over the house.
Should I have access to cameras currently playing media from outside and track what people at home are doing? How is this controlled (notify user at home to grant access), allow a “ghost mode”.
I am currently shocked to be able to locate my mate using the location screen in Tado or to access the listening history in Alexa. Privacy is a difficult but important topic.
a brick to suggest the best UIs scenarios and methods to interact with the house : giant touchscreen were great in Minority Report or in Iron Man but are an ergonomical nonsense.
Voice control is convenient for light control or device control. Extend wall-switches usage, user-friendly panels or apps.
Action-less interfaces: how to get rid of interfaces to improve ergonomy and have the system anticipate your needs based on the context
As you can see, my concern is not about the bindings and features in OH: they are great, and allow integrating a wide range of hardware.
My point is OH to provide several “standard arrangements of bindings” under the form of a frameworks or bricks to to serve home the automation behaviors listed above. With such a logic, unskilled people can find howto’s guidelines to setup the functionality they need and reduce the need for scripting. The bricks should interfaced in a standard way also which would expose the “features provided by the brick” (scenarios), the inputs/outputs expected by the brick to work (connectors, kind of meta-bindings), and the possible interactions between bricks
Today, we are all writing and rewriting similar scripting logic in DSL or JSR. The bricks framework would provide the behaviors already coded in an existing scripting language and if possible . And anyone could choose either to deploy and use the bricks features out of the box or to customize the brick 's code to implement different behaviors.
Because developers want to be able to code rules in the language of their choice. So they volunteer their time and efforts to make openHAB support their language of choice. Should the project say “no, we don’t want your free volunteer work because we want to keep openHAB ‘pure’” or should we say “thank you for your contributions!”?
JSR223 is already dead in openHAB. JSR223 was an old approach that allowed the JVM to run other scripting languages like Python and embed those languages into a Java program. It has long since been replaced by a Java Scripting interface which in turn has been replaced by GraalVM (if I understand the history correctly).
Jython was ported from JSR223 to Java Scripting. The upstream Jython project hasn’t ported to GraalVM. Furthermore, development has all but stopped on the Jython project and there is no Python 3 version of Jython in sight. Do you want to volunteer to go take over the Jython project and get ti going?
We backed the wrong horse. In 2008 it looked very promising as development was very active and it looked well supported and maintained. Now no one is supporting it and it’s not part of the openHAB project so we don’t have power over that. But there was twelve years of good use of Jython. I’d hardly say that openHAB was wrong to support it way back then. There was every indication it would continue to be supported and a Python 3 version would be developed.
They sound like good ideas. Do you have plans to work on implementing something like this? I’m not being glib here. openHAB is a 100% volunteer effort. Having a good idea is almost never enough to actually get something to happen in openHAB. Someone has to volunteer their time and effort to build it. You are unhappy with what the volunteers have built so far and have good ideas of your own. But if you are not willing to implement it they are not worth much.
Much of the foundation for these “room” ideas is already there. It’s one of the things that kind of drove the creation of the semantic model in the first place. In the model you attach devices to rooms. Those rooms to map to the house and are used to generate a UI. Locating users is hard but if you have a solution that can implement this idea I’m sure we are all ears. And one can write rules (or even better install rule templates) to provide functionalities per room. And rule templates can be reused. And all interactions in OH occur in a standard way through Items.
From what I can see, OH already implements 90% of what you are asking for in terms of rooms. But you don’t like how it implements it, which is fair and fine. But if the current set of developers haven’t implemented it the way you like so far, what makes you think they will in the future? That’s why anyone with a good idea like this needs to back it up with some effort too. Otherwise it’s just complaining.
I can go through the rest but ultimately it’s the same thing. Good ideas. Much of the infrastructure to build it is there already. Some would argue much of what you are asking for is already implemented. So, who is going to implement the rest?
And don’t forget to answer the question: “what if I want to do it another way?” and “What if I don’t want to do it that way?”
Ultimately this whole list looks like a great one for people to use to start implementing rule templates. A number of them already are. It’s kind of the whole point of rule templates really. To eliminate the need for users to have to copy/paste/edit code or come up with scripts on their own. Notice how among those links that I posted in my previous reply I included links to a number of rule templates and UI widgets?
Those two rule templates and UI widget allows one to set a bunch of devices how they want them, capture their current states and save that to a “scene” that can be restored at any time. No coding is required. Users just need to install the templates and UI widgets and instantiate them.
Right now the marketplace is in it’s infancy and there are limitations but there is talk of being able to bundle bindings, UI widgets, and rule templates into one installable add-on so, for example, you don’t only get the ability to create Things and Item that interact with your HVAC system, but you also get a nice UI widget to control it and some standard rules that implement behaviors for it. That sounds a whole lot like one of your bricks to me.
Yes : OH implements 90% of what is needed to build a great home automation system but let users set that up the way they want. Providing a skeleton or some generic templates of the features (my bricks) would avoid people recreating the existing and help refining and improving the skeleton through users expectations and ideas. I am not suggesting to organize the bricks the way I proposed, I am just saying it would be nice and helpful to define a set of bricks that we could then develop by contribution of developers and requirements.
[quote=“rlkoshak, post:17, topic:125231”]But if the current set of developers haven’t implemented it the way you like so far, what makes you think they will in the future? That’s why anyone with a good idea like this needs to back it up with some effort too. Otherwise it’s just complaining.
It should be implemented ‘the way the community wants’. I mean: let’s agree on some common basic functional behaviors and everyone keeps free to customize and develop them. And at each iteration we decide if those extension are worth extending the initial brick based on their popularity.
The question is not to say that my ideas are the good ones. The question is what development process we can set up over the community and put in OH a set of common bricks that can be used to deploy at not cost “basic functionality” and letting the possibility to customize or simply replace these basic bricks by custom ones. But that should be seen as part of the architecture of OH.
Everyone remains free of doing the way he wants. But the content bricks are the “common consensus” at the time of release of what the community believes to be the “right way” to achieve one functionality.
And in a certain way, it provides a summary of improvements made by the community on scripting and also a howto base for new coming OH users.
The biggest thing in my reply I think was missed though. What is it about the Marketplace and rule templates that cannot serve this purpose?
That’s literally what rule templates are, a way to distribute functionality that one can just use our can customize as desired. A use need never even look at code to deploy quite complicated capabilities.
There are actions one can use from a rule to determine the position of an item in the semantic model (what equipment, what location, what property it represents, etc.).
I really do think the infrastructure for what you are suggesting is there. We just need people to start writing and publishing rule templates.
Right now though creating rule templates is pretty primative and manual but there is lots of work ongoing to make the creation of rule templates better and easier.
I’ve written several relatively basic templates already to do things like scheduling the execution of a rule based on the state of a DateTime Item, Time of Day, MQTT Event Bus, Item state Denbounce, alerting when items violate a threshold, and alerting when Things go offline or online.
Theses are just basic building blocks because that’s where my interest and skills lie. But I expect over time everything in your lists to become rule templates.
I agree, we need some common consensus to move forward. The semantic model is a great start. We can’t always revert to OH default that we should allow anyone to do anything any way they want. We can’t make HA easier unless there are shared common ideas. In reality, Amazon (Echo), Apple (Homekit) are driving this for everyone.