Groovy scripts: how to import another groovy file?

As my DSL rules compilation time got untolerably slow, I started to migrate my rules to JSR223 using Groovy.
I got one rule file migrated and working, but now I want to move reusable code to a separate groovy file which I can then import in multiple groovy files.
But I can’t seem to get it to work…

Here’s a very simple example… file AA.groovy:

class AA {

}

… which succeedes with

2020-09-23 21:38:25.287 [INFO ] [me.core.service.AbstractWatchService] - Loading script 'AA.groovy'

2020-09-23 21:38:25.293 [DEBUG] [ipt.internal.ScriptEngineManagerImpl] - Added ScriptEngine for language 'groovy' with identifier: file:/etc/openhab2/automation/jsr223/AA.groovy

and another file BB.groovy

import AA

…which results in the following error:

2020-09-23 21:38:56.338 [ERROR] [ipt.internal.ScriptEngineManagerImpl] - Error during evaluation of script 'file:/etc/openhab2/automation/jsr223/BB.groovy': org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:

Script66.groovy: 1: unable to resolve class AA

@ line 1, column 1.

   import AA

   ^

1 error

My expectation is that both files get compiled at runtime, one is on the classpath of the other (or, should order matter, at least AA is on the classpath of BB and can be found (in the default package)).

Can someone help me out? Should such a simple task be really that hard?

What steps did you follow for setting up Groovy? There are only a very few people using Groovy with OH, and the helper libraries are basically non-existent, so you will need to use the raw API for everything. This can be extremely difficult, but you know some Python, you can figure some things out by looking through the Jython documentation and helper libraries. There is at least some instruction for the installation of Groovy…

https://openhab-scripters.github.io/openhab-helper-libraries/Getting%20Started/Installation.html

You will find that Jython has by far the most documentation, support in the forum, and has the most evolved helper libraries for scripted automation. I have even put together an add-on for the installation of both…

As for your specific issue, I’d be glad to help out, but I would need some more details about your installation so that I could reproduce the problem. It has been a long while ago, but I do not recall having any issues importing libraries when using Groovy. From the looks of it, you are trying to import a script, not a library, into another script, which can’t be done. Depending on how you have added Groovy to the classpath, you will likely need to add your library to the same directory.

Hey there,
I have the Groovy installation running (basically just copied the libs from the distribution to runtime/lib/ext directory) and it seems to work.
I have a strong Java background, so I really want IDE (IntelliJ) support for my rules in typed language (thats why I really dislike rules DSL and JavaScript/Python is not my preference). I even tried to get Kotlin working to the place where the Kotlin script got loaded, but I have class loading issues with the OH core classes.
I also am perfectly fine with using the raw API (not missing the helper libraries yet).

To the issue at hand: So you’re telling me that I cannot reference classes/methods from another groovy file in the same directory? Is this a limitation of how class loading in JSR223 works?
So how can I structure my code then… I don’t want to have it in one big script :frowning: … do I really have to create a library for my “utility” scripts? That would kind of defeat the purpose of having scripts in the first place…

Then I could just write my own Java lib and drop it in the runtime/lib/ext directory. Which sounds the better the more I think about it. Would that be possible?

VS Code and IntelliJ can both be used with Jython.

Only scripts can be located in the $OPENHAB_CONF/automation/jsr223/ directory. You can only access libraries that are in the classpath. If you have put the Groovy libraries into $OPENHAB_RUNTIME/lib/ext/, then put your other libraries in there too, as I had recommended…

Your “utility” scripts are not scripts… they are libraries.

How do you mean?

Yes, of course.

I think this is the part where I disagree. If I have for example a method in my script that I want to reuse in another script (and still keep the possibility to change it easily and have it reloaded “live”) I don’t see the point in creating a library out of it.

If you say that’s technically not possible (I understood it that way: Each script is compiled separately and only has runtime/lib/ext and the OH core classes on its classpath, not the other scripts?), then it’s something I can live with, but if I have to create a library anyway I can as well code everything in Java.
Thanks for your comments so far! I’d really appreciate it if you could point me to some documentation / code reference where I can learn about class loading in the context of OH scripts/JSR223.

Yes, I know and did, but still autocompletion, parameter hints, refactoring and so on is far more sophisticated with a strictly typed language. But let’s not get down that road here.

1 Like

As I have said, you can’t. You must put your reusable code into a library.

I do not believe it is possible to reload a library in Groovy without starting a new GroovyScriptEngine, which would require restarting the rule engine or OH. I don’t know Groovy all that well though, so I could be wrong. However, you can reload libraries in Jython.

You’d need to read through the automation code, and also javax.script. Automation in OH is very poorly documented and is sorely lacking a maintainer.

1 Like

Hi all,
I did a lot of fiddling around with this lately and finally found a solution. It’s quite hacky, but it works. If anybody’s interested here’s what I did:

  1. I added an additional System property groovy.path to EXTRA_JAVA_OPTS which points to a lib directory outside the normal jsr223 directory, like they do for Jython here https://openhab-scripters.github.io/openhab-helper-libraries/Getting%20Started/Installation.html
    E.g., EXTRA_JAVA_OPTS="-Dgroovy.path=${OPENHAB_CONF}/automation/lib/groovy"
  2. I put my utility classes to ${OPENHAB_CONF}/automation/lib/groovy. Note that utility class must compile without errors when put to the normal jsr223 directory. (Not sure what happens when you leave the utility class in the default jsr223 directory…)
    My Utility class currently looks like this:
RuleBase.groovy
import org.slf4j.LoggerFactory
import org.slf4j.Logger
import org.openhab.core.automation.*
import org.openhab.core.automation.util.*
import org.openhab.core.automation.module.script.rulesupport.shared.simple.*
import org.eclipse.smarthome.config.core.Configuration
import org.eclipse.smarthome.core.items.Item
import org.eclipse.smarthome.core.items.ItemRegistry
import org.eclipse.smarthome.core.items.events.ItemStateChangedEvent
import org.eclipse.smarthome.core.items.events.ItemStateEvent
import org.eclipse.smarthome.core.items.events.GroupItemStateChangedEvent
import org.eclipse.smarthome.core.types.State
import org.eclipse.smarthome.core.events.Event

scriptExtension.importPreset("RuleSupport")
scriptExtension.importPreset("RuleSimple")


// Ugly workaround for openhab issues, see https://community.openhab.org/t/groovy-rules-no-longer-work-after-update-to-2-5-1535/68767/12
Class.forName("org.eclipse.smarthome.core.items.events.ItemStateEvent");
Class.forName("org.eclipse.smarthome.core.items.events.ItemStateChangedEvent");
Class.forName("org.eclipse.smarthome.core.items.events.GroupItemStateChangedEvent");
Class.forName("org.eclipse.smarthome.core.events.Event");

public abstract class AbstractSimpleRule extends SimpleRule {
  
  protected Logger logger = LoggerFactory.getLogger("jsr223.groovy.base."+this.getClass().getName());
  protected ItemRegistry itemRegistry;
  protected Item item;
  
  public AbstractSimpleRule(ItemRegistry itemRegistry, String itemName)
  {
    setName(itemName);
    this.itemRegistry = itemRegistry;
    this.item = itemRegistry.getItem(itemName);

    logger.debug("Initializing " + this.getClass().getSimpleName() + " for " 
                    + this.item.getName());
  }
  
  public void addOnChangeTrigger(Item item)
  {
    addTrigger(
      TriggerBuilder.create()
          .withId(this.uid + "-" + item.getName() + "-change")
          .withTypeUID("core.ItemStateChangeTrigger")
          .withConfiguration(new Configuration([itemName: item.getName()]))
          .build()
      );
  }

  public void addOnUpdateTrigger(Item item)
  {
    addTrigger(
      TriggerBuilder.create()
          .withId(this.uid + "-" + item.getName() + "-update")
          .withTypeUID("core.ItemStateUpdateTrigger")
          .withConfiguration(new Configuration([itemName: item.getName()]))
          .build()
      );
  }

  public void addTrigger(Trigger trigger)
  {
     List<Trigger> triggers = new ArrayList<Trigger>(getTriggers());
     triggers.add(trigger);
     setTriggers(triggers);
  }
  
  public Object execute(Action module, Map<String, ?> inputs) {
      def event = inputs.get("event");
      onEvent(event);
  }
  
  public void onEvent(Event event)
  {
      if (event instanceof ItemStateEvent)
      {
        onUpdate(event.itemName, event.itemState);
      }
      else if (event instanceof GroupItemStateChangedEvent)
      {
        onGroupItemChange(event.itemName, event.memberName, event.oldItemState, event.itemState);
      }
      else if (event instanceof ItemStateChangedEvent)
      {
        onChange(event.itemName, event.oldItemState, event.itemState);
      }
      else
      {
        logger.warn("Unsupported Event Type:" + event.getClass().getName());
      }
  }

  public void onUpdate(String itemName, State itemState)
  {
    // Empty Implementation
    // logger.warn("onUpdate("+itemName+", "+itemState+")");
  }
  
  public void onGroupItemChange(String groupItem, String memberItem, State oldState, State newState)
  {
    onChange(memberItem, oldState, newState);
  }

  public void onChange(String itemName, State oldState, State newState)
  {
    // Empty Implementation
    // logger.warn("onChange("+itemName+", "+oldState+", "+newState+")");
  }
}
  1. In your actual Script create a new GroovyShell using the ClassLoader and Binding of the current instance:
GroovyShell shell = new GroovyShell(this.getClass().getClassLoader(), this.binding);

This makes things like itemRegistry, automationManager etc. known to the new shell.

Determine the groovy scripts path (just to make sure we have the right path)

File groovyLibPath = new File(System.getProperty("groovy.path"));

Load all your “library”-Scripts using shell.evaluate(). E.g.

shell.evaluate(new File(groovyLibPath, "RuleBase.groovy"));

Finally, run your actuall script using

shell.run(""" < YOUR SCRIPT > """, "MyScriptName", [])

The three quotes mark a multi-line string. Hence you need to be a bit carefull, that everything between these quotes is considered a string… Don’t use three quote multi-line strings inside this block!

Putting this all together it looks like this:

alarm.groovy
// Path is set using EXTRA_JAVA_OPTS="-Dgroovy.path=${OPENHAB_CONF}/automation/lib/groovy"
File groovyLibPath = new File(System.getProperty("groovy.path"));

// Create a new GroovyShell using the current ClassLoader and current Binding
GroovyShell shell = new GroovyShell(this.getClass().getClassLoader(), this.binding);

// Load any libraries from groovy lib path
shell.evaluate(new File(groovyLibPath, "RuleBase.groovy"));

// Run the actual script
shell.run("""
import org.slf4j.LoggerFactory
import org.slf4j.Logger

import org.eclipse.smarthome.core.items.ItemRegistry
import org.eclipse.smarthome.core.types.State

import AbstractSimpleRule;

Logger myLogger2 = LoggerFactory.getLogger("jsr223.groovy.base");

class AlarmRule extends AbstractSimpleRule {
  
  public AlarmRule(ItemRegistry itemRegistry, String itemName)
  {
    super(itemRegistry, itemName);
    
    addOnChangeTrigger(this.item);
    addOnUpdateTrigger(this.item);
  }
  
  public void onUpdate(String itemName, State itemState)
  {
    logger.debug("onUpdate("+itemName+", "+itemState+")");
  }

  public void onChange(String itemName, State oldState, State newState)
  {
    logger.debug("onChange("+itemName+", "+oldState+", "+newState+")");
  }
}
  
automationManager.addRule(new AlarmRule(itemRegistry, "gAlarm"));

""", 
// Script name - we use the current script name. Not sure if this matters
this.getClass().getName(), 
// Script parameters - none
[]);

There’s one known limitation to this approach: Changes to the utility class will not directly trigger a reload of the script and will not automatically be reflected. To have become active you need to trigger a reload of each script using them.

Best,
Andreas

If anyone is still looking for a solution (that does not involve writing your whole script as a string):

conf/automation/jsr223/lib.groovy:

import org.slf4j.LoggerFactory

class TestLib {
    static LOG = LoggerFactory.getLogger("org.openhab.core.automation.lib")

    static def testMethod() {
        LOG.info("in TestLib.testMethod")
    }
}

conf/automation/jsr223/foo.groovy:

import org.slf4j.LoggerFactory

LOG = LoggerFactory.getLogger("org.openhab.core.automation.foo")

Class TestLib = this.class.classLoader.parseClass(new File('../conf/automation/jsr223/lib.groovy'))

LOG.info("in main script")
TestLib.testMethod()

Output when the script is loaded:

 [DEBUG] [rt.internal.loader.ScriptFileWatcher] - Dequeued file:/openhab/conf/automation/jsr223/foo.groovy
 [INFO ] [rt.internal.loader.ScriptFileWatcher] - Loading script '/openhab/conf/automation/jsr223/foo.groovy'
 [DEBUG] [ipt.internal.ScriptEngineManagerImpl] - Added ScriptEngine for language 'groovy' with identifier: file:/openhab/conf/automation/jsr223/foo.groovy
 [INFO ] [org.openhab.core.automation.foo     ] - in main script
 [INFO ] [org.openhab.core.automation.lib     ] - in TestLib.testMethod
 [DEBUG] [rt.internal.loader.ScriptFileWatcher] - Script loaded: /openhab/conf/automation/jsr223/foo.groovy

Only thing to note is that the library file should be a class. The methods don’t have to be static, you can do var t = TestClass.newInstance() and call member methods on that.

ref: Load script from groovy script - Stack Overflow

2 Likes

Nice! Just tried it (I had to adapt the path to load the file from to an absolute one, but that’s not a problem) and it works. Far better than the other hack, almost sleek :sunglasses: - thank you!

Hi,
this works, too.

/srv/openhab/conf/automation/jsr223/mytest/Util.groovy

package mytest

class Util {
    public static String getHello() {
        return "Hello world!"
    }
}

/srv/openhab/conf/automation/jsr223/loadertest.groovy

import org.slf4j.LoggerFactory

def logger = LoggerFactory.getLogger("org.openhab.core.automation.examples")

GroovyClassLoader cl = getClass().getClassLoader();
logger.info "classloader1 {}", cl

cl.addClasspath('/srv/openhab/conf/automation/jsr223')
Class Util = cl.loadClass("mytest.Util")

logger.info("hello {}", Util.getHello())

It would be nice, if openhab would do
cl.addClasspath('/srv/openhab/conf/automation/jsr223')
automaticly.
That would allow imports like

import mytest.Util

Modifiying org.openhab.automation.groovyscripting.internal.GroovyScriptEngineFactory would do the trick.

public class GroovyScriptEngineFactory extends AbstractScriptEngineFactory {

    private final org.codehaus.groovy.jsr223.GroovyScriptEngineFactory factory = new org.codehaus.groovy.jsr223.GroovyScriptEngineFactory();
....
    @Override
    public @Nullable ScriptEngine createScriptEngine(String scriptType) {
        if(scriptTypes.contains(scriptType)) {
            org.codehaus.groovy.jsr223.GroovyScriptEngineImpl engine = (org.codehaus.groovy.jsr223.GroovyScriptEngineImpl) factory.getScriptEngine();
            engine.getClassLoader().addClasspath("/path/to/lib");
            return engine;
        }
        return null;
    }
}

Edit: fixed addPath to addClasspath

This sounds promising. I understand we’d need to create a PR against this file?

I’m not yet sure about the path to the lib directory, though. There might be multiple script locations (afaik there are at least 2 different OH config locations) and I think there is not one “standard” - e. g. Raspbian might use adapted paths.

Yes …
i wouldn’t even use engine = factory.getScriptEngine() , because it creates a GroovyClassLoader for every Script. This is a pain in the ass, because your shared classes would be from different class loaders.

Better would be a shared GroovyClassLoader.

    private final GroovyClassLoader gcl = new GroovyClassLoader();
    public GroovyScriptEngineFactory() {
        gcl.addClasspath ("/path/to/lib"); // FIXME something from the config
    }
...
    @Override
    public @Nullable ScriptEngine createScriptEngine(String scriptType) {
        if(scriptTypes.contains(scriptType)) {
            org.codehaus.groovy.jsr223.GroovyScriptEngineImpl engine = new org.codehaus.groovy.jsr223.GroovyScriptEngineImpl(gcl);
            return engine;
        }
        return null;
    }
}

This would tremendously ease up developing custom GroovyRules.
You could write loads of custom Helper-classes with the full Java toolchain/ecosystem.

With minor changes you could deploy helper.jars. Just load them in your classloader.

Hmmm, this really works. Looking at the ScriptFileWatcher in the core showed how to obtain the directory name…

The only remaining problem now is the order of the files during initital loading. If the Utils class is loaded as last script, it fails with an error. Saving the referencing script again, so it gets reloaded, solves it, though.

@Component(service = ScriptEngineFactory.class)
@NonNullByDefault
public class GroovyScriptEngineFactory extends AbstractScriptEngineFactory {

    private final Logger logger = LoggerFactory.getLogger(GroovyScriptEngineFactory.class);
    private static final String FILE_DIRECTORY = "automation" + File.separator + "jsr223";

    private final org.codehaus.groovy.jsr223.GroovyScriptEngineFactory factory = new org.codehaus.groovy.jsr223.GroovyScriptEngineFactory();
    private final GroovyClassLoader gcl = new GroovyClassLoader();

    private final List<String> scriptTypes = Stream.of(factory.getExtensions(), factory.getMimeTypes())
            .flatMap(List::stream) //
            .collect(Collectors.toUnmodifiableList());

    @Override
    public List<String> getScriptTypes() {
        return scriptTypes;
    }

    public GroovyScriptEngineFactory() {
        String scriptDir = OpenHAB.getConfigFolder() + File.separator + FILE_DIRECTORY + File.separator;
        logger.info("Adding script directory {} to the GroovyScriptEngine class path.", scriptDir);
        gcl.addClasspath(scriptDir);
    }

    @Override
    public @Nullable ScriptEngine createScriptEngine(String scriptType) {
        if (scriptTypes.contains(scriptType)) {
            return new org.codehaus.groovy.jsr223.GroovyScriptEngineImpl(gcl);
        }
        return null;
    }
}

Ideas?

  • naming conventions (error prone, since the dependencies might be actually a graph, rather than a list)
  • enhancing the script loading infrastructure, leaving the order to the ScriptEngineFactories?
  • …?

btw, I think we also have to respect the OSGI lifecycle of the component… so the final local GroovyClassLoader might need some more thought.

1 Like

The Utils.groovy doesn’t have to be loaded as a script. The GroovyClassLoader takes care of it.
In fact, the “lib”-path shouldn’t be loaded by the script engine factory.

GroovyScriptEngineImpl takes the ClassLoader from the thread as parent by default, which is org.eclipse.osgi.internal.framework.ContextFinder in the OSGI-Context.
But it would be possible to wrap a ClassLoader.
In that way, you could override class loadClass(String to control, which classes can be loaded from the groovy-path.

I’v made a commit on my fork

Then following works, when you place your packages under conf/automation/groovy/

$ cat conf/automation/groovy/mytest/Util.groovy

package mytest

class Util {
    public static String getHello() {
        return "Hello world!"
    }
}

$ cat conf/automation/jsr223/test.groovy

import mytest.Util   // <-------- HERE IT IS
import org.slf4j.Logger
import org.slf4j.LoggerFactory

def logger = LoggerFactory.getLogger("org.openhab.core.automation.examples")
def cl = getClass().classLoader

logger.info "classloader1 {} {}", cl, cl.classPath
logger.info "classloader2 {} {}", cl.parent, cl.parent.classPath
logger.info "classloader3 {}", cl.parent.parent

logger.info "mytest.Util.hello -> {}", Util.hello

OUTPUT

[INFO ] [rulesupport.loader.ScriptFileWatcher] - Loading script '/srv/openhab2/conf/automation/jsr223/test.groovy'
[INFO ] [org.openhab.core.automation.examples] - classloader1 groovy.lang.GroovyClassLoader$InnerLoader@136e750b [/srv/openhab/conf/automation/groovy/]
[INFO ] [org.openhab.core.automation.examples] - classloader2 groovy.lang.GroovyClassLoader@11869d70 [/srv/openhab/conf/automation/groovy/]
[INFO ] [org.openhab.core.automation.examples] - classloader3 org.eclipse.osgi.internal.framework.ContextFinder@534a5a98
[INFO ] [org.openhab.core.automation.examples] - mytest.Util.hello -> Hello world!

FYI @pravussum

is merged