How can I get my ECMA scripts working on openHABian?

Hi everyone,

I am a openHABian beginner setting up a electricity spot market control system as the electricity prices here in Finland is sky rocketing for now. After a lot of try and error I have been able to set up openHABian 64bit and InfuxDB2 on my Raspberry Pi 4.

For almost aweek now I have tried to get my scripts running with no luck. Googling for help has not been successful either. The scripts are known to be running correctly on a other openHAB members installation.

Running any of my scripts ends with the same error: 2022-09-13 08:45:09.580 [ERROR] [internal.handler.ScriptActionHandler] - Script execution of rule with UID ‘5fe8f3d523’ failed: org.graalvm.polyglot.PolyglotException: EvalError: /etc/openhab/automation/js/node_modules/kotikolo/fmi.js

My rule script is here,

configuration: {}
triggers:
  - id: "1"
    configuration:
      time: 09:30
    type: timer.TimeOfDayTrigger
conditions: []
actions:
  - inputs: {}
    id: "2"
    configuration:
      type: application/javascript;version=ECMAScript-2021
      script: |
        fmi = require('kotikolo/fmi.js');
        influx = require('kotikolo/influx.js');

        // Place recognized by Finnish Meteorology Institute's API.
        place = 'numminen';

        // Read weather forecast and write them to the database.
        points = fmi.getForecast(place);
        influx.writePoints('fmi_forecast_temperature', points);
    type: script.ScriptAction

and the fmi.js script here,

/**
 * Javascript module for reading weather data from FMI API.
 *
 * Copyright (c) 2022 Markus Sipilä.
 *
 * Permission to use, copy, modify, and/or distribute this software for any
 * purpose with or without fee is hereby granted, provided that the above
 * copyright notice and this permission notice appear in all copies.
 *
 * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
 * REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY
 * AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
 * INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
 * LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE
 * OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR
 * PERFORMANCE OF THIS SOFTWARE.
 */

/**
 * Exports.
 */
module.exports = {
    getForecast: getForecast,
};

/**
 * Reads the weather forecase from FMI API.
 *
 * @param string place
 *   Place recoginzed by FMI API.
 *
 * @return string
 *   XML response from FMI API.
 */
function getForecast(place) {
    const xml = makeApiCall(place);
    const points = preparePoints(xml);
    return points;
}

/**
 * Makes an API call to the Finnish Meteorology Institute.
 *
 * @param string place
 *   Place recoginzed by FMI API.
 *
 * @return string
 *   XML response from FMI API.
 */
function makeApiCall(place) {
    const http = Java.type("org.openhab.core.model.script.actions.HTTP");
    console.log('fmi.js: Making an API call to FMI API...');
    const url = 'http://opendata.fmi.fi/wfs?service=WFS&version=2.0.0&request=getFeature&storedquery_id=fmi::forecast::hirlam::surface::point::timevaluepair&place=' + place;
    const xml = http.sendHttpGetRequest(url, 10000);
    return xml;
}

/**
 * Parses the forecasted temperatures from the XML response.
 *
 * @param string xml
 *   FMI response in XML format.
 *
 * @return array
 *   Array of point objects.
 */
function preparePoints(xml) {
    console.log('fmi.js: transforming XML to JSON and parsing temperatures...');
    const transformation = Java.type("org.openhab.core.transform.actions.Transformation");
    let tempPoints = [];

    // Early exit in case XML is null.
    if (xml == null) {
	console.error('fmi.js: XML empty, parsing aborted.')
	return tempPoints;
    }
    try {
	const jsObject = JSON.parse(transformation.transform('XSLT', 'xml2json.xsl', xml));
	const members = jsObject['wfs:FeatureCollection']['wfs:member'];
	for (let i = 0; i < members.length; i++) {
	    let feature = members[i]['omso:PointTimeSeriesObservation']['om:featureOfInterest']['sams:SF_SpatialSamplingFeature'];
	    if (feature['gml:id'] == 'enn-s-1-1-Temperature') {
		let tsPoints = members[i]['omso:PointTimeSeriesObservation']['om:result']['wml2:MeasurementTimeseries']['wml2:point'];
		for (let j = 0; j < tsPoints.length; j++) {
		    let point = {
			datetime: tsPoints[j]['wml2:MeasurementTVP']['wml2:time'],
			value: tsPoints[j]['wml2:MeasurementTVP']['wml2:value']
		    };
		    tempPoints.push(point);
		}
	    }
	}
	console.log('fmi.js: Temperatures parsed!');
    }
    catch (exception) {
	console.error('fmi.js: Exception parsing temperatures: ' + exception.message);
    }

    return tempPoints;
}

Any help is greatly appreciated - my head is starting to hurt from banging it to the wall :face_with_head_bandage:

You’ve not mentioned installing the optional JSS rules add=on to support that?

Where did you put that file i.e. what folder?

Yes I have installed the automation add-on JSScripting.

The fmi.js script is installed in the folder /etc/openhab/automation/js/node_modules/kotikolo

What version of OH is it working on and what version is it but working on?

This is a third party library you installed using NPM I assume?

Is there nothing after that error, like a stack trace or the like?

A quick hint whether or not the installed it is if you see GraalVM mentioned, they’ve installed the add-on. Right now the JS Scripting add-on is the only one that uses it.

@rlkoshak I’m the author of that fmi.js, and it Works For Me ™,
see Control a water heater and ground source heat pump based on cheap hours of spot priced electricity - #13 by masipila

I’m running openHAB 3.3.0 on openHabian, also on Raspberry Pi 4 like @ruxu

I was first suspecting that the path in the require statement would be incorrect, so I modified my script action so that the path was incorrect. That results in a different error message in the logs, that would be
org.graalvm.polyglot.PolyglotException: TypeError: Cannot load CommonJS module: 'invalid-path-here/fmi.js'

So that suggests (does not prove, but suggests) that @ruxu’s script action is able to read the fmi.js from /etc/openhab/automation/js/node_modules/kotikolo/fmi.js

@ruxu Let’s try a simple hello world script.

Create a file hello-world.js so that the full path will be
/etc/openhab/automation/js/node_modules/kotikolo/hello-world.js

The hello-world.js will be as follows:

module.exports = {
    hello: hello,
};

function hello() {
    console.log('Hello World!');
}

The script action that will call this js file will be:

hw = require('kotikolo/hello-world.js');
hw.hello();

Executing this script action should result in the following log entry that looks like this:
2022-09-13 21:15:15.258 [INFO ] [g.openhab.automation.script.ui.hello] - Hello World!

Cheers,
Markus

Noniin, I’m almost sure I figured it out. This is most likely a file permission error. I was able to repro the error message by modifying the file permissions so that openhab does not have read access to the file.

openHab runs as a Linux user ‘openhab’. You can’t log in to your Raspberry with this user account because the the shell login is disabled for this user. So you must have created the js file with some other user, but the file / directory permissions are so that openhab can’t read the file.

You can change the file owner and group like this:
In /etc/openhab/automation/js/node_modules

chown -R your-user:openhab kotikolo

This should recursively change the file owners so that your user still owns the files but openhab is added as the group.

You can see the file owner, group and file permissions with ls -la. The correct permissions for the js files would be -rw-r--r--. For the kotikolo directory, the permissions should be drwxrwxr-x

Cheers,
Markus

1 Like

Hi Markus,

Thank you for your quick response and suggestions on what to check out. Unfortunately though the problem was not solved. The directory and file rights are as you described after running chown.

When running your hello-world script example I now get a different error message,

2022-09-14 15:46:06.700 [ERROR] [internal.handler.ScriptActionHandler] - Script execution of rule with UID ‘acd9415235’ failed: org.graalvm.polyglot.PolyglotException: TypeError: Cannot load CommonJS module: ‘kotikolo/hello-world.js’

Could there be a difference in our basic installations - I am running latest openHABian 64bit + InfluxDB2 and all the add-ons you have listed? Everything is at this stage on the SD card. Could there be any small detail in your installation that is not listed in your very detailed instructions?

I got to struggle on - it will be solved at some point, with some persistence…

Thanx for your involvement
Rainer

Can you paste the results of ls -la from all directories from /etc/openhab onewards to your kotokolo directory?

Here is the listing. from what I understand the permissions seems to be correct.

openhabian@openhabianpi:/etc/openhab $ ls -la
total 56
drwxrwxr-x 14 openhab openhab 4096 Sep 11 19:59 .
drwxr-xr-x 96 root    root    4096 Sep 14 19:43 ..
drwxrwxr-x  5 openhab openhab 4096 Sep 13 19:02 automation
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 html
drwxrwxr-x  3 openhab openhab 4096 Sep 11 19:57 icons
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 items
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 persistence
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 rules
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 scripts
drwxrwxr-x  2 openhab openhab 4096 Sep 11 20:00 services
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 sitemaps
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 sounds
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:58 things
drwxrwxr-x  2 openhab openhab 4096 Sep 12 08:41 transform
openhabian@openhabianpi:/etc/openhab $ cd automation
openhabian@openhabianpi:/etc/openhab/automation $ ls -la
total 20
drwxrwxr-x  5 openhab openhab 4096 Sep 13 19:02 .
drwxrwxr-x 14 openhab openhab 4096 Sep 11 19:59 ..
drwxrwxr-x  7 openhab openhab 4096 Sep 13 19:02 jrule
drwxrwxr-x  3 openhab openhab 4096 Sep 11 20:28 js
drwxrwxr-x  2 openhab openhab 4096 Sep 11 19:59 jsr223
openhabian@openhabianpi:/etc/openhab/automation $ cd js
openhabian@openhabianpi:/etc/openhab/automation/js $ ls -la
total 12
drwxrwxr-x 3 openhab openhab 4096 Sep 11 20:28 .
drwxrwxr-x 5 openhab openhab 4096 Sep 13 19:02 ..
drwxrwxr-x 3 openhab openhab 4096 Sep 12 08:32 node_modules
openhabian@openhabianpi:/etc/openhab/automation/js $ cd node_modules
openhabian@openhabianpi:/etc/openhab/automation/js/node_modules $ ls -la
total 12
drwxrwxr-x 3 openhab openhab 4096 Sep 12 08:32 .
drwxrwxr-x 3 openhab openhab 4096 Sep 11 20:28 ..
drwxrwxr-x 2 openhab openhab 4096 Sep 14 15:30 kotikolo
openhabian@openhabianpi:/etc/openhab/automation/js/node_modules $ cd kotikolo
openhabian@openhabianpi:/etc/openhab/automation/js/node_modules/kotikolo $ ls -la
total 44
drwxrwxr-x 2 openhab    openhab 4096 Sep 14 15:30 .
drwxrwxr-x 3 openhab    openhab 4096 Sep 12 08:32 ..
-rw-rw-r-- 1 openhabian openhab 3243 Sep 12 08:28 date-helper.js
-rw-rw-r-- 1 openhabian openhab 3754 Sep 12 08:28 entsoe.js
-rw-rw-r-- 1 openhabian openhab 3278 Sep 12 19:59 fmi.js
-rw-rw-r-- 1 openhabian openhab   95 Sep 14 06:32 hello.js
-rw-rw-r-- 1 openhabian openhab 7478 Sep 12 08:51 influx.js
-rw-rw-r-- 1 openhabian openhab 6797 Sep 12 08:28 nibe.js
-rw-rw-r-- 1 openhabian openhab 3321 Sep 12 08:28 waterheater.js
openhabian@openhabianpi:/etc/openhab/automation/js/node_modules/kotikolo $

I think I will make an openHABian 32bit installation on another SD card and see what happens there with the scripts.

PS. Your “toinen juttu” comment duly received, you are right I will “otan opikseni” as someone said…

The only difference that I can see compared to what I have on my installation is that I have also changed the file owner of the js files to be openhab, you have them owned by openhabian. That should not make any difference, because the openhab gropu has also read and write permissions.

As per this official documentation, your files are located in the correct directory: JavaScript Scripting - Automation | openHAB

I’m as confused as you are @ruxu . Ideas anyone?

:slight_smile:

One more thought.

Did you re-save your rule AFTER you modified the file permissions? Whenever the js files are modified, the rule must be re-saved. Otherwise openHab will continue to use the previously loaded version of the file.

Yes I have saved the rules everytime after any change but still no luck. Could there be something wrong in my setup so that the binding between the two script functions are unrecognized?

I made a test rule where there in the first script was only the request statement for the second script and a log message_1. In the second script I put only a log message_2 and nothing else. When the rule was run both messages where printed out to the log and no error messages shown.

So at least it seems that the file rights are correct and the second script file can be run.

But if you run the rule again only the first script message_1 one is printed out propably because the second script request is memorized?

So you’re saying that you are able to run some other js files from the same /etc/openhab/automation/js/node_modules/kotikolo directory, but the fmi.js and hello-world.js both fail?

Can you show the actual code what script and rule you managed to get working?

Cheers,
Markus

Finally some success!

It came to my mind that the file xml2json.xsl might have wrong permissions and so it was and I corrected it to

-rw-r--r--

Then I got a little bit further ending to the following error,

SyntaxError: /etc/openhab/automation/js/node_modules/kotikolo/influx.js:41:35 Missing close quote

I had left out the starting quote from,

bucket: spot_control_data',

Now the scripts finally run,

2022-09-16 09:50:58.540 [INFO ] [nhab.automation.script.ui.5fe8f3d523] - fmi.js: Making an API call to FMI API...
2022-09-16 09:50:59.001 [INFO ] [nhab.automation.script.ui.5fe8f3d523] - fmi.js: transforming XML to JSON and parsing temperatures...
2022-09-16 09:51:00.768 [INFO ] [nhab.automation.script.ui.5fe8f3d523] - fmi.js: Temperatures parsed!
2022-09-16 09:51:00.807 [INFO ] [nhab.automation.script.ui.5fe8f3d523] - influx.js: Preparing to write 36 points to the database for fmi_forecast_temperature
2022-09-16 09:51:01.471 [INFO ] [nhab.automation.script.ui.5fe8f3d523] - influx.js: Points successfully saved for measurement fmi_forecast_temperature

Still confusing is that the small hello-world example you sent still doesn’t run for some reason?

Thank you once more Markus for your kind involvement!

1 Like

Actually Markus the starting quote is missing also in your original influx.js.txt file. Just so you know.

Thanks for letting me know, I’ll fix it later today and also mentiin the file permissions for the xml2json transformation file.

I don’t use .js files for my rules. But isn’t it the case that your .js files that need to be treated like rules need to be in the $OH_CONF/automation/js folder (or a subfolder not under node_modules)? Maybe it’s being loaded but not executed because OH thinks it’s a library.

Regarding the scripts for my application they are all running now without errors - so for the time being I am in a don’t fix it, if it ain’t broken situation…

Better hit my next issue when running pigpio,

2022-09-16 18:14:04.241 [WARN ] [l.handler.PigpioDigitalOutputHandler] - An error occured while changing the gpio value: (-99999999)