NGRE (JSR223 Python) - Phantom Rules

Platform Information

  • Hardware: Raspberry Pi 3B+
  • OS: Raspberry Pi OS (32 Bit - Buster)
  • Java Runtime Environment:
  • openjdk version “1.8.0_152”
  • OpenJDK Runtime Environment (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 1.8.0_152-b76)
  • OpenJDK Client VM (Zulu Embedded 8.25.0.76-linux-aarch32hf) (build 25.152-b76, mixed mode, Evaluation)
  • Jython 2.7.0
  • openHAB 2.5.5 (Release Build)

Hello Community,

I’m using the NGRE with Python scripts. These rules seem to rules seem to work well, but some how I’ve managed to create some “Phantom Rules”, and short of hacking at the JSON DB, I can’t figure out how to get rid of them. (These “Phantom Rules” are more than cosmetic as they are not just in the PaperUI, and actually triggering according to the logs.) Does any one have any suggestions? Here is examples and and what I’ve tired to remove the rules.

Phantom Rule Example

In the above screenshot, I apparently have two rules named “Front Porch Lights On at Civil Dusk”.
However, I only have one Python script file, and that file only has one such rule.

What I’ve Tried to Delete the Duplicates

  1. From the PaperUI, I’ve tried to delete both rules. However, I get a 404 error in the PaperUI, and this is logged.
2020-06-21 15:15:57.983 [INFO ] [utomation.rest.internal.RuleResource] - Received HTTP DELETE request at 'rules/1f004450-df27-4a4a-86b9-51b5a9530911' for the unknown rule '1f004450-df27-4a4a-86b9-51b5a9530911'.
2020-06-21 15:16:22.849 [INFO ] [utomation.rest.internal.RuleResource] - Received HTTP DELETE request at 'rules/229e9d59-ecb4-4849-a9c8-da379ef4e62e' for the unknown rule '229e9d59-ecb4-4849-a9c8-da379ef4e62e'.
  1. I’ve tried to delete the rules through the REST API. Unsurprisingly, I still get a 404 and the same messages logged in point 1 above.

  2. I’ve tried:

  • Stopping openHAB [sudo service openhab2 stop]
  • Renaming the Python script file to .old
  • Clearing the cache [sudo openhab-cli clear-cache]
  • Starting openHAB. (After the I start it, PaperUI shows no rules, which I expect.)
  • Renaming the Python script file to .py
  • Restarting openhab. (After the restart duplicate rules show up again in PaperUI.

Regards,
Burzin

@Burzin_Sumariwalla - I have been seeing some strange behaviour and just sat down to figure it out - I have the exact same problem. Many of my Jython rules are duplicated.

@5iver thoughts? After OH starts, if I edit and resave the file, only a single version of the rule is created

I am running 2.5.6~S150

HI @mjcumming,

The only thing that I can think of is that what I reported was somehow caused by Java exceptions due to a poor first attempt at converting my rules to the NGRE/Python. That theory doesn’t quite hold water though because I have a rule that gives a WARN for an unknown item. I remember fixing that after I had my rules stabilized. I should mention that I just reinstalled the org.openhab.core.automation.module.script.scriptenginefactory.jython-2.5.0-SNAPSHOT JAR. See:

It hasn’t made any difference in the problem I observed.

Regards,
Burzin

I have a manual jython installation. What’s interesting is this only happens with one of my python files. The other 10 files load fine.

Did the duplication you see go away after saving a second time (no OH restart)?

@mjcumming,

I haven’t been able to get rid of the duplicated rules. Just to be clear though, the rules don’t keep duplicating. I just just can’t get rid of whatever’s duplicated/phantom. It’s curious that you mentioned that your problem only happens with one of you python files. I have my rules jammed into one Python file. There aren’t too many rules though, but I plan on using a second file to contain any Beta rules I’m testing out.

Regards,
Burzin

I don’t recall ever having this occur, but I could see it happen if there was an error in the rule or possibly if there were timers running. If you can distill it down to a specific rule that will reproduce the issue, please post it. @Burzin_Sumariwalla, just out of curiosity, what is in your “Front Port Lights On at Civil Dusk” rule?

BTW, there is nothing specific to Jython that would cause this, so if there is an issue, it would be in the new rule engine.

Hi @5iver,

This is my current “Front Porch Lights On at Civil Dusk” rule.

@rule("Front Porch Lights On at Civil Dusk")
@when('Channel astro:sun:local:civilDusk#event triggered START')
def front_porch_lights_on_at_civil_dusk(event):
    front_porch_lights_on_at_civil_dusk.log.info("Rule: 'Front Porch Lights On at Civil Dusk' is executing.")
    events.sendCommand("Front_Porch_Lights", "ON")
    front_porch_lights_on_at_civil_dusk.log.info("Rule: 'Front Porch Lights On at Civil Dusk' has completed.")

I believe the old rule is this. However, I can’t figure out where it’s stored.

@rule("Front Porch Lights On at Civil Dusk")
@when("Channel astro:sun:local:civilDusk#start triggered START")
def front_porch_lights_on_at_civil_dusk(event):
    events.sendCommand("Front_Porch_Lights", "ON")
    front_porch_lights_on_at_civil_dusk.log.info("Rule: 'Front Porch Lights On at Civil Dusk' fired.")

Oddly, the “automation_rules.json” file (/var/lib/openhab2/jsondb) is basically empty. It only has “{}” Equally curious, I do see an “automation_rules_disabled.json” file. That file contains the UIDs of disabled rules, but the rules don’t seem to stay disabled and the UIDs seem to reenumerate after restarts.

Also, my “Phantom Rules” (I can’t think of anything better to call them) did appear when I was testing. I did have timers that went awry and error in various rules.

Regards,
Burzin

Okay, I figure out how to get the two rules from the REST API. Since I’m terrible at parsing JSON correctly, I dumped the output from the REST API into a JSON to CSV converter (https://konklone.io/json/). I’ve attached the CSV output of the two “Front Porch Lights On at Civil Dusk” rules.

Python_Rules.csv.txt (1.1 KB)

Hi @mjcumming and @5iver,

I figured out the problem. There was actually another Python script file in the Personal directory. When I was troubleshooting my Python rules, I made a copy of my existing Python rules and just appended .example to it via Finder (OSX). For whatever reason, on an OSX device, .py was appended to .example. Now the curious thing is that in OSX, the file does not show the .py extension. The .py extension does show up in the OS and when viewing the share from Windows 10.

Regards,
Burzin

1 Like

Interesting. I have looked through my files and cannot find a duplicate.

@5iver - if you delete a .py file does the .class compiled file get deleted? Could that be the issue?

No, but it will be checked when the module is recreated. I think we are talking scripts here though :slightly_smiling_face:.

I dug around to see if I had duplicate files or extra .class files. There are not any. Yet, when OH starts, for the one python file I always get duplicate rules. A simple save solves that.

@5iver are files in the lib\python\personal directory loaded automatically or only when imported?

If you post or send, I’d like to take a look.

Only when imported

Found the problem. I am using a script to load a module (that contains rules) in the lib\python\personal directory. The script in the jsr223\python\personal directory that loads the module from the lib directory uses reload and that seems to result in duplicated rutes.

Solution is to wrap the rules in a function and then call that function from the script in the js223 dir.

2 Likes