Jruby items.builder persistance

1

openHAB 4.3.2
RPi5 with openhabian

I have started delving into jruby for rules on openHAB

I was using blocky (i know, i know) before out of a need to get started with rules.

I have a thing that is monitoring a MQTT feed from a lithium solar battery where the feed sends the name of the cell with the highest voltage and the voltage in two items.

Each time the cell name can change to reflect the cell with the highest voltage

My jruby rule creates the new items as the cell name value changes which is great and stores the voltage accordingly. This is all working.
However when the script stops those are items deleted. Because they are not persistent.

I understand that I can add them to a group to make them persistent in the standard RRD4j Service.

This does not seem to ensure that they are “persistent” in the openHAB json DB or is it by adding them to this RRD4J persistence service (and make them restore on startup) is the only way to make them persistent. The documentation seems to only claim that they will not be persistent ) Note that by default items are not persisted to storage, and will be removed when the script unloads. It does not give any example of how to persist to the openHAB system (json) store if thats even possible.

My concern is if the jruby rule stops the items will be deleted and then any widgets using those will not be able to get any state info because the object (item) no longer exists.

Any advice on this would be great.

2

I don’t know how jRuby does this but there is some confusion here I think which I can clear up.

When you create an Item, Persistence and RRD4J have nothing to do with saving the Item once the rule is unloaded. Persistence saves the states of the Item as they change and can be queried and restore the state of an Item on a restart of OH. But it has nothing to do with how and whether the Item itself remains once the rule that created it is unloaded.

If you have widgets that depend on these Items existing, why create them dynamically from a rule in the first place? Based on what you are describing, the name of the cell with the highest value should be the state of a statically defined Item, not have a whole new Item created dynamically. Depending on the specifics of the MQTT messages, you should be able to route the voltage value to the right Item that corresponds to the battery name easily enough as well.

Dynamically created Items can be powerful and useful in a lot of situations However, as far as I can tell, they cannot be made permanent. If you have other parts of OH that depend on the existence of these Items to work (persistence, other rules, UI widgets), the Items should be static because otherwise you can never guarantee that the Item exists.

3

Hi Rich

Thanks for the answer

In this case the the data in the two channels/items looks like this(over time)
Cell_id maxCellV
1::3 29
1::12 28.5
2::5 30
Etc changing every 1 second as data from the BMS (Battery Management System) monitors the cells.

I have multiple systems with different number of batteries different sizes and different number of cells and the manufacture does not expose (in any documentation) how many cells are in any one supplied battery or how they are named

In this case the names of the cell seem also to change based on the battery BMS and yes i agree that this is an issue for widgets but once the rule had run for a while my thinking was the items would exist and therefore could be relied upon.

When jruby creates the items with the “ items.builder do” it creates them with a lock against them in the UI as if they were created in files but when i disable the rule they all disappear and hence why i was looking to ensure they get stored in the openHAB json store.

I agree with you assessment and my first thought was to create manually but there are many batteries with many cells and the names vary so not a easy or simple process but one that only needed about 6 lines of jruby to solve

4

Firstly. I’d like to clarify that this sentence does not refer to the Persistence database (e.g. rrd4j, influx, etc):

And I believe @BrettLHolmes understands this.

What it means is that if you unload / remove the script, the item will be gone / removed from the “system”.

The solution is not to let the script get unloaded.

However, if I understand your requirements correctly, you want to keep track of all the possible cell names overtime and eventually you want to have an item for every cell name that you’ve ever received in the past, even after you’ve restarted openhab?

What you can do save the list of cell items as you receive them into a file. It could be a simple text file with one line per item name, eg

Cell1
Cell2
Cell12

Then in your script, you can load these names and create the items accordingly.

If you can post your actual script, including its triggers, I can advise further.

5

The following made me come to a different conclusion. I apologize if I misunderstood:

However when the script stops those are items deleted. Because they are not persistent.
I understand that I can add them to a group to make them persistent in the standard RRD4j Service.
This does not seem to ensure that they are “persistent” in the openHAB json DB or is it by adding them to this RRD4J persistence service (and make them restore on startup) is the only way to make them persistent.

I like the idea of saving the names to a file the best. But you could also use the REST API and make some HTTP calls to create the Items.

6

Using REST API is different to creating the items in JRuby itself. When using the REST API, you are basically manipulating the jsondb (items that are maintained by the ManagedProvider), and not actually creating the items by JRuby’s Item provider.

  • jsondb Items are owned and managed by the ManagedItemProvider
  • .items Items are owned and managed by the GenericItemProvider
  • JRuby created items are owned and managed by JRuby’s Item Provider.When the script that creates the item is unloaded from memory, JRuby will remove the items created by that script as a clean up process.

For example, my items (900+) are created by JRuby. In my case, my script reads a YAML file where I kept the “master” definition of my items, and creates openHAB Items from that yaml data. My Items contine to live for as long as I don’t delete the script that created the item. This script loads up on openhab startup. So essentially it works just like .items file, or like jsondb items.

The difference is that if I ever edited my jruby script, the mechanism is to unload the script, which would remove all my items, and reload it, which the script then would execute, reads the YAML file, and (re) creates the items again. This normally doesn’t happen unless I noticed a bug or wanted to add more features to my yaml-loader-item-creator script.

But because it’s a script, it can react to triggers. In MY case, it watches for changes in my YAML file, re-reads that yaml file and updates (add new item, delete deleted items, updates changed items, etc).

In your case, your script will react to new MQTT messages and create new item on the fly, should it encountered a new (not yet created) item, and it would save that new item to your text file, so that the next time your script loads (e.g. openhab startup), it can recreate them.

You can of course deal with the REST api way of creating / managing your items using JRuby, as long as you understand the difference.

Whether you want to add these items to your Persistence config (rrd4j, influxdb, etc), is an entirely separate issue and you would configure them as usual, just like how you would configure Persistence for any other item. In this case, Persistence is the storage of a series of item states over time into a database.

Perhaps we should change the verbiage in JRuby’s documentation to avoid using the word “persist” to avoid this confusion.

7

Hi @jimtng

The jruby looks like this below(please excuse structure and conventions but critique welcome)
For reference example of a cell_name.state is “ 1 ::8”

Note that its a little more complex since not only do I have high Voltage but I have highTemp, LowTemp and low voltage so the work to manually create these goes up exponentially.(see triggers of the maxVoltage and MaxTemp for only 2 batteries at the bottom) the _id field is the field that holds the cell name and per battery there are 8 triggers and then up to 20cells per battery)

Would have been (too)easy if the items.builder had the option to persist in openHAB json storage.


triggerItem = event.item.name

cellName = event.item.state.gsub(/\s+/, "")
cellName = cellName.gsub(":","_")
varName = triggerItem.gsub("_id", "")

nameOfTheCellItem = triggerItem.gsub("_id", "") + "_"+ cellName

logger.info ("hasRuleTest varName: " + varName)
logger.info ("hasRuleTest cellName: " + cellName)
logger.info ("hasRuleTest CellItem: " + nameOfTheCellItem)
logger.info ("hasRuleTest varValue(new): #{items[varName].state}" )

if items[nameOfTheCellItem]
  items[nameOfTheCellItem] << items[varName].state
  #logger.info "hasInfoLithiumCellUpdate: Cell Item found "
else
  logger.warn "hasInfoLithiumCellUpdate: Cell Item NOT found will create it"
  items.build do
  number_item "#{nameOfTheCellItem}" ,state: items[varName].state  
  end 
end

Current Triggers

- id: "3"
    configuration:
      itemName: solarLithium01_lithium01_maxCellV_id
    type: core.ItemStateUpdateTrigger
  - id: "2"
    configuration:
      itemName: solarLithium02_lithium02_maxCellV_id
    type: core.ItemStateUpdateTrigger
  - id: "4"
    configuration:
      itemName: solarLithium01_lithium01_maxCellTemp_id
    type: core.ItemStateUpdateTrigger
  - id: "5"
    configuration:
      itemName: solarLithium02_lithium02_maxCellTemp_id
    type: core.ItemStateUpdateTrigger
8

Thanks yes and i have to think about persisting all this data in support of warranty claims or if its even usefully for any long term diagnostics (useless data stored is just electronic garbage)

9

You have two options:

  • As @rlkoshak mentioned, you could use REST API and let core + jsondb deals with saving this item and reloading it when openhab starts, or
  • Use JRuby and save the list of items into a text file. You’ll need to create a the items on startup. See below.

Currently JRuby doesn’t provide a convenient way to dealing with REST api, so you’ll have to do it using manual HTTP request, just like how you would do it in other languages. However I would say that implementing this in Jruby is much easier and the code would end up looking nicer.

So if you want to continue using the JRuby item provider / builder way, here’s what you can change to your script, including some fixes to your existing code

ITEMS_FILE = OpenHAB::Core.config_folder / "items/cell_items.txt"

if event.is_a? OpenHAB::Core::Events::StartlevelEvent
  File.readlines(ITEMS_FILE, chomp: true).each do |item_name|
    items.build { number_item item_name }
  end
  return
end

# In a rule that triggers on a state update, use `event.state` to get the actual state update
# instead of the state of the item
var_name = event.item.name.gsub("_id", "")
cell_name = event.state.gsub(/\s+/, "").tr(":", "_") # using gsub is fine too, but tr is more efficient for non regex
cell_item_name = "#{var_name}_#{cell_name}"

# Either use parentheses without space, or no parentheses at all
# e.g. logger.info "hasRuleTest", or logger.info("hasRuleTest")
# logger.info<space>("hasRuleTest") is basically the same as logger.info "hasRuleTest", or logger.info(("hasRuleTest"))
logger.info("hasRuleTest var_name: #{var_name}")
logger.info("hasRuleTest cell_name: #{cell_name}")
logger.info("hasRuleTest CellItem: #{cell_item_name}")
logger.info("hasRuleTest varValue(new): #{items[var_name].state}")

if items[cell_item_name]
  items[cell_item_name] << items[var_name].state
else
  File.write(ITEMS_FILE, "#{cell_item_name}\n", mode: "a")
  items.build { number_item cell_item_name, state: items[var_name].state }
end

And add a system startlevel 50 trigger to that same rule.

10

@jimtng

I will be revising my rule to include the suggestions above.

For the Section on the event.state I need to to some more testing to fully understand the differences between the two.

Thanks for the advice and comments on structure really appreciated.

11

@rlkoshak yes I have been doing some work with accessing the API from rules and that I used from blockly and I had considered this. For both the file and the API options I see benefits and caveats and the one additional requirements for the file based approach is the need to include the file in the backup/restore strategy even though its in the “OpenHAB::Core.config_folder / "items” folder

Thanks for your comments and advise as always

12

From memory (so I might be wrong, in which case it doesn’t matter):
Because the status update to the item is asynchronous, it may be possible, although perhaps rare (depending on system load?), that when you received a status update event, the item’s state hasn’t managed to be updated. So in this case, the value of event.state (in rulesDSL it’s the equivalent of newState) is most definitely the actual state being updated to the item, whereas item.state may still hold the previous state, and this may not matter if the updated state is the same anyway.

Also if my memory serves, this situation doesn’t quite apply to changed trigger, because the changed event is fired after the item’s state has actually been changed, so the event.state, although, there may be a chance of item.state being changed again after firing the event… so perhaps it’s still better to use event.state

I only picked the config_folder/items/ folder at random here. It could be stored anywhere you like because it’s only used / loaded by your script (because it has a .txt extension).

I’ve just remembered / realised, that you can actually use JRuby to create your items in ManagedItemProvider (and thus in the jsondb). Sorry I should’ve realised this earlier. It’s as simple as calling build with :persistent argument, i.e.: items.build(:persistent)

Here’s your revised rule - remove that system startlevel trigger!

# In a rule that triggers on a state update, use `event.state` to get the actual state update
# instead of the state of the item
var_name = event.item.name.gsub("_id", "")
cell_name = event.state.gsub(/\s+/, "").tr(":", "_") # using gsub is fine too, but tr is more efficient for non regex
cell_item_name = "#{var_name}_#{cell_name}"

# Either use parentheses without space, or no parentheses at all
# e.g. logger.info "hasRuleTest", or logger.info("hasRuleTest")
# logger.info<space>("hasRuleTest") is basically the same as logger.info "hasRuleTest", or logger.info(("hasRuleTest"))
logger.info("hasRuleTest var_name: #{var_name}")
logger.info("hasRuleTest cell_name: #{cell_name}")
logger.info("hasRuleTest CellItem: #{cell_item_name}")
logger.info("hasRuleTest varValue(new): #{items[var_name].state}")

if items[cell_item_name]
  items[cell_item_name] << items[var_name].state
else
  items.build(:persistent) { number_item cell_item_name, state: items[var_name].state }
end

Note here that you’ll have to delete the items using MainUI, although you can also use JRuby to delete it but the point is, the items aren’t going to disappear when your rule is unloaded.

13

This is the main reason to prefer the use of event.state for updated and changed triggers. The rule fires after the Item is updated but the Item might update again while the rule is running. I wanted to add that for commands, the rule triggers before the Item updates so it’s almost always going to be the case that the rule is running before the Item’s state changed in response to the command.

In both cases using the event instead of the Item’s state will avoid problems.

14

Thanks very much for the clarity on the event.state i understand.

Having the ability to add them to the config permanently is great and then should be added to the docs if you would like to point out how i make the request i will do that pr to close the loop (trying to learn how to give back a little in openHAB)

Thanks again for the comprehensive explanations. Changed the solution to this entry accordingly…

15

It is already in the docs, my bad.

image
image1828×238 23.1 KB

If you clicked on that link to items.build you’ll see the full explanation and options!

16

Hi @jimtng i did click on that the challenge for me was it is not readable - If you look at my original code you would see the :persistence and the click through link kind of shows the options but it is not clear how to use them perhaps I need more of a jruby or programer mind set but I could not find how to use the :persistence (despite trying various options) before reaching out to the community(yourselves).

Thanks for the additional comms. - Brett

17

Should we add an example for persistent?

18

Yes thats what i was thinking.

19

I’ve created a PR to add an example for persistent:

It may seem a bit “all over the place”, because from the main page, you’d have to click to items.build, then click through DSL::Items::Builder to get to the example, and then you’d have to go back to items.buid to re-read the parameter list again… but it is more organised this way, instead of having the examples spread over many places.

20

@BrettLHolmes: I’d love for you to review @jimtng’s change before I merge it. It seems clear enough to me, but I wrote the original documentation that is a bit all over the place, too! (Partly due to how the items builder itself is dynamic code). But it’s always good to get feedback from people that weren’t involved in writing things, but are using them.

Tangential - @jimtng perhaps it’s time we figure out a way to publish a temporary version of the yardoc website for a PR just like the openhab-docs repo does.

And finally…

I’m unclear here - are there still things in your JRuby version of rules that you need to access the API? I’d be curious to hear about them. While there’s nothing wrong with popping out to the API, and can be necessary if you’re communicating from one openHAB installation to another, it’s almost always possible to implement the same thing directly within the Java environment within the rule, avoiding the network request which has significantly more overhead. It might not be pretty doing it “the Java way… within Ruby,” though. Which is why we have a helper library! If it’s something that might be valuable for others, we could probably wrap it up nicely in the helper library.