Ecoflow on Openhab + OH Auto Integration recomenations

[Matej_Kotnik]

Hi

This is the 4 part series on my OpenHAB excperience, vision for improving OH and lately integrating Ecoflow devices, and migration to OH4.
So the topic is mainly focused on Ecoflow integration and some coments on my setup and vision/ recomendations for future openhab upgrades, at integration workflow on config files.

This multi content post is not quite up to convention, but we will seperate topics if required, this is my folow up as i didnt post a while and more questions arised.
While first i am sharing my Ecoflow project for others to test.

1.EcoFlow
I enjoy keeping an eye on the projects from Cody at WranglerStar Youtube channel, he does great off grid projects and he got me familiar with Ecoflow powerstations and solar equipment.

I have 2 EF devices, River Powerstation and Sloar microinverter as the new addition for balcony solar.

I am exploring the solutions that they offer with conectivity and automation.

They offer Cloud API, but dang i want shit out of the cloud and local control, so any ideas what can be done for River 2 Max and the Powerstream.
For now i share the API bridge to MQTT, that is based on API entities documented on Github HASS IO Ecoflow

Here is the code thar runs as API client and publishes MQTT metrics to local OH server.
It requires setting key and secret of Developer acount token and serial_number of the device.

Ecoflow_V9_clean.py.txt (8.9 KB)
It is done partialy by AI and i only did some fixes to entities, so thanks to AI for help.

So requirements are mqtt broker on server pc, and python 3 with libreries like paho-mqtt…
There is a slight bug that MQTT seems not to reconect if script looses connection to MQTT breoker, so advise apriciated how to fix it.

I run the script as a service on linux by systemd, by foloving this tutorial Automating Python Scripts with Systemd

Then comes the part i become anoyed of all the copy pasting and typing to make congig files for OH
So i use AI again to generate openhab configs. And fighting with AI to make it right, deciding when to step in to fix manually and so on.
Writing MQTT.things, Ecoflow.items and making sitemap entries, icons, and such.
Solar Items file Ecoflow_Solar.items.txt (4.4 KB)
River and Solar Sitemap snippet Sitemap elements EcoFlow River 2 and Solar.txt (10.3 KB)
MQTT things snippet Things mqtt EF_R2M and EF_Solar.txt (7.8 KB)
Pwerstation Items file Ecoflow.items.txt (3.4 KB)
Powerstation Sitemap file ecoflow_river.sitemap.txt (6.9 KB)

I feel it as ground breaking news, that auto integration is posible, as i see it takes way too much time to integrate devices and then i don’t bother unless there is something as exciting as Ecoflow gear, and then also i used AI burned it into the ground with data limits, as it makes misktakes, then i fix some and stuff it into another acount to finally get results i wanted, and then i customized it to finally get the dasboard bellow.

Sitemap River 2 max

Sitemap Ecoflow Solar inverter

Here is my amazing Power Monitoring Display, that swohs curent on 3 pases in house and also shows solar power in right chart below, 82 meaning 82% of full solar power 650W out of 800w max. It’s a simulator picture for beter visibility. It is a Nextion display powered by raw meter data from MQTT and also OpenHAB procesed data of max current. Below thin lines are house load and solar power.
It’s one ofe my biggest projects casue of the electrical overlaod situations visible on picture in L3.

2. Vision on imporving OH
Openhab is constantly upgrading and evolving but still there is all the work to be done on manually configuring each device, i wish i could make device templates that can be used to connect more similar devices with minimal effort like add device rename and done.

Here the openhab in VS Code can be improved with some more workflow automation like it does generating Items from things, but it lacks putting all items in sitemap in one go, lacks making MQTT things from MQTT broker data and such so it is like AI, but without noisy decisions it is precise based on data it gets,

I did a project with Openhab config Builder for my MQTT devices.
It actually reads mqtt topics and values and generates all the things, items, and builds sitemap, it is like black magic on steroids, but it uses dated rules for CSV parsing i used on legacy ESP32 devices, anyhow, i may upgrade it some day, as that is what Home Automation is, i feel it is mandatory automating the integration of devices, then you do all the custom work on automating and inter operation.

The workflow automation is a vision that i plan, is there anything similar happening that i am unavare of, mainly for MQTT, besides HA autodiscovery, that only does half of the stuff as since OH2.5 there is no auto items generation, why a downgrade??

3. My Server situation
And my server in need for cleanup of OH rules and VLC encodes grinding hard at all the data, asking for maintenence idk whatis the cause, but OH needs that exact task manager/rule manager tool to see what script uses what resources, that ttop is no good to see rule names and cpu usage, disk usage, net usage and such…Possibly i pussh it too much, or some bad code idk, but its hard to debug when no errors, just lots of data processing and Influx DB seems heavy on querries.

I built whole dashboard to manage what needs procesing to test load , so cleanup is in progress.

4. Migration to OH4
I am still migrating all the stuff to OH4, its lots of devices and the milestone is that both run for all diagnostics, while the old one is the automation guy.
The new N100 mini pc is running without Influx DB just on rrd4j but i wonder if rrd4j is wearing SSD more than influxDB, as i constantly agregating old data, that also meses up some discrete values, posibly i need to learn how to configure it in multy categories?

So the OH 2.5 is still the grandpa in demand, as the new guy isn’t cutting it yet in all the tasks all the new bindings like Luxtronik and more needed rewriting config code.

So is OH2.5 still secure, considering it is patched for log4j., i mean it is mainly offline from cloud as the cloud connector semms broken, anyow i use VPN.

Thanks for reading

All welcome to test ecoflow API integration project and coment on my thoughts and situation.

Cheers
Matej

[rlkoshak (Rich Koshak)]

@Matej_Kotnik

There is a slight bug that MQTT seems not to reconect if script looses connection to MQTT breoker, so advise apriciated how to fix it.

Paho lets you register handlers that get called on certain events like onConnect, onDisconnect, as well as a function that gets called upon receipt of a message.

You need to register a function to be called on disconnect and start a loop with a sleep to try reconnecting to the broker. Here is snippets from some paho code in my sensorReporter: sensorReporter/mqtt/mqtt_conn.py at main · rkoshak/sensorReporter · GitHub.

...
    self.client.on_disconnect = self.on_disconnect
...
    def _connect(self) -> None:
        while not self.connected:
            try:
                self.client.connect(self.host, port=self.port,
                                    keepalive=self.keepalive)
                self.connected = True
            except socket.error:
                self.log.error("Error connecting to %s:%s",
                               self.host, self.port)
                self.log.debug("Exception: %s", traceback.format_exc())
                sleep(5)
                self.connected = False

        self.log.info("Connection to MQTT is successful")
        super().conn_is_connecting()
...
    def on_disconnect(self,
                      client:mqtt.Client,
                      userdata:Any,
                      retcode:int) -> None:
        """ Called when the client disconnects from the broker. If the reason was
            not because disconnect() was called, try to reconnect.
        """
        self.log.info(
            "Disconnected from MQTT broker with client %s, userdata " "%s, and code %s",
            client,
            userdata,
            retcode,
        )

        self.connected = False
        super().conn_went_offline()
        if retcode != 0:
            self.log.error(
                "Unexpected disconnect code %s: %s reconnecting",
                retcode,
                mqtt.error_string(retcode),
            )
            self._connect()

@Matej_Kotnik

Then comes the part i become anoyed of all the copy pasting and typing to make congig files for OH

I’ve also got an example in sensorReporter on how to support the Homie standard for MQTT. sensorReporter/mqtt/homie_conn.py at main · rkoshak/sensorReporter · GitHub

If you went down this path the advantage is that the Things would simply be disovered by OH and no configuration would be required outside of the MQTT broker Thing.

Once you have the Things, you can use “add equipment to model” to create the Items linked to the Thing’s Channels all in one go.

So from the end user’s perspertice the steps would be:

1. install and configure the python script
2. instal the MQTT binding
3. create and configure the MQTT Broker Thing
4. open the Inbox and you should already see the Ecoflow Thing present, accept it
5. From the Ecoflow Thing’s Channels tab, click “add equipment to model”, fill out the form and you are done.

In general, AI is not good with OH configs and OH rules.

@Matej_Kotnik

i wish i could make device templates that can be used to connect more similar devices with minimal effort like add device rename and done.

MainUI supports copy/paste/edit for Things and “add equipment to model” will create the Items very quickly. Obviously automatic discovery of the Things is even better. This is what can be done now. There are discussions (but no issues or PRs yet) to improve this process for OH 5. If anything comes of that so much the better.

@Matej_Kotnik

but it lacks putting all items in sitemap in one go,

That’s what the Group sitemap element is for. You put your Items into a Group then use the Group element with that Group Item and it will put all the members on the sitemap in one go. You are kind of stuck with the defaults though so if you need customizations you’ll have to add them manually one-by-one.

In MainUI, if you use the semantic model this happens automatically. As the name implies, “add equipment to model” uses the semantic model so if you create the Items that way, it will automatically appear in the locations, equipment, and properties tabs of the overview pages.

@Matej_Kotnik

lacks making MQTT things from MQTT broker data

If your python script followed one of the supported MQTT standard topic layouts:

• EspMilightHub Binding
• FPP Binding
  *…
• Home Assistant MQTT Components Binding
• MQTT Homie Binding
• Ruuvi Gateway MQTT Binding

MQTT - Bindings

then your devices will be automatically discovered and the Things representing those devices are automatically added to the Inbox.

@Matej_Kotnik

as that is what Home Automation is, i feel it is mandatory automating the integration of devices, then you do all the custom work on automating and inter operation.

That has been a major focus of OH sicne OH 2 and it has come a very long way over the years. But if you are using .things files et al you have basically chosen not to take advantage of any of it. Unless and until some major additions are made to the VSCode extension, to take advantage of automatic discovery of Things, rapid creation of Items, and automatic creation of the UI you must use a managed config using MainUI for your UI.

@Matej_Kotnik

but OH needs that exact task manager/rule manager tool to see what script uses what resources

It’s almost never the case that a rule causes performance problems with OH unless you’ve a bunch of busy wait loops, long sleeps, or stuff like that.

But you can change the logging level for openhab.event.RuleStatusInfoEvent to Info and when each rule starts running and when it goes back to idle will be added to events.log. You can match up the timestamps of those entries to see if you have any rules that seem to be taking an extra long time to run.

I think if you attached a Java profiler to the JVM running OH you can get down to the individual rule level too but I’ve never done that. But each rule is represnted as a set of independent Objects so it should be possible to figure out how much stuff it’s doing.

@Matej_Kotnik

just lots of data processing and Influx DB seems heavy on querries.

It should be possible to find those rules that do persistence calls at least.

@Matej_Kotnik

I am still migrating all the stuff to OH4,

All the stuff I mention above is mostly talking about what’s possible since OH 3 and there might be some things only supported by OH 4. As an OH 2.5 users, please do not skip the Getting Started Tutorial. The users who have the most trouble are those 2.5 and before users who skip it thinking “I already know how to use OH.” These users have even more trouble than brand new non-technical users of OH.

Note that even though Getting Started is UI focused, almost nothing presented there can only be done through the UI.

@Matej_Kotnik

The new N100 mini pc is running without Influx DB just on rrd4j but i wonder if rrd4j is wearing SSD more than influxDB, as i constantly agregating old data, that also meses up some discrete values, posibly i need to learn how to configure it in multy categories?

For an SDD I doubt that rrd4j is going to cause any more measurable wear on the SSD than InfluxDB. And it’s probably exactly the same if you’ve configured InfluxDB with retention policies. rrd4j does compression on a fixed scedule, not as a constant process.

But if you need discrete values instead of compressed data for old data, you need to use soemthing other than rrd4j.

@Matej_Kotnik

the new bindings like Luxtronik and more needed rewriting config code.

According to the docs this binding supports auto-discovery.

[Matej_Kotnik]

Wow, amazing insights!

Thanks Rich,

Actually i have to learn new MQTT conventions as that looks promising.

So on the project list it goes

1. MQTT reconect script
2. Standardize the MQTT protocol
3. Implement new integration
4. Test Ecoflow integration on OH4

The Main UI integration is actually good way to go, the only step in question is sitemap generation as that add group to sitemap seems missing on the VS Code i use, but seems obvious it should be existing.

Yay, you wrote so much great content
I have to reserve some time to apply it.

Cheers
Matej