[Framework] Homie for ESP8266

So yes, I still don’t see the benefit! Instead of the ESP8266 ID, you’ll take the temp sensor ID, while at it, but I think we both agree there is no benefits doing this. :wink:

And thanks for the suggestion, but I try to keep the API as simple and clutterless as possible. Hope you understand. :slight_smile:

To be honest, the MQTT part is the most unstable, especially with SSL enabled. I’ve managed with the help of @jpmens to make it stable, but a single more subscription might crash the whole thing, so the PubSubClient object is made private. Later, if PubSubClient is more stable with SSL, I’ll maybe make it public.

Anyway, in my conception of a connected sensor, the controller is the only one having the notion of time. Whenever he wants to trigger something based on time, it sends an order to the sensor / actuator.

I see what you mean. In my case, I will have RTC chips in the modules but whitout a battery. so, to set a datetime, th server publishes a topic (/openhab/genera/datetime) on the MQTT so the RTC chips can be updated.

I use that way with touch screen thermostats and touch screen light switches

maybe i’ll try to add some code to subscribe to “none sensor” topic and see how that goes :slight_smile:

It’s quite easy actually to get a notion of time into a Homie sensor. I’m doing this as part of what I call a ping node (which may or may not be used for monitoring the device). Here’s a bit of code:

#include <Homie.h>
#include <TimeLib.h>

...
HomieNode pingNode("ping", "pong");
...

/*
 * message will contain a string which corresponds the epoch
 * time in seconds.
 */

bool epochHandler(String message)
{
        time_t tics = atol(message.c_str());

        if (tics > 0) {
                Serial.println("Setting epoch time");
                setTime(tics);
                return true;
        }
        return false;
}

void loopHandler() {
        /* periodically publish time, say once every 20s */
       Homie.setNodeProperty(pingNode, "epoch", String(now(), DEC), false);
}
...

void setup() {
        pingNode.subscribe("epoch", epochHandler);
        Homie.registerNode(pingNode);

        Homie.setLoopFunction(loopHandler);

        Homie.setup();
...
}

A daemon listens for messages published by Homie to sensors/homie/+/ping/epoch and checks the payload. If it is less than now() - 60s, I publish the current epoch seconds to the sending device at, say, sensors/homie/cf3a07e0/ping/epoch/set, whereupon the device sets its internal notion of time.

The ESP8266 appears to be quite reliable; I haven’t noticed more than a second loss in 24h.

The advantage of using an existing Homie node is, as @marvinroger says, that no extra subscribe is required.

Thanks, I think this could work :slight_smile:

Thanks for the tip, I will try it out

http://www.aliexpress.com/item/D1-mini-Mini-NodeMcu-4M-bytes-Lua-WIFI-Internet-of-Things-development-board-based-ESP8266-by/32529101036.html

http://www.aliexpress.com/item/Relay-Shield-for-WeMos-D1-mini/32596395175.html

1 Like

@marvinroger : I tested what you have in the esp8266 framework git hub as of (2016-03-29) and its working great… deleted the “device_id” in the JSON and it took the esp8266 mac address… also changed the “base_topic” in the MQTT section and that to worked as expected…

Continue your great work… and also those that helped/contributed :smile:

@marvinroger: Thank you for providing (and maintaining!) this really useful framework! I was in the process of implementing something along the same lines - although nowhere near as complete and refined - when I stumbled upon your work. Needless to say, I dropped my own work and switched to yours immediately, :slight_smile:

I have been playing around with a device (based on the NodeMCU v1.0 platform) for a couple of weeks now to get a feel for the framework, the stability, etc. before I start deploying several devices throughout my house. In addition to adding some external sensors like temperature, light, etc. I have also added some “internal sensors” to monitor the stability of the device. These sensors report the number of boots, the uptime (since last boot), the number of Wi-Fi and MQTT connections, etc.

Everything seems to be operating very smoothly and appears rock solid. The only thing I have noticed is that the MQTT connections seems to be dropping once in a while (i.e. my MQTT connect counter increases). The device has not rebooted and the Wi-Fi connection has not been lost (according to my counters).

Anybody else seeing this behavior? Anyone that have a (good) explanation for this? As the device autmatically reconnects and continues to operate smoothly the MQTT reconnects is obviously not a big concern for me, but still it would be good to understand what happens.

@KjetilA if you’re running SSL on your MQTT connexions then yes, its a known issue.

Yes, I did see the discussion around this (in the github issue list), however, I am not using SSL so I am afraid that is not the root cause.

Thanks! I will try to. :wink:

I do experience this behavior, and I have no idea of what is the problem. But I am pretty sure the issue is not coming from the framework itself, but from the Arduino for ESP8266 implementation or PubSubClient. If this is coming from Arduino for ESP8266, there are a lot of fixes in the to-be-released 2.2.0 version, so it will probably help.

I second this, as I have seen something similar with my own attempts before switching to Homie. Again, I was simply curious as to what could be the cause of this.

I’ll keep an eye open for this release, and cross my fingers.

Any chance you could create a pull request so Marvin can look at adding this to the Homie framework, sounds like a very worthwhile addition?

@ben_jones12: I’d be happy to share anything that may be usable, however, what I have done for now is not an extension/modification of the framework so there would not really be anything to pull. What I have done is simply to create a “system” node with the following properties: uptime, boot, wifi and mqtt.

Your comment, however, got me thinking that this could (should?) rather be part of the device properties of the framework.

@marvinroger: Let me know if you think this kind of information would be good to have as device properties, and I can certainly make a proposed implementation for you to review if you like. Or it may simply be quicker for you to do it yourself. The uptime, wifi counter and mqtt counters are obviously pretty straightforward. The only thing that required more than two lines of code is the boot counter since I used the EEPROM to make the counter persistent between boots. Anyway, let me know what your thoughts are on this.

I have been controlling lights with OpenHAB using two different off the shelf products: the Mi Light remote controlled bulbs (for ceiling etc. lights) and KanKun Wifi controlled sockets (for plug-in lamps). The former requires no hacking, just the OpenHAB binding; the latter requires a simple hack to re-program the socket to respond to http on and off requests. With mains voltages involved I prefer to spend a bit more for something that has been professionally designed into a package that is (hopefully at least) safe.

On the other hand I have been delighted by the use of the Homie framework with a single D1 mini to measure seven temperatures using 18B20 devices, one temperature and humidity using the DHT22 and the presence of moisture. After a few initial misunderstandings it didn’t take long to get everything going and to set up the alarms to warn me when something is going wrong with the heating (which until now has happened with depressing regularity!).

It’s a known fact that the ESP8266 software is not yet bulletproof, and the community is very fast at fixing bugs, so I am not worried. :slight_smile:

I am not sure the number of MQTT, Wi-Fi reconnects and boot would be an useful piece of information, except to test the robustness of the framework, as you did (let me know if you think this would be useful for you, and the reason why). However, I like the idea of having an $uptime device property, just like a server has.

The $uptime property would contain a number of seconds (the max value on Arduino being unsigned long, this allows for an uptime of up to 136 years, which I think is enough :smile:). But at which rate would it be refreshed? And what would be the actual value? The actual ESP8266 uptime or the uptime since the latest MQTT connection?

I completely agree that this information is mostly for testing the stability/robustness of the device (and not just the framework) and therefore may not be obvious candidates for device properties. I am happy to keep this as properties on “a standard Homie node” like have done. This is something that I keep on a compiler switch and that I will likely disable when I am confident that the device operates as expected.

Yes, I also think this one is a definite candidate for a device property, and I would very much like if this was taken into the “official framework”.

Yes, that is excatly what I have done. Before publishing to MQTT, however, I run the internal uptime (in seconds) through a formatter function that converts it to a string on the form:

12 days 13:45

Meaning the system has been up for 12 days, 13 hours and 45 minutes. Obviously this conversion could (and maybe should?) be done on the “other side of the MQTT link”, in my case on the openHAB server.

Given that I publish a string (as shown above) with the resolution set to minutes, I refresh the value once a minute only. If the uptime is published as the “raw value” (number of seconds) instead, then maybe a refresh rate of 15 seconds is OK?

I strongly believe this should be the true uptime of the system (device), meaning the number of seconds since the system was last started/booted.

With this in mind, one could also consider other types of “system information” like memory usage, CPU usage, etc.

Any thougths on this? It could be good to also get information about the amount of memory used (in %) as well as the CPU load (again in %). Memory usage, I guess can, be retrieved by some library function, but the CPU load I am not so sure about. Is there some way of measuring the amount of “idle time” in the framework, or better in the underlying “operating system”?

Yes, the data emitted by the sensor must be as simple as possible. :wink:

I guess 10 bytes more or less is not really a big deal, when you see the bandwidth the ESP8266 can handle. I would go for a 1 minute resolution, we don’t need 15sec resolution here, the sensors are meant to be up for days, weeks, months… So 1 minute resolution seems to be a good compromise.

The only thing that can be measured at runtime is the available heap, with system_get_free_heap_size(). But this is not a reliable indicator as you can have 20k of available heap, but because of the fragmentation, maybe the biggest memory block available for you to alloc on will only be 1k.

I am not aware of a way to measure CPU load in the SDK. The only possible solution I see would be to check the interval between two Homie.loop(). I measured a blank sketch takes about 5ms (let’s say it represents 1% of load), and at some points it might take up to about 500ms (let’s say 100%). So the percentage of load would be load = (100 * interval_in_ms) / 500. It is flawed and not reliable at all.

OK. I just briefly saw that there were some libraries/functions (e.g. freeMemory) available, but did not look at the details. I think this kind of information would be mostly interesting for the purpose of detecting a possible memory leak, and as such falls into the category of “debugging utilities” and not something that should need to be part of the framework.

To be honest, something like this was really what I was thinking about. I did some work in my setup on timing intervals between loops, and reporting the minimun, maximum and average values. My thinking was that for certain use cases it could be good to know how often one could expect some code to be executed (e.g. the polling of a sensor, or whatever).

Maybe it could also be turned around in the sense that it may be so that for the framework (or probably functions that the framework rely on, e.g. Wi-Fi, MQTT, SSL, etc.) to work reliably the Homie.loop must be called at a certain minimun interval and this could be a way to check that you are not getting close to a critical limit.

Again, this may be purely for debugging and thus not to be an integral part of the framework?

Honestly, I think this is superfluous. This is the kind of data that very few people will care about, and as you said, this is easy to implement on the sketch level. If this would be costless, I would implement it, but every single thing the framework has to handle costs processing time, which means less stability, more power consumption, so more cost, etc. :slight_smile: