ESP32 based BLE, 433mhz & IR gateway bidirectional with MQTT --> OpenMQTTGateway

Hardware Home Automation
62

Hi Dom,

For such combination I suggest that you build it yourself with either platformio or arduino IDE.
This way you will get something more adapted to your need.

63

I’ve alredy done that but I’ve got a problem probably with ZmqttDiscovery :frowning:
In my builds it isn’t working and I have no idea why :frowning:
In your build nodemcuv2-rf gateway is publishing 2 topics :

  1. home/OpenMQTTGateway/433toMQTT
    and
  2. homeassistant/… (I’ve forrget save log with this topic )

In my builds I’ve got only first one . Should I do something more than uncomment #define ZmqttDiscovery in User_config.h to make it work?

Could you send/paste User_config.h from your bulids ( nodemcuv2 RF and nodemcuv2 ALL ) ?

BTW. Is it possible to make together RF+RF2+IR+DHT+HADiscovery using nodemcu?

64

@1technophile
Probably I’ve got similar problem :

I set
#define MQTT_MAX_PACKET_SIZE 512
Into pubsubclient.h

And now it work !

65

Great !

66

Just added to OMG:

and

As a part of many feature available for tests on v0.9.4beta

67

v0.9.4 is out for info. It adds the following features/fixes:

Under the hood:

Thanks to the numerous contributors to this version!

Breaking changes:

  • Home assistant discovery of luminance value is now with lx unity instead of lu
  • actuatorONOFF API has been modified, see docs, pin is replaced by GPIO
  • all system attributes published to SYStoMQTT are now lower case, example:
{"uptime":260,"version":"version_tag","freemem":50012,"rssi":-30,"SSID":"omgap","ip":"192.168.1.23","mac":"24:0A:C4:24:90:98","wifiprt":0,"lowpowermode":0,"interval":55555,"modules":"BTHADiscovery"}

You can find it below:

68

v0.9.5 is out, I hope you will enjoy it,
A big thank you to @jmw6773 & @Legion2 for their numerous contributions!

–Here is the content–
New devices:

New boards:

New features:

Aside from the code:

  • A new way to retrieve the compatible devices, boards and parts, through a website

Under the hood:

Breaking:

Deprecate - items that will be removed on the next release:

1 Like
69

Big release time, I hope you enjoy it!

This one brings a lot of features and improvements.

  • In particular the support of RTL_433 library to ESP32 thanks to @NorthernMan54. This is a huge addition to the community as this library enables to support a lot of new protocols. More than 75 433Mhz protocols supported with an ESP32 + a CC1101!
    Here is an example with a weather station:
home/OpenMQTTGateway_RTL_433_ESP/RTL_433toMQTT

{"model":"WS2032","id":38553,"battery_ok":1,"temperature_C":22.3,"humidity":65,"wind_dir_deg":90.0,"wind_avg_km_h":0,"wind_max_km_h":0,"rain":256,"flags":0,"mic":"CRC","protocol":"WS2032 weather station","rssi":-52,"duration":511982}
  • Also, @csiki2 improved the BLE gateway by adding a queue mechanism, bringing more stability to the gateway, and improving the number of BLE packets transmitted. Thanks to this, the gateway can now be configured for the continuous scan (TimeBtwRead:0, Scan_duration:1000) and catch easily BLE PIR and door/window sensors events.

But that’s not all, let’s take a deeper look at all the new devices, boards, and features!

New devices:

New boards:

New features:

Under the hood:

Docs:

Breaking:

Deprecate - items that will be removed on the next release:

  • SimplePublishing and simpleReceiving methods will be removed see why here
1 Like
70

Release time!

The 0.9.7 brings a lot of features and improvements to BLE and TLS functions.

  • The support of a generic BLE connection to devices, you can now trigger a READ/WRITE to a BLE device through your ESP32 with MQTT. The gateway will initiate a connection to the device and acts on the service and characteristic given. For a value reading or writing. This opens a lot of integration possibilities, among others the control of actuators, valves, light, relays…
    It could also be a good way to test a value retrieval with a BLE connection before asking for a native integration into the gateway.

Example MQTT message format:

mosquitto_pub -t home/OpenMQTTGateway/commands/MQTTtoBT/config -m '{ "ble_write_address":"AA:BB:CC:DD:EE:FF", "ble_write_service":"cba20d00-224d-11e6-9fb8-0002a5d5c51b", "ble_write_char":"cba20002-224d-11e6-9fb8-0002a5d5c51b", "ble_write_value":"TEST", "value_type":"STRING", "ttl":4 }'

  • Efforts have been made around the security connection functions, the TLS connection can now be configured within the configuration web page by adding the server root certificate (copy&paste) and checking a box.
    You can alternatively use self-signed certificates and even several sets of them, if you want to handle several MQTT servers. With this comes also the ability to change the MQTT server by MQTT, with automatic return to the previous configuration if not working.

  • So as to make the update of the gateway you are now able to trigger it through MQTT, if the version is different than the current one and if the password is OK, the gateway will download and install the linked firmware.

mosquitto_pub -t "home/<gateway_name>/commands/firmware_update" -m '{ "version": "test", "password": "OTAPASSWORD", "url": "https://github.com/1technophile/OpenMQTTGateway/releases/download/v0.9.6/esp32-m5stack-ble-firmware.bin" }'

But that’s not all, let’s take a deeper look at all the new devices and features!

New devices:

New features:

Under the hood:

Breaking:

Docs:

Deprecate - functions that will be removed or modified on the next release:

  • The default gateway name for ESP will be generated automatically from the MAC address per default, OpenMQTTGateway in the AP name and the topic will be replaced:
    AP Name: OMG_112233445566
    Topic: home/112233445566/BTtoMQTT
    Setting the Gateway_name will not be needed anymore unless you want to do it explicitely.
  • SimplePublishing and simpleReceiving methods will be removed see why here
71

V0.9.9 is out !

What’s Changed

Among new devices and boards this release brings some exciting features like the capability to connect and control a BLE device like a switchbot, here is a sample command for the SWITCHBOT S1:

{
  "ble_write_address": "FF:AA:BB:FF:DD:EE",
  "ble_write_service": "cba20d00-224d-11e6-9fb8-0002a5d5c51b",
  "ble_write_char": "cba20002-224d-11e6-9fb8-0002a5d5c51b",
  "ble_write_value": "570100",
  "value_type": "HEX",
  "ttl": 4,
  "mac_type":1,
  "immediate": true
}

OMG has been supporting for a long time reading of BLE broadcasters, we can also read data through a BLE connection but being able to act on a BLE device is something new. Opening a lot of possibilities and use cases like controlling BLE actuators, lights, valves, plugs…
Feel free to test and share your snippets of OMG command for the others!
Do you like BBQ? So get ready for the sunny months and set up your meat cooking monitoring thanks to 3 Inkbird BBQ sensors supported.
You are a professional looking for an ESP32 firmware that can integrate to AWS and send sensor data, OMG now supports this type of integration.
But that’s not all, here is the exhaustive list of improvements!

NEW BOARDS:

NEW DEVICES:

  • ThermoBeacon WS08
  • ThermoBeacon WS02
  • BlueMaestro TempoDisc
  • INKBIRD 6XS and 2X BBQ sensor support
  • GOVEE H5075, H5072 and H5102

NEW FEATURES:

FIX:

BREAKING:

  • [BREAKING] Change unique id structure for HASS discovery by @1technophile in #1147
    You will see duplicate entities if you use HASS autodiscovery, explanation here
  • [BREAKING] Differentiate WM version by @1technophile in #1148
    If you are using your own PlatformIO environment you will need to add the link to the suitable wifimanager library directly into your environment library list:
lib_deps =
  ${libraries.wifimanager8266}

or

lib_deps =
  ${libraries.wifimanager32}

MISCELLEANOUS:

New Contributors

Full Changelog: v0.9.8…v0.9.9

72

v0.9.12 is out!

@h2zero integrated natively SwitchBot S1 BLE control to this release, the switch is auto-discovered like all the other BLE sensors with OMG.

What’s Changed

New Contributors

Full Changelog: v0.9.11…v0.9.12

1 Like
73

v0.9.15 is out:

New sensors:

Others:

Full Changelog: v0.9.14…v0.9.15

74

An exciting release V1.0.0 for OpenMQTTGateway, with 3 major changes:

  • @NorthernMan54 added the support of the HELTEC LORA 32 433Mhz with RTL_433 :signal_strength:. Isn’t it huge?:partying_face:
    No need to solder or check pins. Just upload heltec-rtl_433 from the web upload to the board, and you will benefit from the most powerful 433mhz library decoding, RTL_433 , we support Pulse Position Modulation (OOK_PPM) and Pulse Width Modulation (OOK_PWM) as a start (this is 80 decoders🙂).
    A great way to pick up weather stations, sensors, door bells… the list of compatible devices is huge. Feel free to share your feedback in the forum.
    image
  • Added to the 433mhz, @DigiH did a huge work on the Bluetooth decoder, adding many new devices and improving the existing ones!
    The decoder now supports more than 55 devices!
    You can enjoy this changes with OpenMQTTGateway, but also with the TheengsGateway, the Theengs App will be updated soon.
    Note that Theengs Gateway can also work with OpenMQTTGateway as satellites/proxies. See our MQTTtoMQTT function implemented in June in OMG.
  • Thanks to @BadWolf42, we can now persist the BLE configuration to don’t rely only on the broker-retained values

And plenty of other changes thanks to the help of our contributors.

What’s Changed

New BLE devices and changes:

Devices Model Measurements
Amazfit Smart Watch/Band steps, activity heart rate (when activated in the Zepp Life settings)
ClearGrass CGG1 multi firmware support PVVX-ATC
Hydractiva Digital Amphiro/Oras sessions/time/litres/temperature/energy
April Brother ABTemp uuid/mfid/major/txpower @ 1 m/temperature/battery
April Brother N03 temperature/humidity/luminance/battery
ClearGrass Clock LYWSD02 temperature/humidity/battery
Oria/Brifit/SigmaWit/SensorPro TH Sensor temperature/humidity/battery
Qingping CGPR1 presence/luminance/battery
Qingping CGDN1 temperature/humidity/PM2.5/PM10/carbon dioxide
RDL52832 mfid/uuid/minor/major/txpower @ 1 m/temperature/humidity/acceleration
RBaron b-parasite moisture/temperature/humidity/luminance (v1.1.0+)/voltage
SmartDry Laundry Sensor temperature/humidity/shake/voltage/wake
Switchbot Motion Sensor movement/light level/sensing distance/led/scope tested/battery
Switchbot Contact Sensor contact/movement/scope tested/light level/battery
Switchbot Curtain motion state/position/light level/battery/calibration state
Thermobeacon WS02 temperature/humidity/voltage/timestamp/maximum temperature/maximum temperature timestamp/minimum temperature/minimum temperature timestamp
Thermobeacon WS08 temperature/humidity/voltage/timestamp/maximum temperature/maximum temperature timestamp/minimum temperature/minimum temperature timestamp
ThermoPro TP357 temperature/humidity
ThermoPro TP358 temperature/humidity
ThermoPro TP359 temperature/humidity
Oria/Brifit/SigmaWit/SensorPro TH Sensor temperature/humidity/battery
Otio/BeeWi Door & Window Sensor contact/battery

Features and fixes:

Breaking

New Contributors

Full Changelog: v0.9.16…v1.0.0

1 Like
75

OpenMQTTGateway V1.2.0 released!

What’s Changed

This one is huge. More boards supported, more devices, and some exciting changes to make your life easier :confetti_ball:
The auto-discovery is seeing new improvements to have better control of the gateway, but in particular, it now supports RTL_433 ESP32 !!
So with one binary flash in less than 5 minutes, you can retrieve your 433Mhz sensors, and OMG will automatically create the devices into your favorite Home Automation controller.

Added to this, you can also see the RTL_433 sensor messages on the screen. Thanks @NorthernMan54 .
image

We also reached a significant milestone of the Theengs Decoder with V1.0.0 thanks to @DigiH, more than 70 BLE devices supported :rocket:
New boards:

New BLE devices and changes:

Devices Model Measurements
BlueMaestro TempoDisc 1 in 1 temperature/battery
BlueMaestro TempoDisc 4 in 1 temperature/humidity/pressure/battery
ClearGrass alarm clock CGD1 temperature/humidity/battery
ClearGrass alarm clock CGC1 temperature/humidity/battery
GOVEE H5074 temperature/humidity/battery
GOVEE H5101 temperature/humidity/battery
GOVEE H5106 PM2.5/temperature/humidity/battery
GOVEE H5174 temperature/humidity/battery
GOVEE H5177 temperature/humidity/battery
Oria/Brifit/SigmaWit/SensorPro TH Sensor T301 temperature/humidity/battery
Mopeka Pro temperature/level/sync status/voltage/battery/reading quality
Sensirion MyCO₂/CO₂ Gadget temperature/humidity/carbon dioxide
Sensirion SHT4X TH sensor temperature/humidity
ThermoPro TP393 temperature/humidity

Auto-discovery:

  • [Discovery] - Transform switch discovery to button when relevant by @1technophile in #1356
  • [Discovery] - Add numbers for interval and scans netween connects by @1technophile in #1357
  • [Discovery] Save the BLE settings and change switch to optimistic false by @1technophile in #1364
  • [Discovery] Remove auto discovery switch by @1technophile in #1365
  • [BREAKING] Add RTL_433 discovery with Home Assistant convention by @1technophile in #1379 , if you add sensor for RTL_433 OMG you will have to update your topic structure as it changed with this PR.

Features:

Miscellaneous:

Documentation:

New Contributors

Full Changelog: v1.1.1…v1.2.0

3 Likes
76

I’m happy to share with you the launch of Theengs Plug

The first product powered by OpenMQTTGateway out of the box.

The plug is available for North American customers and can:

  • Be controlled remotely
  • Act as a Bluetooth Gateway to retrieve data from sensors
  • Measure the energy consumption of the plugged device

Here are the characteristics:

Power Supply 100-120VAC, 60Hz
Max current 15A
Microcontroller ESP32
WiFi Band 2.4 GHz
Protocol MQTT
Operating Humidity 0%～95% (No condensation)
Operating Temperature 0 ºC ~ 40 ºC
Storage Temperature -10°C ~ 50°C
Dimensions 4.05in * 1.37in * 2.4in/103mm * 34.6mm * 61mm

Theengs Plug hardware is UL certified.

The plug will be preloaded with OpenMQTTGateway and will support over 50 BLE devices.

This is also a way you can support the OpenMQTTGateway project!

1 Like
77

New release v1.4.0

Adaptive scanning

This release brings a very interesting feature for Bluetooth Low Energy scanning.
You may have already asked yourself, what should I set for the interval between my scans? Do I use passive or active scanning? Or how do these parameters impact the battery life of my sensors?
To remove these questions, @DigiH and I have integrated an innovative function - Adaptive scanning

Adaptive scanning will decide automatically following your devices if you need to use passive/active/continuous scanning.

For example, a door or a PIR sensor will require continuous scanning, so if detected, the gateway will reduce its time between scans to the minimum. Or your devices may also require active scanning to retrieve data. The gateway will trigger active scans at regular intervals in this case.
Behind the scenes, a big work on device categorization and analysis has been done in Theengs Decoder to enable this feature.

Why are we doing this?
Increase your sensors’ battery life, decrease network traffic, and reduce gateway power consumption.

This new feature is activated by default, you can deactivate it, and the gateway will go directly into continuous active scanning.
Consequently, the esp32ble-dev-cont environment has been merged into esp32ble-dev, and the “Active scan” command has been removed. You can remove the topic from your broker to delete it from the controller.

Theengs Plug

Also, this release brings support for the second RGB Led of the Theengs Plug alongside the capability to reset at the start. I encourage Theengs Plug users to update following this guide.

New boards

New BLE devices

Devices Model Measurements
Polar H10 Chest strap activity heart rate

Features

Docs

Miscellaneous

New Contributors

Full Changelog: v1.3.0…v1.4.0

78

What’s Changed

Introducing the latest update for OpenMQTTGateway! With over 75 changes done in just 40 days, this release is packed with exciting new features and improvements. One of the most noteworthy updates is the automatic creation and support of device trackers for presence detection, including popular devices like NUT, Tile, Mi Band, Amazfit, and RuuviTag.
Plus, the support for direct Over the Air updates from the controller means that you can easily stay up-to-date with future releases.

That’s not all - we also made some significant changes to the SSD1306, allowing you to control it directly from the controller or by MQTT commands and even use it to display BLE sensor data. And, with the new logo acting as a screen saver, you can reduce screen burn-in while still enjoying all the information shown.
Also, the RTL_433 library has been updated to provide more sensitivity and support new devices. To avoid creating unnecessary entities, the auto-discovery control is available from the controller and will now work as a pairing mode. It will be automatically deactivated after 30 minutes after its activation.

Finally, the Theengs plug can now retain its state in case of power outages or restart.

Theengs Plug

  • Create in use binary sensor for RN8209 Current sensor by @NorthernMan54 in #1474
  • [ONOFF/RN8209] ESP32 Attempt to recover previous actuator state when power has been lost by @1technophile in #1494

New devices or new data published

New boards

New features

Miscellaneous

Docs

New Contributors

Full Changelog: v1.4.0…v1.5.0

2 Likes
79

What’s Changed

Significant features in this release include a webUI to control your gateway and directly see the sensor’s data, access logs, and change the basic configuration without a controller!

OpenMQTTGateway-webUI

Go to your gateway IP and type admin/OTAPASSWORD. The password can be changed from the Wifi Manager portal.
Thanks, @NorthernMan54, for this add and to Tasmota as they inspired this feature and the code.
Note that the WebUI is unavailable with the Theengs plug in this version.

Also, now we can detect the appliance cycle start and end by leveraging the accelerometers of BLE beacons. The beacon needs to be configured to stop advertising when there is no movement, and the gateway will automatically publish a message to the broker to say that the beacon is offline. This is great for washing machines, dryers, or dishwashers’ end-of-cycle announcements and notifications.

Finally, OpenHAB users can now use any build with autodiscovery. The specificities of OpenHAB HA convention processing can be set at runtime with an MQTT command

Theengs Plug

  • The Theengs Plug will now detect changes of current superior to 0.1A and publishes a message to the broker to speed up the detection when a connected device is powered ON. This is useful to trigger actions when the plugged device starts.
  • We also added a task watchdog to restart the plug if the energy monitoring and overcurrent detection task is stuck
  • A FailSafe Mode also enables loading the wifi manager portal with a minimal setup.

New devices

New features

Miscellaneous

Documentation

New Contributors

Full Changelog: https://github.com/1technophile/OpenMQTTGateway/compare/v1.5.0...v1.6.0enMQTTGateway/compare/v1.5.0...v1.6.0

2 Likes
80

OpenMQTTGateway v1.7.0

What’s Changed

1 - New product We released the Theengs Bridge, an ESP32 based BLE gateway with an external antenna and an Ethernet RJ45 port. If you want to extend the range of your BLE network, the Theengs Bridge will help you reach further sensors than traditional embedded antennas. We used this opportunity to improve the support of Ethernet boards. You can onboard them through the WiFi portal and use Ethernet with WiFi as a backup network. With this evolution, there is no need to put your hands on a development environment or a configuration file!

image

Support our work by purchasing from our store !

2 - More RF devices supported OOTB Our users requested to decode RTL_433 FSK ( FSK is a radio frequency modulation) without building the environment. We now have Lilygo and Heltec environments supporting FSK (beta) from the web upload. And cherry on the cake, you can change the frequency directly from the WebUI. But that’s not all. Users with a CC1101 will be pleased to see that they can change the frequency of their devices and the library they use from the WebUI!

image

3 - More BLE devices supported Over 90 BLE devices are now supported by OpenMQTTGateway and Theengs; look at this impressive devices list!

4 - DIY :hammer_and_wrench: One of my favorite radio communication is LoRa. We added the capability to change the frequency, numerous parameters from the WebUI, and a new ESP32 temperature sender example #1742 . A great way to play with the protocol and build your nodes.

image

5 - Stability :weight_lifting_man:BLE is not forgotten with a new, more stable controller. We also added a central queueing mechanism.

6 - User experience We simplified password management; now, you will have only one password for the WebUI, OTA, and WiFi portal connection. This password is defined during WiFi portal onboarding and is compulsory. If you upgrade from a previous version, the default password for the WebUI, the onboarding, and OTA will be OTAPASSWORD.
I invite you strongly to change it in the WebUI (Configure Gateway).
Also, at the first connection, the WiFi portal no longer needs a password (bye-bye :wave: “your_password”).

7 - Integration Finally, @DigiH improved the discovery experience by tuning the Home Assistant MQTT discovery integration.

Theengs Plug

You can now change the brightness of the Theengs Plug LED with a slider. Suppose you use the plug in your bedroom; for example, you can program the LED to be OFF at night.
The BLE gateway can be deactivated if you want to use it solely as a smart plug.

New BLE devices

Devices Model Measurements
April Brother N07 temperature/humidity/battery/packet ID
BM6 Battery Monitor BM6 battery
GOVEE H5100 temperature/humidity/battery
GOVEE H5104 temperature/humidity/battery
GOVEE H5179 temperature/humidity/battery
INKBIRD IBS-TH12S temperature/humidity/battery
NUT NUTALE tracker
Oral B Toothbrush state, mode, sector, pressure, time
Lippert BottleCheck temperature/level/sync status/voltage/battery/reading quality/acceleration x/y-axis
ShellyBlu Button1 button press type/battery/packet ID
ShellyBlu Door/Window contact/rotation/battery/packet ID
ShellyBlu Motion motion, illuminance, battery, packet ID
ThermoPro TP357s temperature/humidity
  • Support for PVVX encryption
  • Add mac extraction from data
  • Add new devices as presence trackers

Board support

New features

Breaking changes

Miscellaneous

Documentation

Will be removed in the next version

  • SimplePublishing MQTT API
  • Support for Arduino UNO, NANO, MEGA
  • Support of Arduino IDE as a development environment

New Contributors

Full Changelog: v1.6.0…v1.7.0

2 Likes