Theengs Gateway - BLE to MQTT on a Raspberry Pi, Windows PC or Apple Mac

Add-ons Other
21

You can install the gateway on a server and connect it to an mqtt broker. And check if you get the data from your smartwatch.
If ok I can explain how to integrate it.

22

New version v0.5.0:

What’s Changed

New devices and changes:

Devices Model Measurements
April Brother ABTemp uuid/mfid/major/txpower @ 1 m/temperature/battery
April Brother N03 temperature/humidity/luminance/battery
Qingping CGDN1 temperature/humidity/PM2.5/PM10/carbon dioxide
iBeacon protocol uuid/mfid/major/minor/txpower @ 1 m/voltage
Thermobeacon WS02 temperature/humidity/voltage/timestamp/maximum temperature/maximum temperature timestamp/minimum temperature/minimum temperature timestamp
Thermobeacon WS08 temperature/humidity/voltage/timestamp/maximum temperature/maximum temperature timestamp/minimum temperature/minimum temperature timestamp

Full Changelog: v0.4.0
v0.5.0

23

Theengs Gateway is now available also as a docker container

docker pull theengs/gateway

Thanks to @maretodoric

24

I’m trying to use auto-discovery for Openhab3 with Theengs gateway (on an Ubuntu PC) and I can see the homeassistant config payloads are being advertized and then the Bluetooth devices show up as Things in the Inbox.
The problem is when I add these Things from Inbox, the linked Items don’t get updated (despite mqtt payloads coming) and the Thing itself is in the “UNKNOWN” state.

I can only see openhab.log swamped with such messages:
2022-10-13 17:29:58.686 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.fer | is_defined
2022-10-13 17:29:58.686 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.lux | is_defined
2022-10-13 17:29:58.686 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.moi | is_defined
2022-10-13 17:29:58.689 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.tempc | is_defined

Some relevant mqtt messages:

  • discovery:
Client (null) received PUBLISH (d0, q0, r1, m0, 'homeassistant/sensor/C47C8DXXXXXX-tempc/config', ... (438 bytes))
{"stat_t": "+/+/BTtoMQTT/C47C8DXXXXXX", "dev_cla": "temperature", "unit_of_meas": "\u00b0C", "name": "HHCCJCY01HHCC-tempc", "uniq_id": "C47C8DXXXXXX-tempc", "val_tpl": "{{ value_json.tempc | is_defined }}", "state_class": "measurement", "device": {"identifiers": ["C47C8DXXXXXX"], "connections": [["mac", "C47C8DXXXXXX"]], "manufacturer": "Xiaomi/VegTrug", "model": "HHCCJCY01HHCC", "name": "Flower care", "via_device": "TheengsGateway"}}
  • the actual payload:
Client (null) received PUBLISH (d0, q0, r0, m0, 'home/TheengsGateway/BTtoMQTT/C47C8DXXXXXXX', ... (170 bytes))
{"name": "Flower care", "id": "C4:7C:8D:XX:XX:XX", "rssi": -81, "brand": "Xiaomi/VegTrug", "model": "MiFlora", "model_id": "HHCCJCY01HHCC", "tempc": 21.9, "tempf": 71.42}

Please help to troubleshoot this further.

25

Hello,

This is due to the is_defined key below:
{{ value_json.tempc | is_defined }}

Here in the code, if you can replace:
device['val_tpl'] = "{{ value_json." + k + " | is_defined }}"
by
device['val_tpl'] = "{{ value_json." + k + "}}"

It should work

26

I’m using the snap package, so I guess this would have to be done by the maintainers of the package?

27

Let us add it to the new version

1 Like
28

A big release for Theengs Gateway, @DigiH did a huge work adding new devices and improving the existing ones!

Also, the TheengsGateway auto discovery following HA convention can now be compatible with OpenHAB thanks to the work of @koenvervloesem #62 , just set hass_discovery to 0 if you use OpenHAB or other Home Controllers.

I added documentation to the Theengs satellite/proxy feature that was available in the previous release. If you use OpenMQTTGateway web upload binary esp32dev-ble-mqtt-undecoded that will publish directly to home/<gateway name>/BTtoMQTT/undecoded making it directly compatible with Theengs Gateway MQTTtoMQTT decoding feature. In short, you can have satellites ESP32 sending undecoded data to TheengsGateway, and TheengsGateway centralizing the decoding work.

What’s Changed

New devices and changes:

Devices Model Measurements
ClearGrass CGG1 multi firmware support PVVX-ATC
Hydractiva Digital Amphiro/Oras sessions/time/litres/temperature/energy
Oria/Brifit/SigmaWit/SensorPro TH Sensor temperature/humidity/battery
Qingping CGPR1 presence/luminance/battery
ThermoPro TP357 temperature/humidity
ThermoPro TP358 temperature/humidity
ThermoPro TP359 temperature/humidity
Otio/BeeWi Door & Window Sensor contact/battery

Full Changelog: v0.5.0
v0.6.0

29

Hi @1technophile,

Sorry if my question is a bit outside this topic. I am trying to send data from my Xiaomi thermometers LYWSD03MMC to OpenHab through Mqtt using Theengs Gateway.

I have installed Theengs Gateway as a snap. When I launch it, I receive the following from mosquito_sub

Thermometer 1

home/TheengsGateway/BTtoMQTT/A4C138XXXXXX {"servicedatauuid": "fe95", "servicedata": "30585b050129fea538c1a4280100", "name": "LYWSD03MMC", "id": "A4:C1:38:XX:XX:XX", "rssi": -63}

Thermometer 2

home/TheengsGateway/BTtoMQTT/A4C138XXXXXX {"servicedatauuid": "fe95", "servicedata": "30585b050171189b38c1a4280100", "name": "LYWSD03MMC", "id": "A4:C1:38:XX:XX:XX", "rssi": -60}

I was expecting to receive a decoded version of servicedata with temperature and humidity values but I couldn’t find them.
I also tried to run Theengs Gateway after installation using pip.

I obviously did something wrong but I don’t know what and I cannot find information on the Internet about it. Can you please advise?

Many thanks in advance.

Ludovic

30

Hello,

That’s a good question.

LYWSD03MMC with their stock firmware use an encrypted key to advertize their data.

So as to make them compatible with Theengs or OpenMQTTGateway, you need to flash them with a custom firmware.

The process is very easy and takes less than 5 minutes.

The flash is done Over the air from your desktop
https://atc1441.github.io/TelinkFlasher.html

1 Like
31

Hi,

Thank you very much!

It is now working.

Ludovic

1 Like
32

New version available:

What’s Changed

New BLE devices and changes:

Devices Model Measurements
BlueMaestro TempoDisc 1 in 1 temperature/battery
BlueMaestro TempoDisc 4 in 1 temperature/humidity/pressure/battery
ClearGrass alarm clock CGD1 temperature/humidity/battery
ClearGrass alarm clock CGC1 temperature/humidity/battery
GOVEE H5074 temperature/humidity/battery
GOVEE H5101 temperature/humidity/battery
GOVEE H5106 PM2.5/temperature/humidity/battery
GOVEE H5174 temperature/humidity/battery
GOVEE H5177 temperature/humidity/battery
Oria/Brifit/SigmaWit/SensorPro TH Sensor T301 temperature/humidity/battery
Mopeka Pro temperature/level/sync status/voltage/battery/reading quality
Sensirion MyCO₂/CO₂ Gadget temperature/humidity/carbon dioxide
Sensirion SHT4X TH sensor temperature/humidity
ThermoPro TP393 temperature/humidity

Full Changelog: v0.6.6
v0.7.0

33

Is there a chance to add sync time for LYWSD03MMC ? It is possible display clock using PVVX-ATC firmware.

34

Hi @Dom_KS

Sync-time is in the works for LYWSD03MMC with PVVX firmware and should be included in an upcoming release.

1 Like
35

Hello. I want to use on Windows. Installed python 3 latest, installed Theengs Gateway from pip. Theengs Gateway asks for the name of the BLE adapter. I have Intel AX201 card with Intel Bluetooth. What name should I enter?

36

By default Theengs Gateway uses your system’s default Bluetooth adapter. Doesn’t it for you? What if you run it with the option -a "" to set an empty adapter name? If that still doesn’t work, can you show the output of running Theengs Gateway after enabling logging for Bleak?

1 Like
37

Hi @Dom_KS can you try whether your LYWSD03MMC with PVVX-ATC firmware is recognized by this tool and that you can set the time with it?

This is the library that will be integrated in an upcoming release of Theengs Gateway to expand the number of devices it can synchronize its time with. I added support for devices with the PVVX-ATC firmware, but I don’t have such a device, so it isn’t tested yet.

Try discovering clocks with:

bluetooth-clocks discover

If your device isn’t discovered, try with a longer scan duration, such as 10 seconds:

bluetooth-clocks discover -s 10

After a successful discovery (this is really needed, the MAC address alone if you know it already isn’t enough), take note of the MAC address, e.g. 12:34:56:78:9A:BC, and then synchronize the time with:

bluetooth-clocks set -a 12:34:56:78:9A:BC

Let me know whether this works, or if you get an error message or if there’s no error message but no time change either. You can add -v or -vv before the set subcommand to get more or even more information for debugging purposes. Also, if the time is already correct on the device, you can try setting it to another time to see whether the time change really works, e.f. with the option -t 2023-01-10T16:20.

Your feedback is really useful to improve time synchronization in Theengs Gateway.

38

Thank you. I thought that it is necessary to specify the name of the adapter. Everything works without a name :slight_smile:

39

Some qustion from your happy user on Windows Server 2022 / OpenHab 3.4.1

  1. Installation requires c++ compilation. How to change the build system from MSVC (takes up a lot of space on my mini PC - 6 gigabytes) to another compiler. Or how can I build on my workstation (I’m c++ developer) instead of a server? Now, I just installed everything on the workstation and copied it to the server along the same paths (whole python folder).
  2. Where can I read how to add “my devices”. I am interested in Xiaomi Scooter, Huawei Watch Fit 2 and various devices (sensors) from aliexpress.
  3. What python service wrapper do you recommend to run the gateway like the windows service (nssm?).
  4. How to exclude apple devices (0x004c - Apple,Inc ) from mqtt. There are a lot of them around me. {“manufacturerdata”: “4c0012020003”, “id”: “FD:C4:47:DA:4A:26”, “rssi”: -95}.
  5. How to exclude devices by id/mac (FD:C4:47:DA:4A:26) from mqtt. There are a lot of them around me. {“manufacturerdata”: “4c0012020003”, “id”: “FD:C4:47:DA:4A:26”, “rssi”: -95}.
  6. It would be convenient if at least the manufacturer of the device was specified in the MQTT payload (so as not to look at the directory).
{
	"vendor": "Apple, Inc",
	"manufacturerdata": "4c0012020003",
	"id": "FD:C4:47:DA:4A:26", 
	"rssi": -95
}
  1. Do you have plans to add useful properties for devices, such as: last seen (last activity) and availability (device is on, with custom poll period)
{
    "last_seen": "2023-01-22T16:15:42+05:00",
    "availability": "online"
}
  1. Ability to change the location of the settings file (theengsgw.conf) to any folder (to run as a service)
  2. SSL connection to mqtt broker (with certificates)
  3. Could not start Things Explorer on windows, reports that there is no ScrollView

rem python 3.11.1

pip install Textual
pip install bluetooth-numbers
pip install humanize
python TheengsExplorer/__init__.py

Thanks for the cool software, I have been looking for a long time how to attach BLE to OpenHab on Windows!!!

40
  1. Sorry, I don’t have any experience with it.
  2. There’s some information about adding decoders.
  3. I don’t have any experience with Windows services.
  4. Theengs Gateway has a --discovery_filter which filters iBeacons, GAEN and MS-CDP advertisements by default. I think that @Hans_Lree has been working on recognizing Apple Continuity messages, which you are seeing a lot, so once this is implemented in Theengs Decoder you can filter these in Theengs Gateway too.
  5. This isn’t possible, is this still necessary when the solution for 4. is implemented?
  6. For devices supported by Theengs Decoder there’s already a “brand” key. Where would the “vendor” information in your example come from?
  7. I think last seen and availability information should be something that your controller (openHAB, Home Assistant, 
) implements, based on the MQTT messages it receives from Theengs Gateway.
  8. Yes, we could change that Theengs Gateway would look in multiple locations instead of only the user’s home directory, that looks useful.
  9. Yes, this is a feature we need to implement.
  10. Theengs Explorer is not ready yet for general use. Have a look at this discussion.
1 Like