OpenMQTTGateway BLE on a Raspberry Pi, Windows PC or Apple Mac - Theengs Gateway!

Dom_KS · January 23, 2023, 5:45pm

There is something wrong with my Rpi

openhabian@openhabian:~ $ sudo sudo python3 -m pip install bluetooth-clocks
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bluetooth-clocks
  Downloading https://files.pythonhosted.org/packages/1b/56/c33b5aac4fd85626a6b31c888bf4bde777324ab52eca0b0d837f65d7e887/bluetooth_clocks-0.1.0-py3-none-any.whl
Collecting bleak>=0.19.0 (from bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/44/be/da0de00c7c66c14c0ccaa5927abd048927f93d9c7ed2598d99549d7f9109/bleak-0.19.5-py3-none-any.whl (132kB)
    100% |████████████████████████████████| 133kB 896kB/s
Collecting importlib-metadata; python_version < "3.8" (from bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/26/a7/9da7d5b23fc98ab3d424ac2c65613d63c1f401efb84ad50f2fa27b2caab4/importlib_metadata-6.0.0-py3-none-any.whl
Collecting async-timeout<5,>=3.0.0 (from bleak>=0.19.0->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/d6/c1/8991e7c5385b897b8c020cdaad718c5b087a6626d1d11a23e1ea87e325a7/async_timeout-4.0.2-py3-none-any.whl
Collecting typing-extensions<5.0.0,>=4.2.0; python_version < "3.8" (from bleak>=0.19.0->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/0b/8e/f1a0a5a76cfef77e1eb6004cb49e5f8d72634da638420b9ea492ce8305e8/typing_extensions-4.4.0-py3-none-any.whl
Collecting dbus-fast<2.0.0,>=1.22.0; platform_system == "Linux" (from bleak>=0.19.0->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/81/67/a83522d6692a72911c3586e2fcd2363ce22d1522a7e17e1012f255a6083b/dbus_fast-1.84.0.tar.gz (65kB)
    100% |████████████████████████████████| 71kB 2.2MB/s
  Installing build dependencies ... done
Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->bluetooth-clocks)
  Using cached https://files.pythonhosted.org/packages/d8/20/256eb3f3f437c575fb1a2efdce5e801a5ce3162ea8117da96c43e6ee97d8/zipp-3.11.0-py3-none-any.whl
Building wheels for collected packages: dbus-fast
  Running setup.py bdist_wheel for dbus-fast ... \                                                 done
  Stored in directory: /root/.cache/pip/wheels/8f/56/85/2d82b076cf6e733315e4cc4f922055f3f9a47c952b55264659
Successfully built dbus-fast
Installing collected packages: typing-extensions, async-timeout, dbus-fast, bleak, zipp, importlib-metadata, bluetooth-clocks
Successfully installed async-timeout-4.0.2 bleak-0.19.5 bluetooth-clocks-0.1.0 dbus-fast-1.84.0 importlib-metadata-6.0.0 typing-extensions-4.4.0 zipp-3.11.0
openhabian@openhabian:~ $
openhabian@openhabian:~ $ bluetooth-clocks discover
Traceback (most recent call last):
  File "/usr/local/bin/bluetooth-clocks", line 6, in <module>
    from bluetooth_clocks.__main__ import run
  File "/usr/local/lib/python3.7/dist-packages/bluetooth_clocks/__main__.py", line 12, in <module>
    from bluetooth_clocks.scanners import discover_clocks, find_clock
  File "/usr/local/lib/python3.7/dist-packages/bluetooth_clocks/scanners.py", line 16, in <module>
    async def find_clock(address: str, scan_duration: float = 5.0) -> BluetoothClock | None:
TypeError: unsupported operand type(s) for |: 'ABCMeta' and 'NoneType'
openhabian@openhabian:~ $ sudo bluetooth-clocks discover
Traceback (most recent call last):
  File "/usr/local/bin/bluetooth-clocks", line 6, in <module>
    from bluetooth_clocks.__main__ import run
  File "/usr/local/lib/python3.7/dist-packages/bluetooth_clocks/__main__.py", line 12, in <module>
    from bluetooth_clocks.scanners import discover_clocks, find_clock
  File "/usr/local/lib/python3.7/dist-packages/bluetooth_clocks/scanners.py", line 16, in <module>
    async def find_clock(address: str, scan_duration: float = 5.0) -> BluetoothClock | None:
TypeError: unsupported operand type(s) for |: 'ABCMeta' and 'NoneType'
openhabian@openhabian:~ $

koenvervloesem · January 23, 2023, 6:48pm

No, there’s nothing wrong with your Raspberry Pi, the error was all mine. I published a bugfix release. Can you try again with the updated version?

python3 -m pip install -U bluetooth-clocks

maxin · January 27, 2023, 8:42am

Thank you, for answer!

Thanks I got it. As I understand it, the gateway can only process received states, but cannot send (manage)? For example, a Xiaomi scooter requires sending a request for data, and only then sends the required data.
Wrapped python into a service (with virtual environment) using WinSW

<service>
  <id>openhabblemqttbridge.exe</id>
  <name>OpenHAB BLE MQTT Bridge</name>
  <description>Python TheengsGateway BLE MQTT Bridge</description>  
  <executable>r:\servers\python\ble2mqtt\Scripts\python.exe</executable>
  <argument>-m</argument>
  <argument>TheengsGateway</argument>
  <argument>-ll</argument>
  <argument>ERROR</argument>
  <argument>-s</argument>
  <argument>passive</argument>
  <argument>-sd</argument>
  <argument>5</argument>
  <argument>-tb</argument>
  <argument>55</argument>
  <argument>-H</argument>
  <argument>127.0.0.1</argument>
  <argument>-P</argument>
  <argument>1883</argument>
  <argument>-u</argument>
  <argument>MYLOGIN</argument>
  <argument>-p</argument>
  <argument>MYPASSWORD</argument>
  <argument>-pt</argument>
  <argument>ble2mqtt/device</argument>
  <argument>-st</argument>
  <argument>ble2mqtt/device/set</argument>
  <argument>-pa</argument>
  <argument>1</argument>
  <argument>-D</argument>
  <argument>1</argument>
  <argument>-Dh</argument>
  <argument>0</argument>
  <argument>-Dt</argument>
  <argument>ble2mqtt/discovery</argument>
  <argument>-Dn</argument>
  <argument>bledev</argument>
  <logmode>rotate</logmode>
  <stoptimeout>30sec</stoptimeout>
  <serviceaccount>
    <domain>MYSERVER</domain>
    <user>LocalSystem</user>
    <password></password>
  </serviceaccount>
  <workingdirectory>r:\servers\python\ble2mqtt</workingdirectory>
  <env name="VIRTUAL_ENV" value="r:\servers\python\ble2mqtt"/>
  <env name="PROMPT" value="(ble2mqtt) $P$G"/>
  <env name="PATH" value="r:\servers\python\ble2mqtt\Scripts;%PATH%"/>  
</service>

4-5.It would be convenient if we could specify any id, not just those for which the author’s parsers were written. In general, it seems to me that it would be cool to use not only Deny Filter, but also Allow Filter (id/vendor).
6. I didn’t put it that way, of course “brand”. I would like that “brand” was published for any device, and not just for those that the gateway can work with. Of course, if it is specified in manufactordata or servicedata (most devices have this data).
7. I implemented lastseen and availability via OpenHab rules. But a built-in feature would be much more convenient.

For MQTT channel

  - id: lastseen
    channelTypeUID: mqtt:datetime
    label: Last seen
    configuration:
      stateTopic: ble2mqtt/device/FF0727A00CAF
      transformationPattern: JS:device\device_lastseen.js
  - id: availability
    channelTypeUID: mqtt:contact
    label: Availability
    configuration:
      stateTopic: ble2mqtt/device/FF0727A00CAF
      transformationPattern: JS:device\device_availability.js
      off: offline
      on: online

// device_availability.js

(function(i) {
	return "online";
})(input)

// device_lastseen.js
(function(i) {
	function pad(num){
		var norm = Math.floor(Math.abs(num));
		return (norm < 10 ? '0' : '') + norm;
	};

	var date =  new Date();
	var tzOffset = -date.getTimezoneOffset();
	var plusOrMinus = tzOffset >= 0 ? '+' : '-';	
	
	return date.getFullYear() + '-' + pad(date.getMonth() + 1) +'-' + pad(date.getDate()) + 'T' + pad(date.getHours()) + ':' + pad(date.getMinutes()) + ':' + pad(date.getSeconds()) + plusOrMinus + pad(tzOffset / 60) + ':' + pad(tzOffset % 60);	
})(input)

And a rule that runs once a minute and changes states for devices that are more than 10 minutes offline

// include
import java.time.format.DateTimeFormatter
import org.openhab.core.model.script.ScriptServiceUtil

// ble2mqtt device availability
val String[]    bluetooth_ble_device_availability_list = newArrayList("sperl_sp620e_dev01", "sperl_sp611e_dev01", "qhtek_triones_dev01")
val String      bluetooth_ble_device_availability_source = "bluetooth_ble_device_availability"
val Long        bluetooth_ble_device_availability_timeout = 10L

rule "Bluetooth: BLE device availability"
when
    Time cron "0 * * * * ? *"
then
    val _now = ZonedDateTime.now();
    
    bluetooth_ble_device_availability_list.forEach[_item | 
        val _lastseen = ScriptServiceUtil.getItemRegistry.getItem(_item + "_lastseen").state; 
        if(NULL !== _lastseen){            
            val ZonedDateTime _previous = ZonedDateTime.parse(_lastseen.toString() /* 2023-01-26T01:35:09.000+0300 */, DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ"));
            val Duration _diff = Duration.between(_previous, _now); 
            //logInfo(bluetooth_ble_device_availability_source, _item + " last seen " +  _lastseen.toString() + " [" + _previous.toString() + "] -> " +  _diff.toString());
            if (_diff.compareTo(Duration.of(bluetooth_ble_device_availability_timeout, MINUTES)) > 0)
                ScriptServiceUtil.getItemRegistry.getItem(_item + "_availability").sendCommand(CLOSED);
        }
    ];
end

Hans_Lree · January 27, 2023, 1:19pm

Might not an Expiration Timer for your devices’ items be a shorter and easier way to achieve this?

maxin · January 27, 2023, 1:33pm

I completely forgot about him!!! Thank you!

Dom_KS · January 31, 2023, 6:15pm

Sorry but I didn’t have free time. Today I take my test rpi 3 b+ and now I can test what you need.

There is something wrong

openhabian@openhabian:~ $ python3 -m pip install -U bluetooth-clocks
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bluetooth-clocks
  Downloading https://files.pythonhosted.org/packages/3e/d5/0262beb7bfa2847c938861ece41bac44378735b4c2165be74560e616e60f/bluetooth_clocks-0.1.1-py3-none-any.whl
Collecting importlib-metadata; python_version < "3.8" (from bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/26/a7/9da7d5b23fc98ab3d424ac2c65613d63c1f401efb84ad50f2fa27b2caab4/importlib_metadata-6.0.0-py3-none-any.whl
Collecting bleak>=0.19.0 (from bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/44/be/da0de00c7c66c14c0ccaa5927abd048927f93d9c7ed2598d99549d7f9109/bleak-0.19.5-py3-none-any.whl (132kB)
    100% |████████████████████████████████| 133kB 871kB/s
Collecting zipp>=0.5 (from importlib-metadata; python_version < "3.8"->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/01/3c/9d84fc1dbac1c5103bf3cd994e4895642001f75eb2139bddbc02aa1906e5/zipp-3.12.0-py3-none-any.whl
Collecting typing-extensions>=3.6.4; python_version < "3.8" (from importlib-metadata; python_version < "3.8"->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/0b/8e/f1a0a5a76cfef77e1eb6004cb49e5f8d72634da638420b9ea492ce8305e8/typing_extensions-4.4.0-py3-none-any.whl
Collecting dbus-fast<2.0.0,>=1.22.0; platform_system == "Linux" (from bleak>=0.19.0->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/81/67/a83522d6692a72911c3586e2fcd2363ce22d1522a7e17e1012f255a6083b/dbus_fast-1.84.0.tar.gz (65kB)
    100% |████████████████████████████████| 71kB 987kB/s
  Installing build dependencies ... done

Collecting async-timeout<5,>=3.0.0 (from bleak>=0.19.0->bluetooth-clocks)
  Downloading https://files.pythonhosted.org/packages/d6/c1/8991e7c5385b897b8c020cdaad718c5b087a6626d1d11a23e1ea87e325a7/async_timeout-4.0.2-py3-none-any.whl
Building wheels for collected packages: dbus-fast
  Running setup.py bdist_wheel for dbus-fast ... |                                                 done
  Stored in directory: /home/openhabian/.cache/pip/wheels/8f/56/85/2d82b076cf6e733315e4cc4f922055f3f9a47c952b55264659
Successfully built dbus-fast
Installing collected packages: zipp, typing-extensions, importlib-metadata, async-timeout, dbus-fast, bleak, bluetooth-clocks
  The script bluetooth-clocks is installed in '/home/openhabian/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed async-timeout-4.0.2 bleak-0.19.5 bluetooth-clocks-0.1.1 dbus-fast-1.84.0 importlib-metadata-6.0.0 typing-extensions-4.4.0 zipp-3.12.0
openhabian@openhabian:~ $
openhabian@openhabian:~ $
openhabian@openhabian:~ $ bluetooth-clocks discover
-bash: bluetooth-clocks: command not found
openhabian@openhabian:~ $ sudo bluetooth-clocks discover
[sudo] password for openhabian:
sudo: bluetooth-clocks: command not found

Dom_KS · January 31, 2023, 6:49pm

I restarted rpi and now it works. There is probably problem with time zone. There is always 1h mistake (I live in +1h zone). My rpi has set good time&date. I took screens from Telink Flasher.
First one is before any changes, second after bluetooth-clocks set -a A4:C1:38:34:A4:EC and the last one is taken after bluetooth-clocks set -a A4:C1:38:34:A4:EC -t 2023-01-10T16:20

openhabian@openhabian:~ $ bluetooth-clocks discover -s 10
Scanning for supported clocks...
Found a PVVX: address A4:C1:38:34:A4:EC, name ATC_34A4EC
Found a PVVX: address A4:C1:38:20:C2:AB, name ATC_20C2AB
Found a PVVX: address A4:C1:38:DD:13:4E, name ATC_DD134E
Found a PVVX: address A4:C1:38:C6:AF:20, name ATC_C6AF20
Found a PVVX: address A4:C1:38:9A:E4:40, name ATC_9AE440
Found a PVVX: address A4:C1:38:D6:5E:0B, name ATC_D65E0B
openhabian@openhabian:~ $ bluetooth-clocks discover
Scanning for supported clocks...
Found a PVVX: address A4:C1:38:34:A4:EC, name ATC_34A4EC
Found a PVVX: address A4:C1:38:DD:13:4E, name ATC_DD134E
Found a PVVX: address A4:C1:38:C6:AF:20, name ATC_C6AF20
Found a PVVX: address A4:C1:38:9A:E4:40, name ATC_9AE440
Found a PVVX: address A4:C1:38:20:C2:AB, name ATC_20C2AB
openhabian@openhabian:~ $ bluetooth-clocks set -a A4:C1:38:34:A4:EC
Scanning for device A4:C1:38:34:A4:EC...
Writing time to device...
Synchronized time
openhabian@openhabian:~ $ bluetooth-clocks set -a A4:C1:38:34:A4:EC -t 2023-01-10T16:20
Scanning for device A4:C1:38:34:A4:EC...
Writing time to device...
Synchronized time
openhabian@openhabian:~ $ date
Tue 31 Jan 19:38:43 CET 2023
openhabian@openhabian:~ $

First one is before any changes,

bluetooth-clocks set -a A4:C1:38:34:A4:EC

bluetooth-clocks set -a A4:C1:38:34:A4:EC -t 2023-01-10T16:20

koenvervloesem · February 1, 2023, 8:42am

And is the clock also showing the time one hour off on its display?

Dom_KS · February 1, 2023, 9:45am

Yes it is. Screens shows exactly what is displayed on display.

BTW.
This 10s scannig discovered 6 thermometers but I have 7 .

BTW 2.
Now I’ve got runing second Rpi 3 b+ (test one) so I can do more test if you need.

koenvervloesem · February 1, 2023, 10:13am

I found the error, this will be fixed in the next release. But I’m glad setting the time works, as I don’t have such a device myself, so I had to resort to reverse engineering the original code. Thanks for trying this out! I’ll let you know when you can try the new release.