Wanting to post a quick introduction and tutorial for our new Theengs Gateway feature of quickly and locally getting the presence status for Apple Watch, iPhone and iPad devices over Bluetooth, with an addition security relevant unlocked property, I found the following issues when using the homeassistant convention discovery from the Theengs Gateway created discovery entries, when adding them to Model.
• Small issue for binary_sensor, which seems to be the only class which does not honour the name, only filling in a generic value, whereas other discovered sensor entries of the same device do have the correct entries. Nothing major, as it can always be entered manually, but annoying nevertheless, as other sensor types are parsed correctly. E. g. for an Inkbird TH1
• More seriously though, any device_tracker only seems to be possible to have as a compete model equipment with no separate channel, without me finding any way of having an item with it’s value_template definded Home/Away state. So the two issues appear combined with the local Bluetooth Apple Watch, iPhone and iPad recognition.
While personally I do all my definitions purely textually, and creating a Home/Away item with a Home status while regular MQTT messages come in, plus an expire timer for the Away state, and using the unlocked property for additional security sensitive rules like opening gates, garage doors or turning off the alarm system when coming home is all fine.
But trying to write a introduction/tutorial with the above issues - all with the current latest stable 4.1.1, for others who might be in interested in this with their Apple devices, does not quite work.
Is it my complete inexperience with HA convention auto-discovery in openHAB, or are these issues known with device_tracker and binary_sensor discoveries?
The actual MQTT messages to monitor after a possible correct discovery or manual definitions look like
In 4.3, the name of binary sensors works as you’re hoping.
Device tracker is still not a supported component. I don’t have any devices that use that component. I’ve looked at the docs, but it’s not necessarily a straightforward mapping to openHAB semantics. Basically Home Assistant allows the same entity to contain something simple like “home” or “away,” OR to contain actual latitude/longitude, and even to switch back and forth at any time without changing the discovery information. I’ll probably just end up exposing it as two channels, always, in openHAB - a string and a location, and one or the other will be NULL depending on if any appropriate messages have been received.
But yes, I don’t see posts on the forum unless I’m specifically mentioned. I do look for new MQTT related issues on GitHub periodically (and @lsiepel does a pretty good job of making sure I don’t miss things there either - thanks!).
Thanks for the feedback, I will give 4.3 a test run sometime soon and will let you know of my findings.
It would be great if Bluetooth and RTL_433 OpenMQTTGateway could finally be easily and nicely discoverable and not just usable for the manually tech savvy users.
I’m just a bit worried now about the mention of values intermittently going to null, as mentioned in this other thread but will see it all for myself in a 4.3 test run.
Thanks to your replies I have tested OH 4.3 M4 today, and HA convention MQTT discovery has definitely greatly improved from when I last tested with 4.1.1 (or so). A major step in the right direction!
I tested with the following devices:
Qingping Air Monitor Lite
Xiaomi CGD1 alarm clock
Inkbird TH1 thermometer
Xiaomi Mi Band 4
Apple Watch
and these are my findings.
All sensor device class channels are discovered correctly, properly named and display the correct values when a model has been created, e. g. for the Qingping Air Monitor Lite
I didn’t really expect any regression here, as the sensor device class always seemed to have worked, but I am wondering if the HA discovery workaround with having to remove is_defined from the value_template from the HA native version is still required, and if this might possibly also be the null resetting issue @DrRSatzteil is seeing? Waiting for confirmation on his setting of this.
Also the fact that the Semantic Class and Semantic Property are not take over from the discovery messages at all , instead just generally assigning Point to all, would be another point on my wish list for a future slick HA convention discovery.
The above mentioned binary_sensor issues seem to have been addressed name wise, but I am seeing a failed NULL binary_sensor for the Inkbird TH1 thermometer, with its discovery message value template being "val_tpl": "{{ value_json.extprobe }}"
and the MQTT status messages correctly having … "extprobe": true … or. … "extprobe": true …
making me assume that all other binary_sensors with the same simple value template and boolean values will have the same problem still.
Only the Apple Watch, which has a more verbose value template, implemented for security reasons to automatically set it to LOCKED when the device tracker is recognised as AWAY, with "val_tpl": "{% if value_json.get('unlocked') is true -%}True{%- else -%}False{%- endif %}"
correctly discovers and immediately displays its unlocked state.
Any kind of such verboseness should not be necessary however for the vast majority of binary sensors which are not security or privacy related.
I have to admit that I didn’t even try discovering a BLE device_tracker as a binary sensor, due to the issues stated just above, but also because HA distinctly differentiates between device_trackers and binary sensors. it would probably work, but also mean that every MQTT implementation, not just us, would need to duplicate all their local device_tracker discoveries, one the fully HA native way and then the OH half-HA way, with user having to make sure they do select the correct preferences setting before they have anything discovered.
In case you are interested in the device_tracker discovery message, here it is for a Mi Band 4 for example
As I stated before, I do not use auto-discovery personally at all, I decided to stick with OH2 for the time being, but since collaborating on OpenMQTTGateway, Theengs Gateway and Theengs Decoder I have experience quite a bit of the other home automation system out there.
In HA, an Apple Watch discovered by Theengs Gateway is shown in the Inbox, and only one single click on the Add to Sahboard (page) button is required to get this
with the correct Bluetooth device tracker icon and lock icons included due to the information in the discovery messages.
I am not posting this to be mean, but because I have been with OH from very early on, and being saddened having read through all the recent hundred post long threads about how to make openHAB more popular again and remembering all the multiple threads on people trying to make their iPhone/Apple Watch for presence detection.
I really feel that a fully and properly working HA convention MQTT auto-discovery could be a big step towards acceptance of less tech-savvy users.
All the above are purely my personal views and can be taken to heart or be ignored at your discretion.
I’m working on device_tracker right now, and I’ll probably just install the Theengs Gateway on one of my Linux boxes to test with Apple devices. For your Inkbird thermometer not reading the value, can you please file a separate issue on GitHub, and include the discovery JSON, and the state (and JSON attributes, if present and a different topic) values, as well as which software is producing it (OpenMQTTGateway?)?
Unfortunately openHAB does not yet have a clear way for bindings to push this information through to newly created Items. There’s a Category property on channels, but that generally lines up more with the classic Icon set, and definitely not with Semantic type. I’m not even sure if MainUI (I only use file-based items, most of them auto-generated by Ruby) will push that through to Category on newly created Items (which is effectively icon for the item), which at least now openHAB supports MDI icons. Cause I agree, I’d love to see the point where I can just add a Thing in MainUI, it populates lots of semantic tags, and then say the HomeKit add-on can read well-defined semantic tags to easily find everything it needs on an equipment. It recently got unitHint support, so I’m hopeful we can keep moving in the right direction there.
That is great news and thanks for the confirmation
Wow, also great new, I take it you have a Mac to get the required Identity Resolving Keys for your Apple devices, and follow our instructions at
Alternatively you could also use any of the bold devices in the top paragraph or any devices with a tick mark in the Decide Tracker column on our compatible devices’ list
For your Inkbird thermometer not reading the value, can you please file a separate issue on GitHub, and include the discovery JSON, and the state (and JSON attributes, if present and a different topic) values, as well as which software is producing it (OpenMQTTGateway?)?
It was also with Theengs Gateway during my 4.3 M4 testing earlier today/
I will log a bug, especially again now that I saw that openHAB finally also became a bit more professional by implementing the GitHub stale action - hopefully project wide. The previous haphazard frantic issue closing and nuking, especially before big major releases, still valid or not, to message the issue count had me on a issue logging embargo
If you have any of the following devices yourself though, they should also display the binary_sensor boolean NULL issue:
Qingping Motion & Light
Mopeka/Lippert gas tank level monitors
SwitchBot Blind Til
SwitchBot Contact Sensor
SwitchBot Motion Sensor
SwitchBot Curtain (2/3)
Shelly BLU Door/Window
Shelly BLU Motion
as well as a few other more obscure devices/properties.
I understand if this is due to a technical restriction. From my side it just looked like an omission of clearly present information in the discovery messages.
Thanks for being so responsive and let me know if you haver any queries about setting up and running Theengs Gateway.
Yes, I have a Mac and had no problem getting my IRK. I did notice that I never see "unlocked": true in the state message on my iPhone 16 Pro running iOS 18.2 beta. But that’s not really relevant to openHAB’s implementation of the Device Tracker component.
I assume you mean with the unlocked binary_sensor on an Apple device? I’m getting a proper OFF value for an item linked to this channel, but it was NULL until Theengs Gateway published the next update, since the message isn’t retained.
yes, this is a know issue still for some users, and was mentioned in the original presentation post I deleted again out of frustration. unfortunately all my iPhones and iPad work as expected, so do all the device’s I had access to from friends, from ancient devices running iOS 12 to 18.1.1
I’ll either need to get my hand on an iPhone where it doesn’t work to debug the decoder or see which setting might make the difference, or the unlocked key needs to be removed in the future for iPhone. hence also the Apple Watch screenshot above, where it is working perfectly for everyone
If you do have a HomePod, see whit it makes any difference after you AirPlyed some music to it from your iPhone, one user reported that this was all it took for him to have it working as well.
No, the unlocked key was fine, as it has the verbose value template "val_tpl": "{% if value_json.get('unlocked') is true -%}True{%- else -%}False{%- endif %}"
but any of the other mentioned device, the Inkbird TH1 with an external probe socket had a constant NULL, even when the published MQTT messages from it had
“extprobe”: true
with the discovery messages with a less verbose value template for binary_sensor being
Ohhhh… I bet I know what’s going on. You haven’t yet posted the state message that’s not working. I betcha it’s getting sent as true or false, but the discovery message has "pl_off": "False", "pl_on": "True" - i.e. openHAB is matching exactly, case-sensitively, and since it doesn’t match either, it’s just ignoring it and leaving it as NULL. I wonder if Home Assistant is doing the comparison case-insensitively. Their docs don’t specify at all. I’ve also run into cases that it seems Home Assistant is stripping leading and trailing whitespace, but again, their docs don’t say if that’s intended behavior.
I was actually thinking that might be as well when posting the discovery messages. yes, HA does not care about the casing, so it’s working nicely there, but does brake on OH.
I can confirm that it does not do this, for the source_type at least, as we had a bug a while back with a leading space causing it to not be correctly recognised.
So now we know what was causing the issue there, best if we keep the casing consistent (lower) for full HA/OH compatibility.
Supposing your device_tracker implementation will also be case sensitive I’m also amending the above casings in the verbose device_tracker value template.
I really thought it was the casing pasting the discovery message, but testing it out now, using lower case in the discovery message, as with the boolean values in the actual published MQTT messages, it now does work in openHAB 4.3 M4, but it breaks in Home Assistant, the latest Core 2024.11.3/Supervisor 2024.11.4
Bed time here for tonight, but I’ll have to have another look tomorrow and also talk to the colleague who originally implemented these upper case discovery values in OpenMQTTGateway, obviously for a very good reason, which is now apparent, as inconsistent as it might look.
Well, the Python (HA) boolean upper case convention requires the upper case, at least in any value template definition, as in the Apple Watch “val_tpl”: “{% if value_json.get(‘unlocked’) is true -%}True{%- else -%}False{%- endif %}”
while all other binary sensors can be fixed by turning the playlaod definitions into proper booleans from string versions, so that it both works on HA Python as well as OH Java.
This either means we’ll have to keep a separate openHAB discovery setting, to send different value templates, differentiated only by their boolean casing or you see a way to also accept the upper case booleans from value templates in OH, as this would not just affect our discoveries, but any other MQTT discovery implementation with boolean in value templates, for any other
… projects that have such workarounds to let them know once openHAB 4.3 is released.
It also means that any other project using the upper case quoted payloads only discoveries will have to change them to true booleans for compatibility, all boiling down to the different boolean casing, when quoted in the payloads, or unquoted, but upper case in value templates…
I assume this to be the only remaining language difference issue affecting HA convention discovery between Python HA and Java OH.
:windblown: WTF. How?! This is the sort of stuff that pisses me off about Home Assistant (and Python projects in general). a) there are payload_on and payload_off values that are clearly strings (except when they’re not; I see that zigbee2mqtt publishes payload_on that are actual true values) that should be being used to compare things. It shouldn’t matter that it’s ostensibly a boolean value - it should be being treated as a string. b) the output of a Jinja template is a string. c) it’s coming from JSON, where the standard is quite clear that the values are true and false, not True and False (and null not None). The YAML spec even goes so far to have an example of True not resolving to a tag:yaml.org,2002:bool. Yet most Python projects I’ve used not only accept True, False, and None in YAML, but have them as their examples in documentation. Some (Home Assistant!) go so far as to have things like the payload_reset defaulting to None. Case insensitivity (and leading and trailing whitespace ignorance) is reasonable, especially when it’s expected that the input is from a user (less so when it’s for a defined interface between programs, but Home Assistant doesn’t really distinguish because it uses the same code for loading locally defined entities from YAML as it does for configuring entities via MQTT discovery). But forcing other applications into your Python-centric view of the world that is different than every other programming language is just plain frustrating.
</ rant>
I’ll try and find some time to dig into it more, but it probably won’t be today. Can you maybe try payload_on being true instead of "True" like zigbee2mqtt? Or "ON" (which is actually the default), and a value template that translates the JSON value to ON?
while all other binary sensors can be fixed by turning the playlaod definitions into proper booleans
so not an issue for the payloads with that change, which any other projects would also need to apply if they so far are using strings, but remaining is the fact that within value_templates, like for Apple Watch unlocked
“val_tpl”: “{% if value_json.get(‘unlocked’) is true -%}True{%- else -%}False{%- endif %}”
they still need to be defined as upper case for HA Python to recognised them correctly as booleans.
Welcome to controller agnostic multi-language world, what we are facing with our projects also with Theengs Gateway running on Windows, Linus and macOS (the latter not really useful with its Bluetooth MAC address masking into inconsistent UUIDs though), So trust me when I say I’d rather have it universally correctly recognised as well.
But did you do it as a string, or as a boolean? This is what zigbee2mqtt looks like, and it works fine with openHAB, and presumably fine with Home Assistant:
so these are fine and no issue any longer, both correctly discovered in openHAB and Home Assistant. Still requires the payload definitions in Python to be assigned with upper case bools
device["pl_on"] = True
device["pl_off"] = False
And this is also the reason why, whenever there are value_templates which require conditional payload state assignments, instead of just the above value_json.xxxxx being taken over, to also have capitalised booleans, as in the mentioned
Only working in HA: “val_tpl”: “{% if value_json.get(‘unlocked’) is true -%}True{%- else -%}False{%- endif %}”
as this currently does not work in openHAB, but making them lower case breaks it in Home Assistant.
Only working in OH: “val_tpl”: “{% if value_json.get(‘unlocked’) is true -%}true{%- else -%}false{%- endif %}”
So value_template assigned booleans seems to be the only issue left from what I can tell at the moment.
but will see it all for myself in a 4.3 test run.
