Best Presence detection

Setup, Configuration and Use Beginners
, ,
82

I don’t know if there is some sort of consensus. I have benne looking at different strategies to get the best presence detection. My conclusion so far is, that there doesn’t exist one particular method that will cover everything. All methods have it’s own strengths and weaknesses. So I basiclly believe that a good solution is to combine different methologies to get the optimum solution.

I see the sensors grouped in personal / non personal sensors and portable / non portabel sensors.
I guess almost all non personal sensors (at least all those I can think of), is also non-portabel:

  • Door/Window contact
  • PIR Sensor
  • Proximity sensor

Where I have knowledge of both kinds of Personal sensors:

  • OwnTracks (Portable)
  • Beacon (both)
  • Wifi (Non Portable)
  • Bluetooth (Non Portable)

Amongst others…

The non portable sensors has the strength that “false positives” is very close to none. That is since it typically detects if a eg. Wifi newtwork is visible, and thus that defines the geofence. On the other hand It doesn’t deal very well with “false away”. (E.g., if I turn off my phone while at home, presence will be offline)

The portable sensors is OwnTracks or similar. OwnTracks, is event based, so where it really doesn’t matter if I turn off my phone at night (since the transition wouldn’t occur). I considder it as quite stable and accurate, Though I sometimes (rarely) see that the transition doesn’t happend.

The nonpersonalized sensors is also interesting, but I believe that they should be more an supplement, than a primary decission.The Wasp In A Box Algortihm has some thoughts about this. Seen from my perspective, the usage of those sensors could not be of much use alone, but I you build a model and monitors a series off events (eg. pir sensors and door sensors), you could have quite a good guess if somebody is leaving or comming. The problem is that you need to build an individual model, so the precision of the outcome is highly related to how good the model is.
The more I think about it, I believe that I could obtain more or less the same precission by combining the two personalized sensor types. At least if I would create a genric algorithm that is usefull for every body. Maybee some selftraining netwok could be usefull to build a good model.
I still believe that there is a lot off information in the non personalized sensors.

Actually I think, that if I setup a couple of Raspberry Pi’s with a WiFi dongle in monitor mode and also scans for particular Bluetooth MAC’s, combined with OwnTracks. I would have a hit rate close to 100% - at least in my case.

1 Like
83

It would be nice if there was a way to hook into the google location tracking service as then we ould easily poll this service to fjnd the last known location. I like this idea as we dont need to install any other apps on the phone and google is tracking this already (if enabled)

I have been jsing ifttt but i find 25% of the time the trigger doesnt fire.

Really wish there was a good solution out there.

Does anyone know if FIND does much battery drain?

84

I think, that in the end, face recognition, people counter and movement direction, all in the camera, is the most promising solution. Of course still attached with a higher cost. But you don’t really want to lit up the room and greet the occasional thief welcome :slight_smile:
Personally, dragging stuff along with me, just for the house to know it is me, is not a solution.
Google location @peoples home sounds more like a nightmare than a feature to me :smile:
(not saying it is wrong or a bad solution for others)

85

Find claims That the battery drain i low.

86

For me it isn’t really an option to add cameras all around the house. And therefore the above solution is not and option for me. So the question is what do we mean be Best Presence Detection?

  1. Best technical solution
  2. Best solution seen from an individual perspective

For me it is number two, and thus it is also important to think about what I want and don’t want. Do I want to share my location with various cloud services? Do I want to add cameras around my house?

Personally I want my Presence detection to be as “silent” as possible (eg. no cameras inside the house). I don’t want to geotrack each member of the house - a geofence is OK. I also dislike using public cloud services for presence detection (and sensors at all), so I always go for solutions that can be self hosted (OwnTracks, Find, …).

With that in mind, I don’t think you can say that one solution is better than another. That depends on our individual priorities. I believe that the possibility of mixing different services would be the best solution.

87

I’ve been using OwnTracks on iOS for many months now, and if you set it up using the geofences it works really well. Very reliable home/away events.

88

Well, I am certain we all are just discussing “Best solution seen from an individual perspective” here, since we want to solve the probelm. Anything else is secondary. The future that I prodposed is not what you might consider cameras per se , but small sensor that recognizes who is moving in what direction (if any), and reports this to OH, or whatever HA-system is around. There are already such devices, and they are not intended to produce a picture, just information such the ‘X’ peoples are in the room, and ‘micael’ is once of them.
I never said this is the one and only solution, which is best for everyone, I just stated my view. Hence I started with ‘I think’.

1 Like
89

Some time ago I promised that I would share my python script when I had something usefull. Right now I have stopped working on my presence detection, primarily because I want to see how clsoe the Ubiquiti Unifi Binding brings me to the goal. Right now it seems like it could bring me very close to the goal. I am already using OwnTracks, and I am looking for Bluetooth device scanning and/or FIND server as a third possibility.
I will most certain end up with some logic in python, my guess is that it will be very little.

90

I agree with vespaman. Face recognition is a close to perfect solution, as this will target an individual. (dont know how it will handle twins though).
Wether or not it´s an option to add cameras all around you home, you actually just need one in your doorway to state the home/away. Together with the Ubiquiti binding you´ll know when a person leaves, (as I doubt your family members want to look into the optic lens before leaving :slight_smile: ). The camera will therefore only be needed to know, who arrives and when.

91

Hej,
I’ve forked and created some Arduino code to flash on an ESP8266 which scans all WiFi channels for a specific MAC address, if found provides it to the MQTT broker, where it can be setup as a contact device. If it disapperars for over 10 minuntes from all channels, the corresponding OpenHAB item is informed via MQTT again, that it’s gone, this is working pretty well and does not require any app or modifications on the mobile device. So you can more or less track every device having a MAC address.

If this helps you, checkout my code here:

It’s really easy to setup, as you only need e.g. a wemos D1 and nothing more and it works nearly out of the box.

3 Likes
92

Kai has put up a blog post on Smart Home Day.
Last presentation is on presence detection.

Oh, and FIND is to become part of openHABian.

93

@andreaslink-de

I also moved to using ESP8266 sniffer (Wemos D1) and it seems to work pretty well (better than the network binding as far as I can see). I did the following changes to the code:

  1. Moved to use REST API instead of MQTT
  2. The sniffer first get all items tagged with [“Presence”] from openab and only for those items it send status.
  3. Flag to lock to channel of router so detection is faster (no need to scan all channels).
  4. The sniffer sends the notification directly to the item and not via proxy item.
94

Ok, that sounds pretty cool, (sorry I did not respond earlier to your former comments, as I was lying ill in bed some days), is there anywhere the source available, where I can see your improvements/changes?
And did you fully delete MQTT or did you just add the REST API in addition? I really would like to add your improvements to my solution, but still stick to MQTT here.
I like the idea of the channel-lock, but I’m a little concerned, if this will work in all cases? As I have several access points split over my house and they are all working on different dedicated channels, will that be caught by your setup from point no. 3? So the same device can switch channels while moving through the house.

95

@andreaslink-de

I have removed the MQTT support. Probably it will make more sense to support both but because I need to query the items via REST API I didn’t see any point to keep the MQTT (as I already have HTTP support).
I will send you my code and you can take the parts that you want and maybe integrate them into the MQTT version.
Currently the lock is done to the given SSID. It shouldn’t be hard to add support for more then one SSID.
Note that the lock is for “friendly” presence. I also plan to add extra wifi sniffer for “intruders” which will scan all channels.
Another enhancement can be adding some openhab items that will be used to configure the sniffer. This way after sniffer boot / reboot, it will access those items for configuration.

1 Like
96

Just looking into FIND, looks impressive. Seem to have it all running, but can’t get the simple bit of logging the information to an openhab item to work.

The docs give this example for the item:

String mqttfind_johndoe "John Doe is @ [%s]" (All) {mqtt="<[find:YOURGROUP/location/USERNAME:state:JSONPATH($.location)]"}

I’m getting state = “JSONPATH($.location)” when using this. I’m guessing there’s just a syntax error, but I’m not sure what it is?

97

Just answered my own question, I needed to install the JSON transformation

98

moring sir, I’ve try your sketch, it seems working, however the serial console and mqtt pops up lots of MAC address, something like this:

{"MAC":{"106682813de4":"OPEN","f42853370d4c":"OPEN","d8150d5d83f4":"OPEN","6cb0ce1f6fe0":"OPEN","8cf228054b5a":"OPEN","ec26caffec22":"OPEN","206be76f1f4d":"OPEN","c83a35452ae8":"OPEN","f428534c3588":"OPEN","f428533eb0d4":"OPEN","c83a352d3918":"OPEN","d0fa1d6a8dd9":"OPEN","503aa0a85b6e":"OPEN","c061180a3c70":"OPEN","f4285347e09c":"OPEN","503aa09306f2":"OPEN","246968009426":"OPEN","44975a67d92c":"OPEN","3496727abaa3":"OPEN","8cbebe26d1b9":"OPEN","64098069dbd8":"OPEN","88259364c0fe":"OPEN","3496723e9bea":"OPEN","5c63bfc7e3ac":"OPEN","f4cb520a5378":"OPEN","5001d93bda04":"OPEN","349672f8dfcc":"OPEN","8cbebe289e0c":"OPEN","f0b42943d41e":"OPEN","bc469931ca50":"OPEN","30fc68ae9b74":"OPEN","02fc68ae9b74":"OPEN","487d2e1ac55d":"OPEN","c88d83b5059c":"OPEN","b827eb0fc3fd":"OPEN","600194989ee1":"OPEN","f42853370d48":"OPEN","0c9160868b90":"OPEN","106682813de3":"OPEN","ecfabc877cab":"OPEN","eccb3036b804":"OPEN","60019499ac7d":"OPEN","b43052792998":"OPEN","48a9d25ba1f4":"OPEN","9c443d3a36a3":"OPEN","48a9d2409b5c":"OPEN","8cbebe26d1b8":"OPEN","5001d93bd9fb":"OPEN","844bb7025b70":"OPEN","00e04c06bec7":"OPEN","30749619905c":"OPEN","8c89a502573c":"OPEN","60019485b594":"OPEN","2c3ae837d6be":"OPEN","7c7d3da53894":"OPEN","683e34e27eb0":"OPEN"}}

What’s that? Does these super long MAC addresses are all my neighbor’s wifi router? or, these just my local wifi’s devices?(cuz I think that’s too many, I don’t think I have such this amount of devices connecting to my wifi router, I think it’s 64 devices…)… so the question is, what are they? How to get rid of them? What’s Device DB? I think this amount of devices is coming from the Deivce DB?

Connecting to 'mywifi'
.....
WiFi connected
IP address: 192.168.0.42
Connecting to MQTT... connected as ESPClient-8878523

Number of devices: 66
{
  "MAC": {
    "106682813de4": "OPEN",
    "f42853370d4c": "OPEN",
    "d8150d5d83f4": "OPEN",
    "6cb0ce1f6fe0": "OPEN",
    "8cf228054b5a": "OPEN",
    "ec26caffec22": "OPEN",
    "206be76f1f4d": "OPEN",
.......(many mac address....)

Also, I think the codes is getting the beacons even far away from my house(I even get the car driving recorder coming from neighbor’s… lolz…)

Can I add my devices to the list(or I should say the DB) instead of a super huge amount of “scanning ap mac” keep flushing my mqtt server every few seconds?(It’s very huge…)

99

@kevinshane

Have you tried setting the

#define MINRSSI -150

in WiFi_Sniffer_OpenHAB.ino

to -90 or -80? If the code works as I would expect, the list should be considerably shorter then.

100

Oh, that sounds great so far, there are two major answers for this, one could be to reduce the default MINRSSI value as this sets the minimal strength, when a detected beacon is taken into consideration, so this equals somehow the distance to the device.
Or the second one is related to devices near by :slight_smile:, if you have of course multiple devices (e.g. a flat in a big crowded house), than this is the explanation (remind there are a lot possible devices, like every phone, computers, webcams, sometimes printers and so on), but if at least one of the devices is an Apple iPhone or iPad and this device is not connected to any(!) wifi network, each sent beacon is sent with a random MAC. This can also pile up entries a lot. Usually everybody has his phone connected to at least his home wifi spot, so this should not be a concern, but could lead to a high amount of fluctuating random devices, if not.

Another solution could also be, to modify the code to only filter on a specific set of MAC addresses.

101

That would work, thanks man!
I also set the channel to only 6, which I can limit my router channel to also 6 so it only shows every device connected to channel=6, this will greatly shorter the device list down to very few… previously if I do nothing, I can get 100+ devices around me. ^^