TL;DR A set of tools to help you set up Face Identification on Azure and use it with OH to trigger personalized rules.
We all know it. The door rings and before we have a look at the camera someone already opened up and the in-laws populate your living room. Would it not be convenient if someone had released the hounds before?
Fear no more: Machine Learning is improving our everyday lives.
I don’t want to bloat my post too much with technical stuff as it is all explained in the readme on GitHub. There you can get the code and hopefully all the information.
A camera somewhere in your network and openhab is able to access images on it. I have two cams that both have a jpg stream where I can access the images via http://myUrl/current.jpg. I store the image triggered via the exec binding and then run the image identification on it.
You know a tiny bit of programming (Python here), working with web APIs and Postman
You have good quality image of people you’d like to identify
You have about 2-3h of time
When everything is in place you will be able to trigger the script via some sort of OH rule and get the ID of the person the service has identified. Then you need to act upon the information.
E.g. set a specific scene if your significant other comes home, switch of the alarm if a known face approaches… etc.
I still have to figure out a convenient way to map the IDs back to real names… maybe via a map…
What is your idea?
*disclaimer: No RefLinks were used in this project. short URLs for comical and convenience purposes only. Azure was only chosen as the results are quite good and it is free to use for this purpose.
thanks for this interesting post. But WHY has this all to happen in Azure? Come on - this kind of image processing can easily be done on local hardware. I personally have very reserved thoughts about this, as it - at least in the EU - night be heavily violating privacy laws if you transfer other persons’ photos without asking them and letting them know.
Sure - interesting thing, but could you perhaps elaborate a bit more on such concerns about privacy?
I personally run my OH on a Raspi which is not powerful enough to do this kind of processing. So it was the logical choice for me outsource this.
If I am not mistaken you can download a Docker image from Azure to do the same kind of processing on premise. This way you are in control. I have not tried if this actually works, but have a go and let me know if this works as well.
thank you for your reply. Indeed that sounds interesting though I would still be careful with such images, but behind a firewall it should do no harm.
Please don’t feel offended - to me the scenario described just sounded like an 1984 nightmare and scares me out to a certain extent after having lived some time in China. Don’t take it personal
Great news, another thing to add to the list to check out IF I ever get spare time. I am sure you will already know that Home Assistant has been doing this for a few months now using https://www.tensorflow.org/
A video showing tensor flow in action and how to setup.
@antares2001 No offence taken. Data privacy is an important topic and should not be neglected.
In this case I am not too worried, as no private information is stored on the cloud servers. The only way to make a connection between the image/face and the person is you via the personID.
@matt1
I am indeed surprised that Tensorflow performs so well on the raspi. Azure is very likely also using a TensorFlow model for the Face Identification. So maybe it might even be possible to recreate this on your own raspi. However I dunno how much tweaking they did to the Algorithm
They usually do only 1 FPS as that is all that is needed, plus there is a USB stick that can accelerate and off load some of the processing. When the pi4 came out with USB3 they were very excited as it meant no more bottleneck getting the data to the USB processor. There are not many people doing it yet from what I gather so keen to see what you can come up with.
