Hi folks. Eleven years ago I built a smart house from scratch (i.e. 200 “things” all managed by 20+ arduinos). I was lucky in that I could build the automation infrastructure together with the house so I could embed a ton of sensors which later allowed me to implement many use cases.
A couple of years ago I started to work on AI. In particular we have been building a platform to lower the entry bar to apply computer vision.
Recently, I realized that on most of the rooms in my house I have cameras that could be used to create new sensors. In particular, on my living room I have contact sensors that are chained together so I cannot tell what is the specific door or window that is opened (but it is very easy to tell from looking at the camera).
I have started to experiment with using AI to create new sensors. So far I have done:
Detect if there are people on the pool (see image)
Detect which door was opened
Detect if an appliance is on (looking at the LCD) (see image)
My learnings so far are very positive. I found that with few images (just ten) I can train a model to recognize one of these cases for my home with good accuracy. The key here is that these models are not general purpose and security cameras are always on the same spot.
All this made wonder if on my next house I would still wire a ton of sensors on the wall or just put cameras everywhere (not on the rooms). I’m curious about your thoughts? I’m also interested into what use cases would you implement using computer vision?
Cheers
PS: Interestingly, eleven years ago I did not plan to have cameras everywhere… cameras are so cheap now that I’m doing some renovations and adding cameras to look at place like (circuit breakers, boiler room, laundry room). I’m even planning to replace outdoor PIRs with cameras since they have many false positives… If you have thoughts about where I need more cameras, let me know now
Sorry for the delay, but I wanted to give you a set of instructions and code that you can use to reproduce what I did. I hope you are familiar with Jupyter Notebooks & Python . I have put instructions and code in collab.
Thanks for making the instructions. However, it’s relying on the cloud to perform the inference. I’ve been trying to find a way to do this offline, using yolo, tensorflow, or something similar.
Also ideally something I can perform in JRuby (so it needs to be native java, not linux binary), or failing that, executable. Since I’m running openHAB in docker, it needs to be easily added to the docker, but I can probably work around this.
I have tried using teachablemachines from google. It allows me to train the model and it would produce a .h5 (keras) or savedmodel format but I haven’t made any further progress on how to actually use it to perform the inference.
My use case, right now, as a start, is to detect whether my rubbish bin(s) is put out near the kerb on the day prior to the rubbish pick up day. But after this, I’m sure there will be lots of other applications, e.g. presence detection, etc.
@JimT Nice use case !. I’m trying to do everything on a Jetson Nano so after decoding 10 RTSP streams and doing motion detection, I don’t think there will be much GPU left. I’ll try to use Frigate’s generic object detection models to reduce the need for cloud analisis. For example, in your case I would ensure that there is a trashcan on the image and then using the cloud I would detect if it is mine or it is the neighbours. I’m hoping I can get number of cloud based images under 100 per day… this way I can have many different models in the cloud
What hardware are you planning to use for this?? If you have a PC with a GPU, LandingLens provides a stand alone free trial app that will expose the API locally (a.k.a. LandingEdge).
. I have put