I just saw this on Hackaday. They appear to have a Node.js based Bluetooth sniffing software stack for detecting the presence of BT devices by proximity. This might be a more thorough and reliable approach than my old sensorReporter script.
They currently have on their website a promise on how to set it up using the built in BT of the Pi 3. And it appears based on a quick scan of the tutorial that one could just use their Owl product to report presence or absence of devices to OH using either OH’s REST or using MQTT if you install the MQTT.js library.
It is pretty extensive and I don’t yet know how much one can do without using their SaaS platform. Perhaps someone with more time can check it out and come back here and report.
Indeed, you can use our open source barnowl package for Node.js to observe and process Bluetooth Low Energy advertising packets captured by reelyActive reelceivers or BLE-capable Linux computers (like the Pi 3). Our pi-relay project gives a good idea how you can POST the real-time events to a remote server using REST.
We’ve updated the hackaday tutorial to show more of what can be done with our open source software stack on the Pi 3.
It would be possible to make an out-of-the-box integration of our open source stack with openHAB following this tutorial. Please let us know if that’s of interest!
I’ve quickly scanned through the out-of-the-box integration tutorial and it looks pretty easy to get set up. I will definitely be trying this in the near future. I do have a few questions:
Does this only work with the Pi 3 or could it work with a Pi 2 with a BT dongle or a laptop with embedded BT running Ubuntu 16? What is the minimum BT version required from the HW?
Is it possible to change the URL for the HTTP POST from http://localhost:3010/events? Or would this require coding and using a Custom Service? openHAB’s REST API would actually be something along the lines of http://localhost:8080//rest/items/My_Phone where “My_Phone” is the name of the openHAB Item defined to represent that BT device internally. I looked at the code for barnaclesrest.js and it doesn’t look too difficult to add that feature. If someone did would you accept the PR (NOTE: I’m not volunteering, my hands are full right now, maybe sometime in the future though if someone else doesn’t run with this).
Writing a Custom Service doesn’t look too difficult so it’s no big deal either way but it would be less work to use what is already there. A lot of home automation hubs, both commercial and FOSS, use MQTT a lot too so it might be worth writing a Custom Service both for openHAB’s REST API and for MQTT. I admit I’ve not done much with Node.js but the example Custom Services look pretty straight forward.
If I have a single Pi 3 using the built in BT, presumably I would never see a displacement event. Or does it report this even as I move towards and away from the single BT sensor? If I set up multiple Pi 3s is there a way to forward the events from one to a single barnowl for consolidation and to get these displacement events, or is that a capability limited to your daisy chained devices?
Ultimately what would be most useful from an home automation context is the ability to detect who is in what room with some degree of accuracy. However, $200 per room is probably out of reach for the home automation enthusiast. We aren’t your primary market so that is understandable.
Thanks for taking a deeper look at our platform @rlkoshak. To answer your questions:
The basic functionality should work with any Linux computer (embedded, PC, etc.) with an integrated or plugged BLE radio. For specifics, take a look at the bluetooth-hci-socket package on npmjs.
The target URL of the HTTP POST can be configured at run time. Our “barnacles” service assumes the route will be /events, but it would only take a minute to create an openHAB service with all the desired configuration parameters. If you provide specifics, we’ll gladly develop and implement it. MQTT is top of our to-do list, so expect that option to be available soon too.
If you have a single receiver such as a Pi 3, you’ll never see a displacement. If you have multiple receivers feeding a single instance of barnacles, aggregation of data is automatic and displacements will be correctly determined based on RSSI. Our open source software was designed with all kinds of such configurations in mind.
While our daisy-chainable reelceivers are designed for B2B (smart building) rather than B2C (smart home), there are people who have indeed implemented them in their home. It’s only $80CAD per reelceiver plus a cable drop (if you can run Cat5) to outfit a room, and you don’t need to waste an AC outlet nor burden your WiFi. You can imagine how that minimises the total cost of ownership for a large building, even if there’s some pain in requiring cable drops initially.
Hope we can complete an integration with openHAB so that users are free to explore the possibilities nonetheless!
Very interested in this, I’ll see if I can get a chance to start playing with it. Please update @rlkoshak or @reelyActive if you do integrate this with Openhab (REST API).
So here is where it can get a little complicated and upon reflection I question whether the REST POST approach is the best.
There are lots of ways to get info into openHAB from reelyActive, both via a push or a pull. NOTE: Below is what would be best from an openHAB perspective, not necessarily easy or feasible from a reelyActive perspective.
The tl;dr is MQTT would probably be best.
rest Push
The absolute best here would be to be able to publish a very simple “ON” or “OFF” to a separate URL per device. For example:
The big thing here is that the best way to set this up would be to allow a different URL per device. openHAB’s rest API provides a different URL per Item and inside of openHAB one would want to have a separate Item to represent each BT device. So for example I would have:
To further complicate matters, I would want to receive an ON via a POST for “appearance”, an ON via a PUT for “keep-alive” or “displacement”, and an OFF via POST for a “disappearance”.
This would provide the simplest integration from an openHAB perspective (though it does eliminate the potential to use the “displacement” event.
Alternatively one could set up a single incoming Item and then write a rule to parse out all the necessary information. So one would have:
In this case all events from reelyActive would go to http://localhost:8080//rest/items/reelyActive. NOTE: Notice how the last part of the URL corresponds to an OH user defined Item’s name.
Then we would need a rule to parse the JSON and extract the device and event type and update the Phone switches as appropriate.
Personally, upon reflection, I find neither of these approaches satisfactory from an openHAB perspective. They both feel hacky and either neuter the full capability of reelyActive or impost a lot of work on the openHAB user.
mqtt Push
This would be far more approachable from an openHAB perspective. We can subscribe to a single topic (or different ones if that floats your boat), parse out the data we need in our Item’s binding and all without doing anything special or complicated with rules within OH or need to code anything (binding or Rules).
rest Pull
Similarly as simple to setup from an openHAB perspective would be an HTTP REST pull. openHAB can periodically poll a URL for updates out of the box and the data returned from a single pull can be farmed out to multiple Items. But polling, yuck. Plus I saw nothing in your architecture to imply that polling would be easy to support on your side.
openhab Binding
I don’t know if it is necessary in this case but a separate binding could be written for openHAB. This binding can open up the standard events URL and internally parse out the JSON and update openHAB Items bound to it.
This would provide a more seamless out of the box experience for openHAB users. However, if you are planning to implement MQTT anyway, personally I think that would give you more bang for your buck. We would just need to add an article with instructions for how to set it up on the wiki and we would be good to go without needing to add new code to openHAB and without imposing a lot of extra work on openHAB users.
That’s my two cents anyway. I don’t speak for the community over all.
I think your right that either a REST PULL or MQTT Push is probably the most standard and also means it could work with any Home Automation product (not that we care about anyone else but openhab ;-)).
I had some time to procrastination and I decided to give this a shot at playing with. I successfully followed the instructions to get reelyActive installed and running on my Pi 3 and it does pick up my phone and FitBit in the Bubblescape.
So now I’m looking to add some integration with openHAB. I’d like to use MQTT so I started looking at the Integrate your Service tutorial and unfortunately, at least for someone who has only a passing knowledge of Node, there is not enough there to write my own barnacle. I want to try my hand at an MQTT barnacle as I think that would have a wider chance for use, at least in the HA space.
So the example barnacles are simple enough to follow and the messages to send are clear enough, but as someone who has done little more than “Hello World” level stuff with Node, I’m not all that certain what I’m thinking is the right approach is indeed the right approach.
In short:
I should, using the existing barnacles as an example, create one in /opt/reelyActive/hlc-server/node_modules/barnacles/lib/services/
Once written run npm install again?
???
Profit
What I don’t know is what or how to get what I’ve installed thus far to use this new barnacle.
Which clears some things up for me but it is still unclear for me how to apply what is present there with what was installed above. After running npm install do I just need to edit the pi-suite.js to add this new barnacle to the notifications? and away we go?
Once I get this working I would be willing to post the MQTT barnacle in a PR and I will certainly be posting a Tutorial here on the OH forums.
Create a file called server.js and paste in the following code:
var barnacles = require('barnacles');
var barnowl = require('barnowl');
var mqtt = require('mqtt'); // See www.npmjs.com/package/mqtt
var notifications = new barnacles();
var middleware = new barnowl();
middleware.bind( { protocol: 'hci', path: null } );
notifications.bind({ barnowl: middleware });
notifications.on('appearance', function(event) {
// Whatever you want to do when a Bluetooth device appears
});
notifications.on('displacement', function(event) {
// Whatever you want to do when a Bluetooth device moves to another Pi/reelceiver
});
notifications.on('disappearance', function(event) {
// Whatever you want to do when a Bluetooth device is no longer detected
});
notifications.on('keep-alive', function(event) {
// Whatever you want to do when a Bluetooth device sticks around
});
Enter all your MQTT code in server.js, and each time you want to run, just type from the command line:
sudo node server
Once you’re happy with the functionality, we can take a look at how to integrate it as a service, which will take the form of a file like this. Hope that’s a good start!
I’m still getting errors but I seem to be getting closer.
Error:
reelyActive Barnacles instance is notifying an open IoT
reelyActive BarnOwl instance is listening for an open IoT
barnacles is listening on port 3005
HCI Bluetooth address is b827eb5ee0a4, query as receiver 001bc5094b5ee0a4
net.js:641
throw new TypeError('invalid data');
^
TypeError: invalid data
at Socket.write (net.js:641:11)
at publish (/opt/reelyActive/ra-mqtt/node_modules/mqtt/node_modules/mqtt-packet/writeToStream.js:288:17)
at Object.generate [as writeToStream] (/opt/reelyActive/ra-mqtt/node_modules/mqtt/node_modules/mqtt-packet/writeToStream.js:22:14)
at sendPacket (/opt/reelyActive/ra-mqtt/node_modules/mqtt/lib/client.js:35:27)
at MqttClient._sendPacket (/opt/reelyActive/ra-mqtt/node_modules/mqtt/lib/client.js:670:7)
at MqttClient.publish (/opt/reelyActive/ra-mqtt/node_modules/mqtt/lib/client.js:380:12)
at Barnacles.<anonymous> (/opt/reelyActive/ra-mqtt/ra-mqtt.js:43:10)
at emitOne (events.js:77:13)
at Barnacles.emit (events.js:169:7)
at /opt/reelyActive/ra-mqtt/node_modules/barnacles/lib/barnacles.js:301:16
I think the error is in the MQTT library so I don’t expect you to be able to help much. My current working theory is I need to convert event into a JSON string.
My code thus far for the curious:
see below
UPDATE:
I’m successfully getting events from reelyActive into openHAB via MQTT. The trick was stringifying the event.
Here is the working reelyActive code and the openHAB Item to receive the flattened JSON.
Now the trick will be figuring out which deviceId goes with which physical device and I’ve got at least one reelyActive node. I’ve only one BTLE device (the Pi 3 itself) so it will be awhile before i can play with displacement events but I’m happy with my current start.
JavaScript
var barnacles = require('barnacles');
var barnowl = require('barnowl');
var reelib = require('reelib');
// MQTT Setup
var mqtt = require('mqtt');
var PORT = 1883;
var ID = 'reelyActive-cerberos';
var USER = 'user';
var PW = 'password';
var LWT_TOPIC = 'status/reelyActive/cerberos';
var LWT_PAYLOAD = 'reelyActive cerberos is down';
var PUB_TOPIC = 'reelyActive/cerberos';
// See https://github.com/mqttjs/MQTT.js/blob/master/examples/client/secure-client.js for a TLS example
// See https://www.npmjs.com/package/mqtt#client for available options
var options = {
port: PORT,
clientId: ID,
username: USER,
password: PW,
will: {
topic: LWT_TOPIC,
payload: LWT_PAYLOAD
}
};
client = mqtt.connect('mqtt://192.168.1.200', options);
// END MQTT Setup
// reelyActive Setup
var notifications = new barnacles();
var middleware = new barnowl();
middleware.bind( { protocol: 'hci', path: null } );
notifications.bind( { barnowl: middleware } );
// END reelyActive setup
// Event Handler
notifications.on('appearance', function(event){
// appears
handleEvent(event);
});
notifications.on('displacement', function(event){
// moves to another device
handleEvent(event);
});
notifications.on('disappearance', function(event){
// no longer detected
handleEvent(event);
});
notifications.on('keep-alive', function(event){
// redetection
handleEvent(event);
});
function handleEvent(event) {
var flattenedEvent = reelib.event.toFlattened(event);
console.log(flattenedEvent);
client.publish(PUB_TOPIC, JSON.stringify(flattenedEvent));
};
As I said above, mapping the deviceId to a physically known device is the last piece and then I can use a JSON transform on the Items to filter the events by specific devices and I will have successfully integrated it with openHAB.
So far so good.
Creating a generic barnacle might be a challenge for MQTT though. The problem is the number of possible combinations of parameters needed for making the connection to the MQTT broker. Without there being some sort of parameters file I’m not yet sure how to handle that in a way to handle all the cases (e.g. TLS, user/pw verses no user/pw, etc.).
Thanks for this. I was looking at this a wee while ago and hoped someone might have a crack at OH integration. Will have to have a go over the weekend.
We’ve finally integrated MQTT into our barnacles software package, which should simplify the code and integration.
Looking at your code @rlkoshak, I think you could simplify it as follows:
// User-defined
var CLIENT_OPTIONS = {
port: 1883,
clientId: 'reelyActive-cerberos',
username: 'user',
password: 'password',
will: {
topic: 'status/reelyActive/cerberos',
payload: 'reelyActive cerberos is down'
}
var TOPIC = 'reelyActive/cerberos';
var HOSTNAME = '192.168.1.200';
// Fire up the reelyActive hyperlocal context server
var server = require('hlc-server');
var app = new server();
// Listen on the BLE radio of the Pi 3
app.bind( { protocol: 'hci', path: 'null' } );
// Set up the MQTT client service
app.addNotificationService( {
service: "barnaclesmqtt",
hostname: HOSTNAME,
topic: TOPIC,
clientOptions: CLIENT_OPTIONS
});
Just need to install hlc-server, bluetooth-hci-socket and mqtt npm packages before you run:
Oh, and the other benefit of running the hyperlocal context server is that you can browse to port 3001 of your Pi and access the APIs and visualise the BLE devices detected via our bubblescape.
@reelyActive is there any demo code on how to acomplish that?
I would imagine, the code in your last post above would have to be changed on the “barnowl” nodes to send the detected pakets to a ‘central’ barnacle instance, correct?
How do I tell the instance on one Pi to send all detected pakets to a “central” instance that does the “displacement magic” and sends the corresponding MQTT messages to my broker?
Hi @ralle, in the case where you have more than one Raspberry Pi acting as a BLE sniffer, you would:
designate one Pi to be the “central” instance, and have it perform the MQTT forwarding
have the other Pis forward their real-time data to the “central” instance via our barnaclesrest service
Central Instance
Put the following in a file called server.js and from the command line run “node server.js”.
// User-defined
var CLIENT_OPTIONS = { /* MQTT client options go here */ }
var TOPIC = 'topic/goes/here';
var HOSTNAME = 'ip-address-goes-here';
// Fire up the reelyActive hyperlocal context server
var server = require('hlc-server');
var app = new server();
// Listen on the BLE radio of the Pi 3
app.bind( { protocol: 'hci', path: 'null' } );
// Set up the MQTT client service
app.addNotificationService( {
service: "barnaclesmqtt",
hostname: HOSTNAME,
topic: TOPIC,
clientOptions: CLIENT_OPTIONS
});
Other Instances
Put the following in a file called server.js and from the command line run “node server.js”. Don’t forget to update the IP address of the “central” instance below:
// Fire up the reelyActive hyperlocal context server
var server = require('hlc-server');
var app = new server();
// Listen on the BLE radio of the Pi 3
app.bind( { protocol: 'hci', path: 'null' } );
// Set up the barnacles REST forwarding service
app.addNotificationService( {
service: "barnaclesrest",
hostname: "192.168.1.101", // IP address of central instance
port: 3001
});
Remember that before running that code on the Pis you’ll need to install the requisite Node.js packages from the command line:
How do you check if it’s working? Browse to the “central” Pi, port 3001, for instance http://192.168.1.101:3001, and you should see the Hyperlocal Context webpage and packets coming in. You can browse the web APIs to confirm that data from all Pis is being received.
Hope that helps! Note that we’ve recently added an SD card image with just about everything pre-installed to our Make a Raspberry Pi Hub Tutorial.
I’m getting closer, I think. I created on one of my Pi (Pi zero W) the “other instances” version of the node.js code and on another node (one that has no BT at all, but a little more CPU that the little Pi zero) I created the server minus the listen to BLE line.
I can see that the zero is talking to the central instance by tcpdumping the traffic to port 3001. If I stop the central instance, the other one complains about an error POSTing to the remote barnacle.
I don’t see anything in the server side however. When I connect to http://centralnode:3001it tells me 'Currently processing 0 packets/s from 0 devices." and Bubblescape is completely empty,
Is there any way to debug what it is doing with the packets it receives over the REST interface?
I’ve followed your post to set up a central instance and a node that forwards to the central instance. I’m now trying to experiment to see how to exploit this technology but I’m running into a problem when trying to use the Hyperlocal Context. I see:
Currently processing 2 packets/s from 4 devices.
and checking the console on the node I can see:
HCI Bluetooth address is XXXXXXXXXXXXX, query as receiver 001XXXXXXXXXXXXX
but when I try and find devices at receiver 001XXXXXXXXXXXXX with
