Hi!
I would like to share how I’m implementing an openHAB voice assistant on a Raspberry Pi using Snips - a voice platform running on your device, with a strong focus on privacy.
In a nutshell, Snips performs Hotword Detection, Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) then outputs intents published on a MQTT topic. openHAB subscribes to these topics and perform actions on your items by interpreting the intent.
First, a disclaimer: I’m not in any way associated with Snips nor am I getting anything for promoting their product, this is actually my second attempt in my quest to build a DIY Amazon Echo/Google Home alternative which respects my privacy, see my CMU Sphinx Speech-to-Text for another (working!) solution. I decided to give Snips a try for various reasons, mostly because their hotword detection accuracy is far better, and because it allowed me to plug the microphone to another Raspberry Pi to put in a better location than my main openHAB server. It is now running in my living room with satisfying results so far!
The hardware:
- A Raspberry Pi 3 - I went with a starter kit (55 €) coming with a power supply, a case and a memory card preloaded with Raspbian because I’m dedicating this Pi to Snips; however, it should work just fine on openHABian along your openHAB server;
- A far-field microphone array - the Snips team actually put a blog post out benchmarking the best candidates, but I already had the MiniDSP UMA-8 (~110 € with shipping, having problems with it though) and a PlayStation Eye (7 €) which gave similar results in my experience. To start, I think the Eye is a great choice;
- A speaker to hear Snips’ audio feedback and dialogue (this is optional because openHAB can take care of this itself as well)
Installing Snips
Follow the documentation here: https://github.com/snipsco/snips-platform-documentation/wiki/1.-Setup-the-Snips-Voice-Platform-on-your-Raspberry-Pi
I went the easy route - enabled the SSH server on the Pi, connected to it and simply installed with:
curl https://install.snips.ai -sSf | sh.
Snips runs on Docker and this script will install it along with the other dependencies, and put commands like snips, snips-watch and snips-install-assistant in your PATH.
It will also attempt to configure your microphone and speakers with ALSA if you haven’t done so yourself.
Configuring the Google ASR (optional, for non-English speakers)
Snips has a built-in ASR for English running entirely on your device for maximum privacy, which is just fine if every user is comfortable with English, but for other languages*, for now you’ll have to use Google 
:
- sign up to the Google Cloud Platform at https://console.cloud.google.com (I created a dedicated Google account),
- configure billing including adding a payment method (unfortunately…) - even though there’s a free tier: you get 60 minutes per month for free then it’s $0,006/15 seconds, see https://cloud.google.com/speech/; You’ll also get approx. 250 € in credits to use the first year
- create a project (https://console.cloud.google.com/projectcreate)
- enable the Cloud Speech API (go to APIs & Services > Library, then Speech API, then Enable)
- create a service account (go to APIs & Services > Credentials) then Create credentials and Service account in the dropdown; fill out the form, choose JSON as the key type and click Create - you’ll get a JSON file download;
- Copy the JSON file from Google on your Snips RPi with SCP, SMB or another way, and put it in
/opt/snips/config/googlecredentials.json
* Snips claims to support French, German, Spanish and Korean in their documentation but German assistants cannot be created in the web console for now. It’s “coming soon” though IIRC.
Building & testing your assistant
Continue reading the documentation linked above (https://github.com/snipsco/snips-platform-documentation/wiki/2.-Running-your-first-end-to-end-assistant), and check out this video as well:
https://player.vimeo.com/video/223255884
I started from the IoT bundle to get the basic intents for operating lights and simple switches, but you can also add your own to launch scenes, control music etc.
Take time to properly test your assistant with the web interface to see how different sentences translate to an intent and slots because that’s what you’ll work with in openHAB later.
Once you’re satisfied with your assistant, download it from the Snips web console, copy it on your Raspberry Pi and install it (you can use the snips-install-assistant command if available).
Then run snips, let it start, run snips-watch in another terminal and say “Hey Snips” then a command - the output of snips-watch should display the events including the resulting intent.
Connecting to openHAB
As mentioned before Snips events are published on a MQTT broker, you can use your own but by default Snips will use its own Mosquitto instance running on port 9898.
I don’t run a separate MQTT broker for other things, so I simply had openHAB connect to Snips’ broker.
The MQTT binding will maintain a connection, subscribe to topics and update the state of openHAB items with messages coming from Snips.
- Install the MQTT binding in Paper UI
-
Configure the MQTT binding
edit
conf/services/mqtt.cfgand add the connection to the broker:
snips.url=tcp://<your_snips_ip>:9898
- Create openHAB Items to hold info from Snips
You can configure items in many ways but I suggest you create at least those below.
Create aconf/items/snips.itemsfile and add:
String Snips_Intent "Snips Intent" { mqtt="<[snips:hermes/nlu/intentParsed:state:default]" }
Switch Snips_Listening "Snips Listening" { mqtt="<[snips:hermes/asr/toggleOn:state:ON],<[snips:hermes/asr/toggleOff:state:OFF]" }
You can subscribe to other topics, in particulier intents are also published to the hermes/intent/<intentName> topics.
Perform actions in openHAB based on the intent
If everything is configured propertly, your Snips_Listening switch will turn on and off when Snips is listening, and most importantly, Snips_Intent will be updated with JSON string representations of the intents as they come.
How you react to those updates is entirely up to you. Typically, it will involve rules.
My solution will perhaps feel a little opinionated: I decided to leverage the JavaScript capabilities of the new experimental rules engine (mostly because JSON parsing is trivial), and the script editor in my Flows Builder app.
Here’s how to get started:
- Create a new flow
- Drag and drop a “When an item state is updated” trigger node, a “Execute a given script” action node on the design surface and link them up
- Click the trigger node and set
Snips_Intentas the Item - Click the action node, set Javascript (application/javascript) as the scripting language, and click Edit script…
Below is an example of my work in progress:
var intent = JSON.parse(state);
if (!intent.intent) {
print('Snips_Intent is not in the expected format!');
} else {
print('Snips intent name: ' + intent.intent.intentName);
}
// find slots of certain types for later reference
var locationSlot = intent.slots.filter(function (slot) {
return (slot.slotName === 'objectLocation');
});
var typeSlot = intent.slots.filter(function (slot) {
return (slot.slotName === 'objectType');
});
var colorSlot = intent.slots.filter(function (slot) {
return (slot.slotName === 'objectColor');
});
var type = (typeSlot.length) ? typeSlot[0].rawValue : null;
var location = (locationSlot.length) ? locationSlot[0].rawValue : null;
var color = (colorSlot.length) ? colorSlot[0].rawValue : null;
if (type) {
print('type=' + type);
}
if (location) {
print('location=' + location);
}
if (color) {
print('color=' + color);
}
function normalizeObjectType(t) {
if (!t || t.indexOf('lum') >= 0 || t.indexOf('lampe') >= 0)
return 'light';
// TODO other types
return t;
}
function getColorValue(c) {
switch (c) {
case 'blanc': return '0,0,100';
case 'rose': return '300,100,10';
case 'jaune': return '500,100,100';
case 'orange': return '25,100,100';
case 'vert': return '100,100,50';
case 'violet': return '280,100,100';
case 'bleu': return '200,100,100';
case 'rouge': return '0,100,100';
default: return '0,0,100';
}
}
function getLightItem(l) {
// operate all lights via the 'Lights' group if no location provided
if (!l)
return 'Lights';
// otherwise, assume the item name is "Hue_" + the capitalized location
return 'Hue_' + (l.charAt(0).toUpperCase() + l.slice(1));
}
var normalizedType = normalizeObjectType(type);
switch (intent.intent.intentName) {
case 'ActivateObject':
{
if (normalizedType === 'light') {
var item = getLightItem(location);
events.sendCommand(item, ON);
} else {
print('TODO: other object types');
}
}
break;
case 'DeactivateObject':
{
if (normalizedType === 'light') {
var item = getLightItem(location);
events.sendCommand(item, OFF);
} else {
print('TODO: other object types');
}
}
break;
case 'ActivateLightColor':
{
var item = getLightItem(location);
events.sendCommand(item, getColorValue(color));
}
break;
}
- Publish the rule by clicking on the “Publish” button in the toolbar
That’s it - Hope this helps, I’ll update with up-to-date info as necessary!
Cheers! 
) but I think I’ll give it a try.
