Hello,
As I really love the Snips openhab combination and I just moved my integration from Nodered to Jython rules for Light intents I thought I would share a relatively complete guide on how to quickly set up an integration. The final aim is to have a voice integration which has some advantages over commercial solutions that make it worth the extra effort. Mainly Stringing commands together and having multiple synonyms for every Item without extra rules and all this without sending your voice to the cloud everytime you want to switch a light. So here we go:
First you will have to install Snips on a RaspberryPi or similar. I would recommend not installing Snips on the same Pi as Openhab as you will probably have memory issues. A guide from Snips on Hardware including a microphone array that you will have to connect to the machine running Snips can be found here:
https://docs.snips.ai/articles/raspberrypi/hardware
They also have fairly good installation Instructions here:
https://docs.snips.ai/getting-started/quick-start-raspberry-pi
Now we have to create an account for the Snips console as this is were we will create our intents and train them to be downloaded to our assistant.
Here you create a new assistant. Once you have done this you have to create an app inside this assistant. I created an example app with a light intent you can add to see the structure the Jython code later on will expect. You can find it here:
Or search for “OpenhabJythonSnipsTutorial” with the Tickbox for only show apps with actions removed.
There is one Intent in this app which is the light intent. If you fork it you can edit it to see its structure and adapt it to your system.
You can see it has 3 Slot Types. One for dimming, one for switch values and one for Item names.
You need to edit the one for the names:
In the value column you have to insert your Openhab Item Names you want to Switch as Snips can only know words and names you explicitly teach the model. Into the right column you can put synonyms for those Items which need to be separated by a comma.
As a next Step you have to use those Names in the model sentences on the first page of the intent. You will have to substitute Item 1, Item2 etc with your Items and synonyms and mark them as type itemname. The more examples you write the better your local model will be. Once you adapted the itemname slot and have rewritten the training sentences you can save it and try if it works on the right side of the console. Type in a sentence and you will see what snips understands in the json output:
This is very similar to what Snips will send over Mqtt.
Now you need to download this assistant model to your Snips Pi using the Sam Tool:
https://docs.snips.ai/reference/sam
Check if your assistant works using the
sam watch
command on your computer.
If you use a tool like like http://mqtt-explorer.com/ and connect it to the snips broker you will now be able to see the topic that snips publishes its intents too when you talk to it. It should be
hermes/intent/JGKK:lights
The next step will be to actually connect Openhab to Snips so that it can receive intents. This happens over MQTT which the whole Snips communication is based on (https://docs.snips.ai/reference).
Snips installs its own instance of an MQTT Broker on the machine it runs. In the standard settings it is reachable without username or password on the standard port.
So all we have to do is to add a Broker Thing with the MQTT2 Binding that points at the IP address of the Snips PI
Once we have done this we create a generic mqtt thing connected to the Snips broker thing and add a text value channel subscribed to the intent topic to it:
Now we create a String item linked to this
String SnipsLicht {channel="mqtt:topic:919dc187:snips_light"}
Now all that we need to do is write a rule to do something with the intents we receive. I wrote mine in Jython which you can enable follwing the awesome tutotrials here:
https://openhab-scripters.github.io/openhab-helper-libraries/index.html
My approach for english is pair based. So this code is fairly simple. It extracts the slot values from the Json Object and than starts at the beginning and every two slots sends a command if it has found a switching/ dimming value and an Item. So you need to use one command for every Item when you speak to Snips but you can String those together. Eg “turn on Item 1 and dim Item2 to 50 percent” works but “turn on Item 1 and Item 2” wont. This will only turn on the first Item. At the end it publishes a session end to snips.
from core.rules import rule
from core.triggers import when
#we need to import json and regex parsing
import json
import re
@rule("Licht Intent")
@when("SnipsLicht")
def SnipsLichtIntent(event):
intentarrraw = json.loads(str(event.itemState)) #convert the raw intent data from json
sessionid = intentarrraw['sessionId'] #saving the sessionid for later
intentvalues = []
item = None
command = None
valuecounter = 0
for value in intentarrraw['slots']: #getting the actual slot values of the snips intent
intentvalues.append(value['value']['value'])
if len(intentvalues) >= 2: #making sure that the intent is valid
for element in intentvalues:
if (element != "ON" and element != "OFF" and re.match("^[0-9]*\.[0-9]*$", str(element)) == None): #looking for item names
item = element
valuecounter += 1
else:
command = str(element)
valuecounter += 1
if valuecounter == 2: #treating all commands as pairs
if item != None and command != None: #making sure their is a command and item
events.sendCommand(item, command) #send the command we found to the item
valuecounter = 0
endsession = json.dumps({"sessionId":sessionid})
actions.get("mqtt","mqtt:broker:6836498e").publishMQTT("hermes/dialogueManager/endSession",endsession) #publish a succesful session end to snips
For the German users I would offer a different approach which works with a look ahead and is more flexible not needing this one to one ratio for items to commands as in German grammar the command always comes after the name so there you can string together multiple Items for one command as long as the command comes after the Items.
from core.rules import rule
from core.triggers import when
#we need to import json and regex parsing
import json
import re
@rule("Licht Intent")
@when("SnipsLicht")
def SnipsLichtIntent(event):
intentarrraw = json.loads(str(event.itemState)) #convert the raw intent data from json
sessionid = intentarrraw['sessionId'] #saving the sessionid for later
intentvalues = []
for value in intentarrraw['slots']: #getting the actual slot values of the snips intent
intentvalues.append(value['value']['value'])
if (len(intentvalues) >= 2 and ("ON" in intentvalues or "OFF" in intentvalues or any(re.match("^[0-9]*\.[0-9]*$", str(x)) != None for x in intentvalues))): #making sure that the intent is valid and we will not be caught in a loop
for element in intentvalues:
if (element != "ON" and element != "OFF" and re.match("^[0-9]*\.[0-9]*$", str(element)) == None): #looking for item names
item = element
if intentvalues.index(element) == (len(intentvalues) - 1): #looking ahead for a command if the item is not the last place in the list
commandindex = 0
else:
commandindex = intentvalues.index(element) + 1
while (intentvalues[commandindex] != "ON" and intentvalues[commandindex] != "OFF" and re.match("^[0-9]*\.[0-9]*$", str(intentvalues[commandindex])) == None): #keep looking ahead till a valid command is found
if commandindex < (len(intentvalues) - 1):
commandindex += 1
else:
commandindex = 0
else:
events.sendCommand(item, str(intentvalues[commandindex])) #send the command we found to the item and repeat till done
endsession = json.dumps({"sessionId":sessionid,"text":"[[sound:mysound]]"})
actions.get("mqtt","mqtt:broker:6836498e").publishMQTT("hermes/dialogueManager/endSession",endsession) #publish a succesful session end to snips
As Jython doesnt handle special characters like ä,ö,ü und ß well for german users I would also add an incoming javascript transformation to the intent channel to remove those as even if you dont have them in item names snips will send them in written german numbers
(function(i) {
snipsinput = i.replace(/[ä]/g, "ae").replace(/[ö]/g, "oe").replace(/[ü]/g, "ue").replace(/[ß]/g, "ss");
return snipsinput;
})(input)
I hope this tutorial gave you some inspiration for using snips and works as a starting point for switching from alexa or google home.
As this is only a basic example and my first tutorial I welcome any feedback.
Best regards Johannes