[OUTDATED Snips is dead!] Voice Control - How to use Snips AI and MQTT 2.4

Hi Guys,

i have a Problem with my Thing config.
I tried so much but nothing helps.

After i set the stateTopic at the channel the whole MQTT Bridge und the Thing goes offline.

2019-06-04 10:26:59.658 [INFO ] [.transport.mqtt.MqttBrokerConnection] - Starting MQTT broker connection to '192.168.0.140' with clientid openhab2 and file store '/var/lib/openhab2/mqtt/192.168.0.140'
2019-06-04 10:26:59.757 [INFO ] [ansport.mqtt.internal.ClientCallback] - MQTT connection to '192.168.0.140' was lost

With the 2.5 snapshot binding i have the same Problem, with cleared Cache.

Here is my Thing config:

Bridge mqtt:broker:SnipsAI "MQTT SnipsAI Broker" @ "MQTT" [ 
  host="192.168.0.140",
  secure=false,
  port=1883,
  qos=0,
  retainMessages=false,
  clientID="openhab2",
  keep_alive_time=30000,
  reconnect_time=60000,
  username="",
  password=""
]
{
    Thing topic SnipsAI "MQTT SnipsAI" @ "MQTT"  {
        Channels:
            Type string : Snips_Intent "Snips Intent" [ 
                stateTopic="hermes/intent/#:state:default",
                commandTopic="hermes/intent/#:state:default" 
            ]
    }
}

Hope you have any advices - i have no idea anymore what that could be.

thx
vaillan

Hello,
I see one thing, the intent topic should be configured as read only. Your replies to snips have to be published to the dialogueManager topics so you should remove the command topic for the intent channel.
Unfortunately I can’t give you any further hints as I don’t use the thing files for the mqtt binding and do all my configuration through Ui.
Have you tested if the IP and stuff are right using a tool like mqtt explorer? Can you connect to the Snips broker see all the Snips topics when you monitor this connection?
Best regards Johannes

Yes that will not work, because the topics are invalid. “#” must be the last character of a topic name. Whatever you have added after it will error out and the binding has only one way to tell you: By going offline.

i also tried that.

But now i deleted the .Things file and made it with UI.
The channel stays online now, i´ll give Feedback, if it is working now.

thx
vaillan

it is working now, thank you.

@Dan which snips app are you using for that configuration?

best regards
Vaillan

He probably made his own. But for the structure that he is working with I would guess that for this basic example he made two intents and one custom slottype in his assistant in the Snips console. The custom slottype is for the device names and contains the openhab item names and than aliases for each. He uses this custom slot type as a the only slot in both intents. He has one intent for switching on and one for switching of and trains each one with an appropriate number of example sentences where he marks the device slot in each. That way he gets the structure seen above. Correct me if I’m wrong @Dan
You could also create a second custom slot type which contains on and of and combine both intents in one and have a more complicated rule on the openhab side.
Using a available app you would always have to go through and insert all your own items so it’s easier to just build your own as its so simple.
Best regards Johannes

once the javascript helper libraries are all done I might rewrite my nodered code i use in conjunction with the openhab mqtt2 binding to work straight in openhab. This is my code in nodered for handling an intent from snips that contains any number of on off and dimming commands strung together in one intent and sends it on to the publish trigger topic of the mqtt binding. Its tried with strings of up to ten commands but in theory you could go on forever:

const alleitems = [
    ["item1", "openhabitem1"],
    ["item2", "openhabitem2"],
    ["item3", "openhabitem3"],
    ["item4", "openhabitem4"],
    ["item5", "openhabitem5"]
]
const dimitems = [
    ["item1", "openhabdimitem1"],
    ["item3", "openhabdimitem3"],
    ["item5", "openhabdimitem5"]
]
var id = msg.payload.sessionId;
flow.set("Id",id);
var slotsarrraw = msg.payload.slots;
var slotarr = [];
var col1 = [];
var col2 = [];
var dim1 = [];
var dim2 = [];
var stopic;
var spayload;
var output;
var outputdim;
var check;
var incnum = function(element) {
    return element.match(/^[0-9]*$/);
}
function colsplit(input) {
    for(var i = 0; i < input.length; i++) {
        col1.push(input[i][0]);
        col2.push(input[i][1]);
    }
}
function dimsplit(input) {
    for(var i = 0; i < input.length; i++) {
        dim1.push(input[i][0]);
        dim2.push(input[i][1]);
    }
}
if (slotsarrraw.length > 1) {
    for(i = 0; i < slotsarrraw.length; i++) {
        slotarr.push(slotsarrraw[i].value.value.toString());
    }
    if (slotarr.includes("einschalten") === false && slotarr.includes("ausschalten") === false && slotarr.some(incnum) === false) return;
    colsplit(alleitems);
    dimsplit(dimitems);
    for(var a = 0; a < slotarr.length; a++) {
        if (col1.includes(slotarr[a])) {
            output = col1.indexOf(slotarr[a]);
            output = col2[output];
            outputdim = dim1.indexOf(slotarr[a]);
            outputdim = dim2[outputdim];
            check = 0;
            for(var b = a + 1; check === 0; b++) {
                if (slotarr[b] === "ausschalten") {
                    stopic = "allItemsin/" + output;
                    spayload = "OFF";
                    node.send({topic:stopic, payload:spayload});
                    check = 1;
                } else if (slotarr[b] === "einschalten") {
                    stopic = "allItemsin/" + output;
                    spayload = "ON";
                    node.send({topic:stopic, payload:spayload});
                    check = 1;
                } else if (slotarr[b].match(/^[0-9]*$/) && dim1.includes(slotarr[a])) {
                    stopic = "allItemsin/" + outputdim;
                    spayload = slotarr[b];
                    node.send({topic:stopic, payload:spayload});
                    check = 1;
                } else {
                    if(b < slotarr.length) {
                        check = 0;
                    } else {
                        check = 0;
                        b = -1;
                    }
                }
            }
        }
    }
} else return;

Hi Vaillan,
right now I’m trying to create my own snips-app to communicate with openhab2…
but my progress is pending cause I got other things to do right now… But I saw that there are few openhab apps for snips in progress, so maybe there’s some useful stuff for you…

If I get any further with my own app I post a link…

and like @JGKK mentioned you can create your own otherwise… I think he gave a good starting point for doing that :wink:

1 Like

Great tutorial, @Dan! I plan to take my very first steps with it this weekend and I am looking forward to it! :smiley:

One remark: Although I have not tried it yet but I believe that something like

if (onlyIntent == 'LampsOn') { ... }

will not work because of

onlyIntent = onlyIntent.toLowerCase

converting the intent’s characters to lowercase letters before. So it should read

if (onlyIntent == 'lampson') { ... }

in this case.

Thank you so much @Dan! for sharing this tutorial, I created my account because I had some troubles implementing your tutorial so i could ask some questions. Luckily I figured it out now :D, Snips is integrated with openhab now and I can control my lights. Really happy with microphone of the PS eye, picks up my voice from the entire room.

Great tutorial, everything works great. But I have one problem, after saying command Snips ends session with error. Can someone help me to send endsession command to Snips via MQTT? I have no idea how to do it

If you scroll to the bottom of the Snips jython tutorial I wrote (Tutorial for using the Snips Voice Assistant with Jython and Mqtt for switching and dimming lights) you can see how to do it. But the basis is to extract the session ID of the original message object to a variable and than publish a session end with the extracted session ID using the mqtt publish action in the correct format to hermes/dialogueManager/endSession.
You can find details on the session ID here:
https://docs.snips.ai/reference/dialogue#intent
and on the session end here:
https://docs.snips.ai/reference/dialogue#end-session

Best regards Johannes

Hi,

I have big trouble to get the mqtt2 binding running with snips.
I´m very new in the topic of mqtt and it is my first work with this binding.

I tried this tutorial in this post but nothing happens.
I also tried this one https://community.openhab.org/t/tutorial-for-using-the-snips-voice-assistant-with-jython-and-mqtt-for-switching-and-dimming-lights/76689 but with no result.

With the Paper Ui configuration the first error is when I try to set up a channel for the generic mqtt thing is “ERROR 500 - Internal Server Error”. When I can create such a channel and I type
hermes/intent/#:state:default
for the state Topic Im recieving: java.lang.Exception: No MQTT client and again nothing is working. With the mqtt Binding everything works fine. But I want to use the newer one.

It is exasperating. I don´t no where is my mistake. Can someone Help me please.

Hmm, you will probably have to give us some more information on your setup. Unfortunately I don’t know how to set up the mqtt binding with thing files as I only use it on conjunction with paper Ui.
Did you create a broker thing connected to the mosquito broker that snips installs and uses? When you use the connection details that you use for the Snips broker thing in openhab to connect to it with something like mqtt-explorer do you get a successful connection and can you see snips publish to the topics you want to use?
Did you choose the right broker (the Snips mosquito thing you created) when you created your generic mqtt thing?
It looks to me like either something is wrong with your broker thing connecting to the Snips broker or something is wrong with your generic thing definition.
Have you tried following the steps of the tutorial I wrote concerning the mqtt connection step by step using paper Ui?
You will probably have to post more details to get to the bottom of this.
Best regards Johannes

It is working now.

@JGKK thank you for your answer.
The solution was in post from @David_Graeff

that will not work, because the topics are invalid. “#” must be the last character of a topic name

How can I send item values to my snips.ai?
For Example when I ask “How is the temperature in the living room” and Snips tell me the actual temperature in this room?
an other question I have is: is it better to have channels for every Intent or divide the Intent in my rule and have a very big snips rule?

Thank you very much.

I extracted sessionid, but I have problem with this line

“endsession = json.dumps({“sessionId”:sessionid})”
Everytime I have this error
Configuration model ‘snips.rules’ has errors, therefore ignoring it: [36,38]: mismatched input ‘:’ expecting ‘}’
[36,39]: missing ‘)’ at ‘sessionid’

Because of it, I can’t publish “sessionId:xxx-xxx-xxx-xxx” to Snips, I can only publish “xxx-xxx-xxx-xxx” if I remove “:” from command, but It isn’t enough to end session. It looks like openhab doesn’t like “:” mark

I think you used the syntax from the jython tutorial in a rules DSL rule. This will not work. You will have to use the syntax of the rules DSL for json formatting. Unfortunately I don’t know that :confused:. A way would be to use a Javascript transformation maybe.
Best regards Johannes

1 Like

You’re right, now I see my stupid mistakes. Thanks for help, now I know what to do :slight_smile:

Snips AI looks really interesting, I will test it soon. Maybe we could propose a simplified integration in openHAB that could hide MQTT and complex rules?