Rhasspy to HABot

Hi,

I’ve setup a Rhasspy installation and was wondering if it is possible to simply send the “output text” (the STT output) of rhasspy as an input to HABot (like what I would type in the chat field).

I found the solution with the following:

In Rhasspy Settings, I do not recommend to set the MQTT broker as your main openHAB one, as the audio stream goes through it (since Rhasspy 2.5). I am using a single setup, so have the Rhasspy MQTT “internal” broker. This broker runs in Rhasspy Docker (unless it is not installed with docker). In order for openHAB to be able to connect to the Thasspy MQTT broker, the docker settings must expose the MQTT port 12183:

docker run -d -p 12101:12101 -p 12183:12183 \
      --name rhasspy \
...

Within openHAB, I created a MQTT broker connection to the Rhasspy MQTT broker (no username/password, so somewhat unsafe):

I am catching the Rhasspy command through the MQTT messages:

things:
    // Rhasspy
    Thing mqtt:topic:rhasspy "Rhasspy MQTT" (mqtt:broker:rhasspy) {
Channels:
    Type string : wakeword "Wakeword [%s]" [ stateTopic="hermes/asr/toggleOn", transformationPattern="JSONPATH:$.reason"  ]
    Type string : transcription "STT [%s]" [ stateTopic="hermes/asr/textCaptured", transformationPattern="JSONPATH:$.text" ]

}

items:
    // Rhasspy
    Group gRhasspy "Rhasspy" <recorder> (gAll) 
      String Wakeword "Rhasspy Wakeword [%s]" <recorder> (gRhasspy) {channel="mqtt:topic:rhasspy:wakeword"}
    String VoiceCommand "Rhasspy STT [%s]" (gRhasspy) {channel="mqtt:topic:rhasspy:transcription"}
    String HABotResponse "HABot Response [%s]" <recorder> (gRhasspy)

And this is the rule:

    rule "VoiceCommand"
    when
        Item VoiceCommand received update
    then
        //define HTTP REST POST request
        var url = "http://YOUR.IP.OF.OPENHAB:8080/rest/habot/chat"
        var contenttype = "text/plain"
        var String POSTrequest = VoiceCommand.state.toString

        //send the request
        var output = sendHttpPostRequest(url, contenttype, POSTrequest)

        // log response (JSON)
        logInfo("HABot Chat:", output);

        // transform response
        var String response = transform("JSONPATH", "$.answer", output)
        logInfo("HABot answer", response)
        postUpdate(HABotResponse, response)
    end

Have fun!

Edit: Since Rhasspy 2.5, the mqtt server has been changed and I have updated the explanation.

5 Likes

Thanks for posting! This is a really cool idea. I’ve moved it to the Tutorials & Examples: Solutions category so more people will see it.

Thanks - sounds good! I could improve on the documentation (settings for Rhasspy, items) if needed.

1 Like

Can you share your items? (ex VoiceCommand)

I edited the first post! Enjoy!

First post updated due to changes in Rhasspy 2.5.x MQTT structure. Added some more details on the configuration of Rhasspy and openHAB.

Hi SpaceGlider,
thanks for this wonderful tutorial.
I am working on exactly this but cannot get it to work.
It would be helpful for me to get some information how Rhasspy has to be configured in the web interface.

My setup is as follows:
openHAB 2.5.11
Rhasspy 2.5.8 (native install on openhabian, no docker)
I can get nearly everything to work, openHAB connects to Rhasspy and Rhasspy transcribes the audio successfully:
[DEBUG:2020-12-31 16:34:04,830] rhasspyasr_kaldi_hermes: Transcription result: Transcription(text=‘wie ist die temperatur im wohnzimmer’, likelihood=1, transcribe_seconds=5.502065632957965, wav_seconds=2.55, tokens=None)

But somehow the output is not sent via MQTT, I also checked with MQTT Explorer.

My Rhasspy settings page looks like this:

Do you have any clue, what might be the problem?

Edit: I already tried different combinations of config, but did not succeed.

Thank you and kind regards
Edcrfv

Hi @Edcrfv

sorry for the late answer.

In openhab, you must connect a MQTT listener to the Rhasspy-internal MQTT server.

The port in rhasspy must be exposed, so that the openhab can connect to the rhasspy-internal MQTT server. It is a bit complex, but that is the only way I got it to work!

Hi @SpaceGlider,
thanks for your help.
But the connection between Rhasspy and openhab seems to work (rhasspy.log):
1612711771: New connection from 192.168.3.7 on port 12183.
1612711771: New client connected from 192.168.3.7 as 2c5d1eb5-b1aa-4212-8494-43b83b85a22f (c1, k60).

It seems that Rhasspy can transcribe the audio but does not send it via MQTT:
[DEBUG:2021-02-07 16:34:11,689] rhasspyasr_kaldi.transcribe: Decoder started
[DEBUG:2021-02-07 16:34:11,767] rhasspyasr_kaldi.transcribe: Finished stream. Getting transcription.
[DEBUG:2021-02-07 16:34:12,175] rhasspyasr_kaldi_hermes: -> AsrTextCaptured(text=’’, likelihood=0, seconds=0, site_id=‘default’, session_id=‘470937ec-f8f7-4add-b4d9-a02e81bfa9f0’, wakeword_id=None, asr_tokens=None, lang=None)
[DEBUG:2021-02-07 16:34:12,177] rhasspyasr_kaldi_hermes: Publishing 170 bytes(s) to hermes/asr/textCaptured
[DEBUG:2021-02-07 16:34:12,179] rhasspyasr_kaldi_hermes: -> AsrAudioCaptured(81644 byte(s)) to rhasspy/asr/default/default/audioCaptured
[DEBUG:2021-02-07 16:34:12,181] rhasspyasr_kaldi_hermes: Stopping listening (session_id=470937ec-f8f7-4add-b4d9-a02e81bfa9f0)
[DEBUG:2021-02-07 16:34:12,203] rhasspyserver_hermes: Handling AsrTextCaptured (topic=hermes/asr/textCaptured, id=69536721-740e-440d-b437-500594b1ff19)
[DEBUG:2021-02-07 16:34:12,215] rhasspyserver_hermes: Waiting for intent
[ERROR:2021-02-07 16:34:12,216] rhasspyserver_hermes: No intent system configured
Traceback (most recent call last):
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1821, in full_dispatch_request
result = await self.dispatch_request(request_context)
File “/usr/lib/rhasspy/usr/local/lib/python3.7/site-packages/quart/app.py”, line 1869, in dispatch_request
return await handler(**request_.view_args)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 1476, in api_speech_to_intent
user_entities=user_entities,
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/main.py”, line 2641, in text_to_intent_dict
result = await core.recognize_intent(text, intent_filter=intent_filter)
File “/usr/lib/rhasspy/rhasspy-server-hermes/rhasspyserver_hermes/init.py”, line 488, in recognize_intent
raise NluException(“No intent system configured”)
rhasspyserver_hermes.NluException: No intent system configured
[DEBUG:2021-02-07 16:34:17,512] rhasspyasr_kaldi.transcribe: ['wie ist die ', 'wie ist die temperatur im wohnzimmer ', 'wie ist die temperatur im wohnzimmer ']
[DEBUG:2021-02-07 16:34:17,514] rhasspyasr_kaldi_hermes: Transcription result: Transcription(text=‘wie ist die temperatur im wohnzimmer’, likelihood=1, transcribe_seconds=5.746688758023083, wav_seconds=2.55, tokens=None)

I could “prove” this with MQTT Explorer as the output is empty:

Can you thereforce give me any hint, how Rhasspy has to be configured in detail?
Especially the intention handling seems to be a problem…

Thank you very much!
I am really looking forward to get this to work :slight_smile:

Kind regards
Edcrfv

I haven’t updated to 2.5.9 yet - maybe something changed?
Edit: I’m using docker.

This is my settings page:

I just checked it also with Kaldi (not Deepspeech) and it works for me.

I also updated rhasspy to 2.5.9

grafik

Here is the Kaldi config (honestly I made no changes from the standard)

Hope you can get it to work!

Hello @SpaceGlider,
thank you very much for your support!
I switched to Docker, played around with the settings a little bit and what should I say:
It works :smiley:

To be honest I have no idea why the native debian package does not work.
It might be some packages or prerequisites missing.

If I have some spare time in the future I might check this again.
But my feeling is, that the docker solution is much more performant.
Maybe I also integrate my big debian server as Remote HTTP Host.

One (hopefully) last question:
Are you using sentences or “Open transcription mode”.
I tend to use sentences as it is much faster and the abilities of HABot are limited anyway.

Thank you very much again and have a nice weekend!

Kind regards
Edcrfv

I’ve played with both modes. With the mozilla TTS, even the open transcription is pretty fast on my RPi 4B since version 2.5.9!

I’m hoping that HABot gets a bit better soon. There seems to be an issue with HABot recognizing the semantic model. I’ve opened a ticket:

Cheers

1 Like

Hi guys,

first of all thank you :wink: I got it to run basically according to the instructions :muscle:

But now I have the problem that Rhasspy always passes spoken numbers like 20 degrees not as a number but as a word to haBot. At least in German he can not (yet) handle it. Do you have an idea how I can get this solved ?

And how do I get the value [enhab.core.model.script.HABot answer] via MQTT back to Rhasspy so I can get the answer on the TTS ?

Best,

Hagi

Hi @haginator ,
sorry for the very very late reply.

when you look at the full MQTT response (I use MQTTexplorer) then you see under IntentParsed the number in numerical format. You have to extract it from there.

I’m catching IntentName and IntentNumber as extra MQTT messages to use them:


String IntentName       "Rhasspy Intent [%s]"               (gRhasspy) ["Status"] {channel="mqtt:topic:rhasspy:intentName"}
Number IntentNumber     "Rhasspy Intent Number [%.0f]"      (gRhasspy) ["Status"] {channel="mqtt:topic:rhasspy:intentNumber"}