Hello community,
I’m trying to set up @Miguel_M.A.D 's great tutorial Dialog Processing with the PulseAudioBinding. First of all, thanks for this detailed guide.
I’m struggling to parse voice commands. The interpretation process always says that there was some error. Playing audio on the pulseaudio server / satelite works.
I followed the guide strictly - beside that I like config file based configuration on openHAB side.
Raspberry Pi with Pulseaudio as satelite
I set up the raspberry and pulseaudio exactly like in the guide.
openHAB
pulseaudio.things:
According to openHAB all things are online.
Bridge pulseaudio:bridge:voiceassistant1 "Voice Assistant 1" @ "office" [ host="192.168.24.138", port=4712 ] {
Things:
Thing sink voiceassistant1_speaker "Voice Assistant 1 Lautsprecher" @ "office" [name="alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo", activateSimpleProtocolSink=true, simpleProtocolSinkPort=4713]
Thing source voiceassistant1_mic "Voice Assistant 1 Mikrofon" @ "office" [name="alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo.monitor", activateSimpleProtocolSource=true, simpleProtocolSourcePort=4714]
}
voice_commands.items
String VoiceCommand
Switch DialogRunning
rustpotterks.cfg
org.openhab.voice.rustpotterks:threshold=0.5
org.openhab.voice.rustpotterks:averagedthreshold=0.2
org.openhab.voice.rustpotterks:scoreMode=max
org.openhab.voice.rustpotterks:minScores=5
org.openhab.voice.rustpotterks:scoreRef=0.22
org.openhab.voice.rustpotterks:bandSize=5
org.openhab.voice.rustpotterks:gainNormalizer=true
org.openhab.voice.rustpotterks:minGain=0.5
org.openhab.voice.rustpotterks:maxGain=1
org.openhab.voice.rustpotterks:gainRef=0.004
voicerss.cfg
apiKey=b7b08xxxxxxxxxxxxxxxxe1be3b
voskstt.cfg
org.openhab.voice.voskstt:preloadModel=false
org.openhab.voice.voskstt:singleUtteranceMode=true
org.openhab.voice.voskstt:maxTranscriptionSeconds=60
org.openhab.voice.voskstt:maxSilenceSeconds=5
org.openhab.voice.voskstt:noResultsMessage="Ich habe dich nicht verstanden."
org.openhab.voice.voskstt:errorMessage="Etwas ist schief gelaufen."
Model
Downloaded vosk-model-small-de-0.15 and placed here:
ls -la ./openhab_userdata/vosk/model
total 16
drwxr-xr-x+ 1 9001 9001 62 Apr 24 15:57 .
drwxr-xr-x 1 9001 9001 10 Apr 24 15:57 ..
drwxr-xr-x+ 1 9001 9001 18 Apr 24 15:57 am
drwxr-xr-x+ 1 9001 9001 38 Apr 24 15:57 conf
-rwxr-xr-x+ 1 9001 9001 10173 Dec 3 2020 COPYING
drwxr-xr-x+ 1 9001 9001 72 Apr 24 15:57 graph
drwxr-xr-x+ 1 9001 9001 142 Apr 24 15:57 ivector
-rwxr-xr-x+ 1 9001 9001 795 Dec 3 2020 README
runtime.cfg
org.openhab.voice:defaultTTS=voicerss
org.openhab.voice:defaultVoice=voicerss:deDE_Hanna
org.openhab.voice:defaultSTT=voskstt
org.openhab.voice:defaultKS=rustpotterks
org.openhab.voice:keyword=hey openhab
org.openhab.voice:listeningItem=DialogRunning
Dialog
I set up and can the see the dialog.
openhab> openhab:voice dialogregs
Source: pulseaudio:source:voiceassistant1:voiceassistant1_mic - Sink: pulseaudio:sink:voiceassistant1:voiceassistant1_speaker (STT: **Default**, TTS: **Default**, HLIs: **Default**, KS: **Default**, Keyword: hey openhab)
Testing
For testing I tried starting a dialog without magic / wake word like this:
openhab:voice listenandanswer --source pulseaudio:source:voiceassistant1:voiceassistant1_mic --sink pulseaudio:sink:voiceassistant1:voiceassistant1_speaker
Logs
I set org.openhab.core.voice and org.openhab.binding.pulseaudio to DEBUG.
I attached 3 runs because they are always slightly different. 
Run 1
2024-04-24 21:00:13.187 [DEBUG] [core.voice.internal.VoiceManagerImpl] - Executing a simple dialog for active source Voice Assistant 1 Mikrofon (pulseaudio:source:voiceassistant1:voiceassistant1_mic)
2024-04-24 21:00:13.188 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Adding new client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 1
2024-04-24 21:00:13.188 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:00:13.819 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStartEvent event received
==> openhab_userdata/logs/events.log <==
2024-04-24 21:00:13.820 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command ON
2024-04-24 21:00:13.820 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from OFF to ON
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:00:14.676 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStopEvent event received
2024-04-24 21:00:14.677 [DEBUG] [.core.voice.internal.DialogProcessor] - SpeechRecognitionEvent event received
2024-04-24 21:00:14.677 [DEBUG] [.core.voice.internal.DialogProcessor] - Text recognized: fehler ist bei der text erkennung aufgetreten ich habe dich nicht verstanden
2024-04-24 21:00:14.677 [DEBUG] [.core.voice.internal.DialogProcessor] - Interpretation result:
==> openhab_userdata/logs/events.log <==
2024-04-24 21:00:14.677 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:00:14.677 [DEBUG] [.core.voice.internal.DialogProcessor] - Empty value, nothing to say
==> openhab_userdata/logs/events.log <==
2024-04-24 21:00:14.677 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'VoiceCommand' received command fehler ist bei der text erkennung aufgetreten ich habe dich nicht verstanden
2024-04-24 21:00:14.677 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from ON to OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:00:14.678 [INFO ] [.openhab.core.model.script.Voice.Rec] - VoiceCommand received fehler ist bei der text erkennung aufgetreten ich habe dich nicht verstanden
2024-04-24 21:00:14.767 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Removing client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 0
2024-04-24 21:00:14.767 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:00:14.767 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:00:32.375 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream disconnecting for Voice Assistant 1 Lautsprecher
2024-04-24 21:00:40.935 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sinks
2024-04-24 21:00:41.453 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sources
2024-04-24 21:00:41.969 [DEBUG] [pulseaudio.internal.PulseaudioClient] - Pulseaudio server 192.168.24.138: 13 modules and 2 items updated
2024-04-24 21:00:41.969 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Source #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo.monitor, muted: false, state: RUNNING, volume: 100 id: volume
2024-04-24 21:00:41.970 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Sink #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo, muted: false, state: RUNNING, volume: 48 id: volume
2024-04-24 21:00:44.768 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream disconnecting for Voice Assistant 1 Mikrofon
Run 2
2024-04-24 21:02:21.999 [DEBUG] [core.voice.internal.VoiceManagerImpl] - Executing a simple dialog for active source Voice Assistant 1 Mikrofon (pulseaudio:source:voiceassistant1:voiceassistant1_mic)
2024-04-24 21:02:22.001 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream connecting for Voice Assistant 1 Mikrofon
2024-04-24 21:02:22.003 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:02:22.003 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:02:22.003 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Adding new client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 1
2024-04-24 21:02:22.003 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:02:22.615 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStartEvent event received
==> openhab_userdata/logs/events.log <==
2024-04-24 21:02:22.616 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command ON
2024-04-24 21:02:22.616 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from OFF to ON
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:02:27.393 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStopEvent event received
2024-04-24 21:02:27.394 [DEBUG] [.core.voice.internal.DialogProcessor] - SpeechRecognitionErrorEvent event received
==> openhab_userdata/logs/events.log <==
2024-04-24 21:02:27.394 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:02:27.394 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Adding new client for pulseaudio sink/source Voice Assistant 1 Lautsprecher. Current count: 1
2024-04-24 21:02:27.394 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
==> openhab_userdata/logs/events.log <==
2024-04-24 21:02:27.394 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from ON to OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:02:27.395 [DEBUG] [eaudio.internal.ConvertedInputStream] - Duration of input stream : 6810
2024-04-24 21:02:27.395 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is a MP3. Trying to reencode it
2024-04-24 21:02:27.396 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is not in the target format. Trying to reencode it
2024-04-24 21:02:27.396 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream connecting for Voice Assistant 1 Lautsprecher
2024-04-24 21:02:29.836 [DEBUG] [seaudio.internal.PulseAudioAudioSink] - Some time to let the system play sound : 4375
2024-04-24 21:02:29.836 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Removing client for pulseaudio sink/source Voice Assistant 1 Lautsprecher. Current count: 0
2024-04-24 21:02:29.836 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:02:29.836 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:02:29.915 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Removing client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 0
2024-04-24 21:02:29.915 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:02:29.916 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:02:29.916 [WARN ] [audio.internal.PulseAudioAudioSource] - IOException while writing to from pulse source pipe: Pipe closed
2024-04-24 21:02:47.109 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sinks
2024-04-24 21:02:47.671 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sources
2024-04-24 21:02:48.191 [DEBUG] [pulseaudio.internal.PulseaudioClient] - Pulseaudio server 192.168.24.138: 13 modules and 2 items updated
2024-04-24 21:02:48.191 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Source #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo.monitor, muted: false, state: RUNNING, volume: 100 id: volume
2024-04-24 21:02:48.191 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Sink #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo, muted: false, state: RUNNING, volume: 48 id: volume
Run 3
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:03:13.143 [DEBUG] [core.voice.internal.VoiceManagerImpl] - Executing a simple dialog for active source Voice Assistant 1 Mikrofon (pulseaudio:source:voiceassistant1:voiceassistant1_mic)
2024-04-24 21:03:13.143 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream connecting for Voice Assistant 1 Mikrofon
2024-04-24 21:03:13.145 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:03:13.145 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:03:13.146 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Adding new client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 1
2024-04-24 21:03:13.146 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:03:13.766 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStartEvent event received
==> openhab_userdata/logs/events.log <==
2024-04-24 21:03:13.766 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command ON
2024-04-24 21:03:13.767 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from OFF to ON
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:03:18.559 [DEBUG] [.core.voice.internal.DialogProcessor] - RecognitionStopEvent event received
2024-04-24 21:03:18.559 [DEBUG] [.core.voice.internal.DialogProcessor] - SpeechRecognitionErrorEvent event received
==> openhab_userdata/logs/events.log <==
2024-04-24 21:03:18.559 [INFO ] [openhab.event.ItemCommandEvent ] - Item 'DialogRunning' received command OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:03:18.560 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Adding new client for pulseaudio sink/source Voice Assistant 1 Lautsprecher. Current count: 1
2024-04-24 21:03:18.560 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
==> openhab_userdata/logs/events.log <==
2024-04-24 21:03:18.560 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'DialogRunning' changed from ON to OFF
==> openhab_userdata/logs/openhab.log <==
2024-04-24 21:03:18.560 [DEBUG] [eaudio.internal.ConvertedInputStream] - Duration of input stream : 6810
2024-04-24 21:03:18.561 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is a MP3. Trying to reencode it
2024-04-24 21:03:18.561 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is not in the target format. Trying to reencode it
2024-04-24 21:03:18.562 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream connecting for Voice Assistant 1 Lautsprecher
2024-04-24 21:03:18.700 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sinks
2024-04-24 21:03:19.227 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sources
2024-04-24 21:03:19.746 [DEBUG] [pulseaudio.internal.PulseaudioClient] - Pulseaudio server 192.168.24.138: 13 modules and 2 items updated
2024-04-24 21:03:19.746 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Source #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo.monitor, muted: f
alse, state: RUNNING, volume: 100 id: volume
2024-04-24 21:03:19.746 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Sink #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo, muted: false, stat
e: RUNNING, volume: 48 id: volume
2024-04-24 21:03:22.324 [DEBUG] [seaudio.internal.PulseAudioAudioSink] - Some time to let the system play sound : 3050
2024-04-24 21:03:22.324 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Removing client for pulseaudio sink/source Voice Assistant 1 Lautsprecher. Current count: 0
2024-04-24 21:03:22.324 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:03:22.324 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:03:22.404 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Removing client for pulseaudio sink/source Voice Assistant 1 Mikrofon. Current count: 0
2024-04-24 21:03:22.404 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Aborting next disconnect
2024-04-24 21:03:22.404 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Scheduling next disconnect
2024-04-24 21:03:50.256 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sinks
2024-04-24 21:03:50.773 [DEBUG] [pulseaudio.internal.PulseaudioClient] - reading sources
2024-04-24 21:03:51.298 [DEBUG] [pulseaudio.internal.PulseaudioClient] - Pulseaudio server 192.168.24.138: 13 modules and 2 items updated
2024-04-24 21:03:51.298 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Source #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo.monitor, muted: false, state: RUNNING, volume: 100 id: volume
2024-04-24 21:03:51.298 [DEBUG] [o.internal.handler.PulseaudioHandler] - Updating states of Sink #0 (Module: module-alsa-card) alsa_output.usb-Jieli_Technology_UACDemoV1.0_5035811513207F1F-00.analog-stereo, muted: false, state: RUNNING, volume: 48 id: volume
2024-04-24 21:03:52.325 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream disconnecting for Voice Assistant 1 Lautsprecher
2024-04-24 21:03:52.404 [DEBUG] [ernal.PulseaudioSimpleProtocolStream] - Simple TCP Stream disconnecting for Voice Assistant 1 Mikrofon
Possible causes or ideas so far:
- I always get this. Is this normal and acceptable?
2024-04-24 21:03:18.561 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is a MP3. Trying to reencode it
2024-04-24 21:03:18.561 [DEBUG] [eaudio.internal.ConvertedInputStream] - Sound is not in the target format. Trying to reencode it
- On the raspberry I get this error at the first moment openHAB connects to pulseaudio.
journalctl -xfu pulseaudio.service
Apr 24 20:44:55 voiceassistant1 pulseaudio[1430]: E: [pulseaudio] module-alsa-card.c: Failed to find a working profile.
Apr 24 20:44:55 voiceassistant1 pulseaudio[1430]: E: [pulseaudio] module.c: Failed to load module "module-alsa-card" (argument: "device_id="0" name="platform-20902000.hdmi" card_name="alsa_card.platform-20902000.hdmi" namereg
_fail=false tsched=yes fixed_latency_range=no ignore_dB=no deferred_volume=yes use_ucm=yes avoid_resampling=no card_properties="module-udev-detect.discovered=1""): initialization failed.
The message is related to alsa_card.platform-20902000.hdmi. This is a different device which I don’t care about in the setup. Can it cause trouble anyway?
-
openHAB seems to get some input stream. But there is no result of the interpretation.
Duration of input stream : 6810 -
Sometimes I get
IOException while writing to from pulse source pipe: Pipe closedwithout any further information.
Any idea what I am missing? Help is highly appreciated. 