How to test Voice Pipeline

holger_hees · August 18, 2026, 11:59am

Currently I work on another topic. I try to activate my Voice Assistent somehow. For that it is necessary to have at least a working voice pipeline.

Now I realized that also the browser based voice pipeline does not work. I don’t get it triggered.

First I configured Voice like below.

Keyword Spotter: Rustpotter
Speech to Text: Vosk (confirmed that the folder name of the model is “userdata/vosk/model”)
Text To Speech: Piper
Voice: Piper … Kerstin
Interpreter: Rule Based (I chooses this one, because I want to see what is coming out from stt service first)
Magic Word: Hey Marvin

I set the following log level:

log:set DEBUG org.openhab.core.voice
log:set DEBUG org.openhab.core.io.voice
log:set DEBUG org.openhab.voice.rustpotterks
log:set DEBUG org.openhab.voice.voskstt

under them menu point “Audio” I choosed my web audio devices

I I click in the Web Main UI on the voice button, I see the following logs on my server

13:53:34.826 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : dm AudioSource tracking 170 MultipleDynamic already active, binding {org.openhab.core.audio.AudioSource}={service.id=3353, service.bundleid=197, service.scope=singleton}
13:53:34.827 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : invoking bind: addAudioSource: parameters [org.openhab.core.io.websocket.audio.internal.PCMWebSocketAudioSource]
13:53:34.828 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : invoked bind: addAudioSource
13:53:34.829 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : dm AudioSink tracking 171 MultipleDynamic already active, binding {org.openhab.core.audio.AudioSink}={service.id=3354, service.bundleid=197, service.scope=singleton}
13:53:34.830 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : invoking bind: addAudioSink: parameters [org.openhab.core.io.websocket.audio.internal.PCMWebSocketAudioSink]
13:53:34.830 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - bundle org.openhab.core.voice:5.2.1 (228)[org.openhab.core.voice.internal.VoiceManagerImpl(1366)] : invoked bind: addAudioSink
13:53:34.831 [DEBUG] [.conversation.ConversationManagerImpl] - Creating new unpersisted conversation
13:53:34.832 [DEBUG] [.core.voice.internal.VoiceManagerImpl] - Starting a new dialog for source PCM Audio WebSocket (ui-81-2) (pcm::ui-81-2::source

and in my browser I can see

Voice: Starting ws connection... index-BJBtX3R_.js:15742:72
Voice: Created audio context with sample rate 48000 index-BJBtX3R_.js:15742:72
Voice: Audio context state 'running' index-BJBtX3R_.js:15742:72
Voice: Resuming audio context, click to continue index-BJBtX3R_.js:15742:72
Voice: Click the widget to trigger the dialog index-BJBtX3R_.js:15742:72
Voice: Connected index-BJBtX3R_.js:15742:72

As I can see, the dialog is initialized and registered. Now I press the button again to start the conversation.

I see the following logs on my server

13:55:43.684 [DEBUG] [b.core.voice.internal.DialogProcessor] - DTTriggeredEvent event received
13:55:43.684 [DEBUG] [b.core.voice.internal.DialogProcessor] - Acquiring dialog group 'default'

and the following logs in my browser

Voice: Starting microphone audio streaming index-BJBtX3R_.js:15742:72

in this moment I can see in the browser network tab that a lot of traffic is sendet to openhab.

But nothing happens. I don’t see any logs on server side. Not for voice, not for rustpotter and not for vosk. Looks like he is sending audio stream forever.

I tested it in chrome and firefox. Each time the same result.

Has anyone an idea?

florian-h05 · August 18, 2026, 5:10pm

First of all, for the Main UI voice feature, you don’t need a server wide keyword spotter. Configuring one shouldn’t harm though.

This also isn’t required for Main UI‘s voice feature. You don’t start the dialog server side through rules, it is started from Main UI.

The AudioStream should be sent into your STT service, which needs to do a VAD and „end“ receiving audio once it doesn’t detect voice activity anymore. Once the STT service finished the transcription, you should see this as core log IIRC.
I’d recommend giving Whisper a try, my first idea would be that there is an issue with Vosk. I’ve successfully used Whisper + Gemini + Piper several times already.

holger_hees · August 20, 2026, 10:59am

with whisper it works. But extremely slow if you host the model local.

for that reason, I decided to use “searches ai” in a container. It provides a flexible way to deal with stt and tts models. But the biggest benefit was to have a easy way for GPU usage and it provides a openAI API.

The whisperstt addon works out of the box with this service, as it is openai API compatible.

The openaitts service works with the API too, but has a lot of hardcoded stuff like voices, language etc…

After I fixed all of this locally, I’m able to run the full voice pipeline using whisperstt and openaitts using my local running “searches AI” service fully accelerated.

my STT service went down from ~20s to 1.2 s… it is acceptable for prototyping.

now I can continue working on my voice assistant integration.

and before anyone asks, I will provide my changes later as patches. Currently I’m in a very early stage where I’m not sure if this is the right way to fix and improve things.

everything is just for playing, exploring and learning.