How to configure a ESPHome Voice Assistent Device

holger_hees · August 14, 2026, 11:25am

Recently, I bought a Satellite1.1 Smart Speaker – FutureProofHomes

I updated it to the latest firmware ESPHome firmware. During this step, I was also able to join this device to my wlan.

Anyway, I struggle to configure this in openhab.

I installed the bluetooth binding (depedency of esphome)
I installed the latest ESP Home binding

and this is my configuration

version: 1
things:
  esphome:device:satellite1-4ff88c:
    config:
      deviceId: satellite1-4ff88c
      hostname: 10.10.0.11

Now I got the error message No encryption key configured. Please set 'encryptionKey' in thing configuration or a default one in binding configuration

As recommended, I have defined an encryption key, with the result that I got now this error message

DEVICE_REQUIRES_PLAINTEXT: Device is configured with plaintext api endpoint, but binding is using encryption.. Will reconnect in 11 seconds

So what could be the next step?

holger_hees · August 14, 2026, 11:37am

I think I got it. I have to re flash my device, to use this encryption key.

florian-h05 · August 14, 2026, 2:25pm

Let me know if you get the satellite working with openHAB‘s voice pipeline.

digitaldan · August 14, 2026, 4:54pm

I might need to order one of these for testing… very cool.

EDIT: I just pre-ordered one, sounds like Sept is the next batch.

sihui · August 14, 2026, 6:30pm

Please keep us posted, I am tired about the “big tech” voice machines …

holger_hees · August 15, 2026, 12:26pm

in case other people are facing the same problem. The procedure to connect to the esp home device is the following

Install ESPHome Device Builder
Plugin your ESPHome device
Wait until it is discovered in the ESPHome Device Builder WebUI
Click on on the Notification Banner to jump to the new detected device
Click “Take Control”. This will compile and install a new firmware on the device
After that, you can configure some basic device parameter, where each change will recompile and install the device firmware. Here you can also activate API Encryption, which generates a new API Secret. After that, the firmware is compiled and installed again.

Now, this API secret can be used in your thing config as the “encryptionKey”

So the first success is that the device is “online” I can’t do anything yet, because it has no channels. But at least it is connected.

seime · August 15, 2026, 12:30pm

No firmware channels neither? (advanced channels).

If not please post debug level logs.

Brg

holger_hees · August 15, 2026, 12:50pm

Sorry, I didn’t express myself clearly. The two firmware channels are there, but no others. However, that might be because no “components” are configured for the device firmware yet.

I’ll keep tinkering with it.

holger_hees · August 15, 2026, 1:34pm

maybe I made a mistake. During my first tests, I flashed the newest version and it looks like it was a plain esphome firmware without any sensors configured.

now, I flashed the latest satellite 1 firmware and I have now these channels available.

latestFirmwareVersion "Latest Firmware Version"
firmwareUpdateAvailable "Firmware Update Available"
button_up__vol__ "Button Up (Vol+)" [deviceClass="None", entity_key=-982545930, entity_type="BinarySensor"]
button_down__vol-_ "Button Down (Vol-)" [deviceClass="None", entity_key=538007993, entity_type="BinarySensor"]
button_left__hw_mute_ "Button Left (HW Mute)" [deviceClass="None", entity_key=706120909, entity_type="BinarySensor"]
button_right__action_ "Button Right (Action)" [deviceClass="None", entity_key=1696744061, entity_type="BinarySensor"]
line-out_connected "Line-Out Connected" [deviceClass="None", entity_key=566290180, entity_type="BinarySensor"]
room_presence "Room Presence" [deviceClass="occupancy", entity_key=-1961057140, entity_type="BinarySensor"]
led_ring "LED Ring" [capabilities="ON_OFF,BRIGHTNESS,RGB", entity_field="light", entity_key=1459351809, entity_type="Light"]
wi-fi_signal_strength "Wi-Fi Signal Strength" [deviceClass="signal_strength", entity_key=1502984052, entity_type="Sensor", unit="dBm"]
ambient_light "Ambient Light" [deviceClass="illuminance", entity_key=1353926066, entity_type="Sensor", unit="lx"]
temperature "Temperature" [deviceClass="temperature", entity_key=899752953, entity_type="Sensor", unit="°C"]
humidity "Humidity" [deviceClass="humidity", entity_key=-717201698, entity_type="Sensor", unit="%"]
radar_targets_total "Radar Targets Total" [deviceClass="generic_number", entity_key=962956815, entity_type="Sensor"]
radar_targets_still "Radar Targets Still" [deviceClass="generic_number", entity_key=-801992853, entity_type="Sensor"]
radar_targets_moving "Radar Targets Moving" [deviceClass="generic_number", entity_key=1228983037, entity_type="Sensor"]
wake_sound "Wake sound" [entity_key=-777800699, entity_type="Switch"]
mute_microphones "Mute Microphones" [entity_key=-1253602096, entity_type="Switch"]
radar_tuner_webui "Radar Tuner WebUI" [entity_key=-865201761, entity_type="Switch"]
restart_sat1 "Restart Sat1" [entity_key=2002820956, entity_type="Button"]
xmos_flash_embedded_fw "XMOS Flash Embedded FW" [entity_key=195567956, entity_type="Button"]
radar_factory_reset "Radar Factory Reset" [entity_key=2045189756, entity_type="Button"]
radar_restart "Radar Restart" [entity_key=2135493855, entity_type="Button"]
usb-c_power_supply "USB-C Power Supply" [entity_key=-222615981, entity_type="TextSensor"]
xmos_firmware "XMOS Firmware" [entity_key=-1245134750, entity_type="TextSensor"]
radar_detected "Radar Detected" [entity_key=-1507766708, entity_type="TextSensor"]
radar_firmware "Radar Firmware" [entity_key=-484535783, entity_type="TextSensor"]
radar_target "Radar Target" [entity_key=-996872679, entity_type="TextSensor"]
offset_temperature "Offset Temperature" [deviceClass="temperature", entity_key=1164407067, entity_type="Number"]
offset_humidity "Offset Humidity" [deviceClass="generic_number", entity_key=902168676, entity_type="Number", unit="%"]
speaker_channel_output "Speaker channel output" [entity_key=905088780, entity_type="Select"]
wake_word_sensitivity "Wake word sensitivity" [entity_key=666792156, entity_type="Select"]
action_button_press "Action Button Press Action Button Press" [entity_key=327646398, entity_type="Event"]

and these warnings

Unhandled message of type io.esphome.api.ListEntitiesMediaPlayerResponse. This is lack of support in the binding. Content: 'key: 2232357057
Unhandled message of type io.esphome.api.MediaPlayerStateResponse. This is lack of support in the binding. Content: 'key: 2232357057

seime · August 15, 2026, 6:40pm

Yes audio/media isn’t ready yet, for other readers the pr is here Audio Support by ccutrer · Pull Request #86 · seime/openhab-esphome · GitHub

holger_hees · August 16, 2026, 1:57pm

I got this branch compiled and running,

The binding is detecting the media player correctly.
Seams that the audio source is detected. At least I can choose it under the config “Audio” as a source. But I can’t choose a sink. Looks like it is not detected. If necessary, I can provide logs.

By default, the device does not response to anything. Also the configured wakeup word (in the device - hey jarvis) does not work.

After installing vosk and piper I called

Voice.startDialog(None, "voskstt", "pipertts", "pipertts:kerstin-low-de_DE", "system,rulehli", "esphome:device:home-speaker-1:source:voice_assistant", "enhancedjavasound", "de-DE", "Hey Jarvis", "pOther_SmartSpeaker_VoiceCommand")

after that call, the led’s from the devices are activated. If I say now “Hey jarvis”, the device responsed with a beep and in the device log I can see that the wakeup word was triggered. But nothing more.

if I call

Voice.stopDialog("esphome:device:home-speaker-1:source:voice_assistant")

the dialog is closed in openhab, but nothing changed on the device side. The led’s are still flashing, and the device itself is responding to the wakeup word endless.

– UPDATE –
the red flashing and rotating led ring is a standard error of esphome devices and means that there is no connection to “home assistent”, or in our case “openhab”. In the logs of the device, I can see how openhab connects, but later I see the log that no api client is connected. That means, it is not a network issues. It is more related how the api is used.

[16:57:20.126][D][api:253]: Accept 192.168.0.27
[16:57:20.178][D][api.connection:2526]: openHAB (192.168.0.27): connected

[16:58:52.566][E][voice_assistant:677]: No API client connected
[16:58:52.567][D][voice_assistant:607]: State changed from IDLE to IDLE
[16:58:52.567][D][voice_assistant:614]: Desired state set to IDLE

Other question is: how to skip the wakeup word detection in openhab, as the device itself provides it locally.

holger_hees · August 17, 2026, 11:22am

The reason why no audio sink is registered, is a not supported audio codec.

OpenHAB supports only WAV, MP3 and OGG. The device reports only FLAC as the only supported Audio Codec. It reports 2 mediaplayers. The only difference is the number of channels. Looks like a mono and a stereo flac media player.

In the device documentation I can see that it supports theoretical WAV too, but it is not reported.

For the moment, I enforce WAV as a fallback if no other audio codec is detected. I don’t know if this works at the end. At least the registration process continues. Now I’m able to select the device as a Audio sink too.

But still, the device shows in the log that openhab connects, but the device voice assistent component says “[voice_assistant:677]: No API client connected”

Also the LED ring is showing the red flashing error to indicate a “broken” or failed connection.

CC @ccutrer

– UPDATE —

After finishinh the rustpotter config (downloading the word file), the registration process seams to work. Its strange, as the rustpotter is not needed at all.

Anyway. If I run

Voice.startDialog(None, "voskstt", "pipertts", "pipertts:kerstin-low-de_DE", "system,rulehli", "esphome:device:home-speaker-1:source:voice_assistant", "esphome:device:home-speaker-1:sink:", "de-DE", "Hey Jarvis", "pOther_SmartSpeaker_VoiceCommand")

I can see the following logs on the device

[14:06:25.369][W][micro_wake_word:365]: Wake word detection is already running

at the same moment, the led ring of the device changes to green.

If I say now “Hey Jarvis”, I can see the following logs on the device.

[14:06:42.440][D][micro_wake_word:334]: Detected 'Hey Jarvis' with sliding average probability is 0.92 and max probability is 1.00
[14:06:42.510][D][speaker_source_media_player:285]: State changed to ANNOUNCING
[14:06:42.561][D][ring_buffer:035]: Created ring buffer with size 9600
[14:06:42.582][D][speaker_mixer:369]: Starting
[14:06:42.622][S][media_player]: 'Media Player' >> PLAYING
[14:06:42.622][S][media_player]:   Volume: 45%
[14:06:42.741][D][voice_assistant:607]: State changed from IDLE to START_MICROPHONE
[14:06:42.742][D][voice_assistant:614]: Desired state set to START_PIPELINE
[14:06:42.752][D][voice_assistant:310]: Starting Microphone
[14:06:42.752][D][ring_buffer:035]: Created ring buffer with size 16384
[14:06:42.753][D][voice_assistant:607]: State changed from START_MICROPHONE to STARTING_MICROPHONE
[14:06:42.776][D][voice_assistant:607]: State changed from STARTING_MICROPHONE to START_PIPELINE
[14:06:42.781][D][voice_assistant:334]: Requesting start
[14:06:42.782][D][voice_assistant:607]: State changed from START_PIPELINE to STARTING_PIPELINE
[14:06:42.783][D][ltr_als_ps:455]: Illuminance is sufficient.
[14:06:42.788][D][voice_assistant:629]: Client started, streaming microphone
[14:06:42.788][D][voice_assistant:607]: State changed from STARTING_PIPELINE to STREAMING_MICROPHONE
[14:06:42.788][D][voice_assistant:614]: Desired state set to STREAMING_MICROPHONE

but nothing happens on the openhab side. If I say again “Hey Jarvis”.

I got the following logs

[14:06:55.431][D][micro_wake_word:334]: Detected 'Hey Jarvis' with sliding average probability is 0.87 and max probability is 0.99
[14:06:55.432][D][voice_assistant:741]: Signaling stop
[14:06:55.432][D][voice_assistant:607]: State changed from STREAMING_MICROPHONE to STOP_MICROPHONE
[14:06:55.432][D][voice_assistant:614]: Desired state set to IDLE
[14:06:55.437][D][voice_assistant:607]: State changed from STOP_MICROPHONE to STOPPING_MICROPHONE
[14:06:55.461][D][voice_assistant:607]: State changed from STOPPING_MICROPHONE to IDLE

milo · August 17, 2026, 12:32pm

@holger_hees Thanks for being interested in it as well! I’ve already opened several threads and GitHub issues about it, but unfortunately, every single one was closed without any comment.

I’m really interested in getting this working, but it seems like your name has a bit more luck than mine.

holger_hees · August 17, 2026, 12:48pm

@milo

My name won’t make a difference here.

The key lies in understanding the nature of GitHub issues. They aren’t a call to action; they are a form of bug documentation. If you’re lucky and someone else thinks they can help with the issue, it might eventually get fixed. Sometimes I provide patches by my self afterwards.

I view GitHub issues—and forum threads here—primarily as a way to share my own experiences.

That doesn’t mean I sit around waiting for help afterward. I keep working on the problem, debugging it, and making progress; if I’m lucky, the forum threads or issues I’ve put out there will attract comments or valuable insights that help me move forward.

In short: when others see that you’re actively working on the issue yourself, they’re more likely to offer tips, pointers, and assistance than if you simply describe the bug and wait for someone else to handle it entirely.

I realize that for many people, waiting for help from others is the only option, so I don’t mean this as a criticism. It’s just an explanation of why I tend to receive more “valuable” feedback.

milo · August 17, 2026, 1:17pm

@holger_hees Given the nature of GitHub issues, bugs are already discussed at length there anyway — I’d rather not go through the whole discussion again.

holger_hees · August 17, 2026, 1:50pm

An initial success.

I attempted to make a recording using the following command:

audio record esphome:device:home-speaker-1:source:voice_assistant 10 test.wav

Afterward, the device’s LED ring turned green. The device waited until I said “Hey Jarvis.” Everything I said during the subsequent 10 seconds was received by openHAB and correctly saved to the file. After, the led rings turned rotating red again. Conclusion, the source audio stream therefore arrived successfully.

– UPDATE –

Continue trying to get output working. Remeber, I “faked” the supported codec. As the device reports only flac. But I added WAV as a fallback audio codec. Theoretically the devices supports this.

First challenge is, that playing a sound seams to work by sending a openhab audio url to the device and the device will pull the data. Anyway, the pull request from @ccutrer has hardcoded the way how the base url is generated. Means it is always using the ip with the original openhab port. I changed it temporary to a resolvable address, so the device is now able to fetch the audio data.

Next problem is theese log on the device side.

[16:40:21.433][D][esp-idf:000][md_reader]: D (69223533) micro_decoder.http_client: Connected: status=200 content-type='audio/wav'
[16:40:21.435][D][esp-idf:000][md_reader]: E (69223533) micro_decoder.audio_reader: Could not determine audio file type from URL or Content-Type
[16:40:21.435][D][esp-idf:000][md_reader]: E (69223534) micro_decoder.decoder_source: Reader failed to open URL

I checked the code of the microdecoder typecheck of esphome and it supports WAV and it is checking for the correct mime type “audio/wav”. I hope it has nothing to do with my faked codec support.

ccutrer · August 17, 2026, 4:44pm

Yeah, I didn’t love how the base URL is form. IIRC I pulled that from some code in core. But it’s been awhile, and I’ve had very little time for “fun” projects like this.

I’m impressed it’s worked as well as it has with relatively few fixes to the code. Love to see it progressing. Good work!

holger_hees · August 22, 2026, 4:51pm

Some more positive news: I’ve managed to get the first complete pipeline up and running. That doesn’t mean it works perfectly—just that it runs in principle.

The microphone can essentially be used right away with the esphome pull request mentioned above. (Including my tiny fixed regarding the base url)
There was a bug in Whisper that caused an infinite loop during periods of silence; this is being fixed here.
The biggest issue is audio output. OpenHAB doesn’t natively support the FLAC format, which is crucial for ESPHome. I temporary solved this by using the speach service directly ans ask for FLAC as the output format. Complicating matters further, quite a few things are hard-coded in OpenHAB—for instance, the “UrlAudioStream” class, where MP3 is hard-coded for some reason. Additionally, the device’s FLAC media player wasn’t recognized because the codec isn’t supported. To get it working anyway, I temporarily intercepted FLAC and effectively swapped it for “MP3.” This doesn’t mean MP3 actually works; it just allows me to use the UrlAudioStream.
On the device side, there is still an issue with an annoying background tone, which stems from the device’s electronics. I also managed to crash the device several times, so it will likely take a few more firmware versions before it runs stably.

Below, you can see the Python code I used to initialize and start the pipeline. I chose Python because I like using it for rapid prototyping.

import threading
import java
import json

from openhab import rule, Registry, logger, services
from openhab.triggers import ItemStateChangeTrigger
from openhab.actions import Voice, HTTP

import http.client

import scope


from org.openhab.core.voice import DialogContext
#from org.openhab.core.audio import AudioManager;
#from org.openhab.core.audio import AudioSink;
#from org.openhab.core.audio import AudioSource;
from org.openhab.core.audio import AudioFormat;
from org.openhab.core.audio import URLAudioStream;
from openhab.triggers import GenericCronTrigger, ItemStateUpdateTrigger

class VoiceAssistent(threading.Thread):
    voice_assistant = None

    @staticmethod
    def stopListening():
        if VoiceAssistent.voice_assistant is None:
            return
        VoiceAssistent.voice_assistant.isRunning = False
        VoiceAssistent.voice_assistant = None

    @staticmethod
    def startListening():
        if VoiceAssistent.voice_assistant is not None:
            return
        VoiceAssistent.voice_assistant = VoiceAssistent()
        VoiceAssistent.voice_assistant.start()

    def __init__(self):
      threading.Thread.__init__(self)

      audio = services.getService("org.openhab.core.audio.AudioManager")
      self.source = audio.getSource("esphome:device:home-speaker-1:source:voice_assistant")

      self.sink = audio.getSink("esphome:device:home-speaker-1:sink:")
      self.sink.process(None)

      self.voice = services.getService("org.openhab.core.voice.VoiceManager")
      self.locale = services.getService("org.openhab.core.i18n.LocaleProvider")

      self.isRunning = True

    def run(self):
        while self.isRunning:
            logger.info("Start listening ...")
            message = self.voice.transcribe("esphome:device:home-speaker-1:source:voice_assistant", "whisperstt", self.locale.getLocale())

            if not self.isRunning:
                break

            logger.info("Start interpreting ...")
            answer = Voice.interpret(message, "gemini", "voice-assistent", "get-date-time,item-get-state,item-send-command")

            if not self.isRunning:
                break

            headers = {}
            content = {
                "model": "ufozone/piper-de_DE-jarvis-high",
                "input": answer,
                "voice": "jarvis",
                "response_format": "flac",
                "speed": "1"
            }
            connection = http.client.HTTPConnection("speaches", "8000")
            connection.request("POST", "/v1/audio/speech", json.dumps(content), headers)
            response = connection.getresponse()
            with open('/openhab/conf/html/voice/response.flac', 'wb') as fw:
                fw.write(response.read())

            self.sink.process(URLAudioStream("http://192.168.0.50:10115/static/voice/response.flac"))

        logger.info("Stop listening ...")

@rule(
    triggers = [
        ItemStateUpdateTrigger("pGF_Workroom_SmartSpeaker_Activated")
    ]
)
class MainSmartSpeaker:
    def execute(self, module, input):
        if input['event'].getItemState() == scope.ON:
            VoiceAssistent.startListening()
        else:
            VoiceAssistent.stopListening()

scope.lifecycleTracker.addDisposeHook(VoiceAssistent.stopListening)

The entire process works as follows:

First, I installed Speaches AI as a Docker container. This allows me to access both STT and TTS functionalities via an API.

Calling “self.voice.transcribe” puts the device into listening mode.
As soon as the wake word is spoken, the subsequent audio stream is sent to openHAB.
This audio stream is then sent from the Whisper add-on to my Speaches AI service and returned as text.
This text is then sent to Gemini via “Voice.interpret” and processed (triggering any resulting openHAB actions).
The response text is sent to the TTS service, and the resulting audio stream is saved locally to the hard drive.
A “UrlAudioStream” resource is created from this local file (which is accessible via HTTP) and finally sent to the device’s audio sink.
The process then starts over to continuously listen for the wake word.

To make it short. I can ask “Hey Jarvis, What is the temperature in the livingroom” and I get the correct answer from the same device.

The goal of all this is to demonstrate that it is fundamentally feasible and to show the direction in which the requirements for a future voice assistant interface are heading.

lsiepel · August 22, 2026, 8:04pm

I know the goal for now is just to put everything together and get a proof of concept, but i’m already very interested in the first insights regarding responsiveness.
The delay between the last spoken word until the noticable action might be a good measure, but for now it is more the impression you have and what your aim is.

holger_hees · August 23, 2026, 11:52am

In total, the initial test setup takes 7 seconds.

Intitial trigger “Hey Jarvis”
Recording of “Wie warm ist es im Wohnzimmer” => 3 seconds
Transkription (Speaches AI - podman/docker container) => 1 second
Interpreting (Gemini) => 2,5 seconds
Answer generation (Speaches AI - podman/docker container) => 0,15 seconds
Answering => …

Currently it is accelerated by a rtx 3050 with 4GB.

Challenges are the quality of the models. e.g. try to find a german model which can read “21 °C” correctly. Only commercial models are able to read “Celsius” so far. At least I have not found a free one.

Also sometimes the answer sentence is stopping somewhere in the middle.

However, these measurements are very imprecise, as they are based on various sources.

I also tried using a stopwatch to measure the time between the last spoken word and the start of the response. It is consistently around 4 seconds. However, there is still room for improvement. Apart from the hardware, the choice of models also has an impact.

— UPDATE —

I compared it with my current system, which should be replaced. The current system are 8 alexa devices with a custom skill. This custom skill is activated by “Computer, Ask Marvin…”. The text after, is captured and send to openhab via REST API. The rest is similar to my custom voice assistent. Text is interpreted by gemini and the result text is send to alexa back as an answer.

I just messured the time between last spoken word and first word from answer. This includes, transcription (stt), interpretation (gemini) and answer generation (tts). The times are usually between 4 and 5 seconds. So, it is certainly competitive.

The times itself are not the problem. The Challenge is the visualization.

My ideal vision looks like this:

Device is listening for wakeup word. No or just a tiny visualization
Wakeup Word is triggered. LED should indicate that we should talk now.
After silence is detected and transcription is started. I would visualize it with a flashing led.
After transcription is done and maybe the interpretation start, we could visualize it with a different flashing style.
While answering, we don’t need a visualization.

It would be perfect if each step will trigger some kind of openhab event, which can be captured by a rule.