SEPIA Open Assistant - Privacy respecting, self-hosted voice-control

Hi openHAB community :slight_smile:

I just wanted to quickly share my open-source voice assistant project with you since it’s using openHAB to control my smart home devices ([Edit] check out the new blog article).

SEPIA is a personal, digital assistant that consists of a Java server (just like openHAB) and a HTML5 based cross-platform client that runs on any browser, Android and iOS. The server has HTTP REST endpoints for natural-language-understanding, dialog management and intent-to-action that can be used stand-alone and offers a WebSocket connection for chat messages and data transfer. You can read more about it on the SEPIA Framework homepage.

In contrast to Amazon Alexa or Google Assistant, SEPIA is self-hosted by YOU :wink: meaning that every bit of data collected by your assistant is 100% under your control and stays private! If you have Java installed already (likely since you are using openHAB) then the installation of the server can be done in 10min or less.
After the installation you can check the “next steps” section to see how you can connect to openHAB.

If you you’re missing some features (openHAB integration is still pretty basic) there is an SDK to build your own smart-services and every bit of code is open-source so feel free to contribute :slight_smile:

Hope you like it! Any feedback is welcome ^_^.

For pictures, videos and latest news you can follow SEPIA on Twitter as well @sepia_fw.

11 Likes

Hi,
I am currently running OH, MQTT , Node Red etc as docker containers in Ubuntu 16.
Would you happen to have Sepia as a docker installation ? If so, can you give the guide for it?

much thanks!

Hi J Wee, thanks for your interest :slight_smile:
I’ve just uploaded a docker image to Docker HUB: https://hub.docker.com/r/sepia/home
It is not really tested and probably needs some more documentation ^^, but maybe it can help you to get started already!

1 Like

Thanks! Will definately give it a go!

Got an error response with “docker pull sepia/home” about manifest not found

Hey J,

sorry for the late reply, I must have missed the part about the error somehow :-/
Can you specify when the error happens exactly? I’ve tried it with Debian and Docker for Windows and both are running fine. I’m also not aware of any manifest file :thinking:

Hi openHAB community,

it’s been a while since the last post and although SEPIA is getting regular updates and I’m using openHAB + SEPIA every day to control my Hue lights there haven’t been any many major changes to the openHAB connector … until now :slight_smile: .

Recently I’ve reworked some parts of the smart home interface to simplify the integration of additional platforms (FHEM is in test-phase) and took the chance to buff the smart home service together with the openHAB code. SEPIA supports around 12 device and room types now (previously it was only lights and heaters) to enable sentences like “Whats the status of sensor 1 in the hallway?” or “set window blinds in the office to 60%”.

A preview of the next SEPIA version v2.4.0 can be downloaded HERE.

It would be great if someone with a more complex openHAB setup could experiment a bit with this version before I make it official :smiley: .

To learn more about how to set up SEPIA and openHAB check this GUIDE. It is still for the current version v2.3.1 but the procedure is mostly unchanged (from user side).

Note: “Set the heater in the bathroom to 25 degrees celsius” is technically possible but might fail since SEPIA is currently not checking if the device accepts degrees or if it uses Celsius or Fahrenheit.

Hope you like it and would love to see some comments :innocent:

Florian

5 Likes

How about MQTT integration? I am using Snips with NodeRed to control my smart home. This includes Lights, Doors, Sensors, Audio, Notifications, Cameras etc. So it will be good have Sepia and MQTT for easy transition from Snips.

Sorry there is no MQTT support yet but I’m currently evaluating it. Do you have some more info about the NodeRED MQTT interface? :slight_smile:

Mqtt is a general protocol. No need to know anything about node red. You need a broker, mosquitto for example, and simply send messages to it. Any software connectee to the broker can subscribe to messages.

I tested snips some time ago and it used mqtt for connecting the satellites to the server (sent the recorded audio to the server) and connecting any home automation to snips.

It had a topic for (from memory)

  • the intend what to do
  • with which object something should be done

Hi Joachim,

I knew a bit about the MQTT protocol but I think I get the idea now how Snips and NodeRED are using it :+1: Is there any convention yet for devices who listen to broadcasted intents/voice-actions?

Hi Florian,

i just looked up the documentation of snips, because my memory is fuzzy. :wink:

  • hermes/hotword/<hotword_id>detected : when a wakeword is detected, this sent, and Snips will listening to your voice command. You can for instance use this message to play a little tone, or start a led animation, indicating the start of a listening session

  • hermes/asr/textCaptured : when listening, Snips will transcribe your query into text, and after a small period of silence, it will stop listening, and send this message. You may use this to play another tone, or stop a led animation, indicating end of listening

  • hermes/intent/<intent_name> : after text has been captured, the NLU module will process the text and transform it into an intent, which is the final representation of your query that you can use as an actionable item. This is where you want to put your intent handler code

I don’t know about a specific convention, i just took a short look at snips some months ago. Last week, when i wanted to start with that voice stuff again, i discovered the Sonos situation.

1 Like

Ok so I’m starting to understand the context, idea and architecture of the related systems that use MQTT and I think it should be fairly easy to add MQTT message broadcasting to certain services of SEPIA.
Maybe its an idea to add MQTT as an additional option to the (SEPIA) HUB selection menu. This would work a bit different than openHAB and FHEM in the sense that it simply translates smart home commands to MQTT messages (compared to the two-way communication with e.g. openHAB’s REST API) but it could be used inside Tools like NodeRED directly :slight_smile: .

@job: Do you know if the same thing exists for smart home control, e.g. something like: /lights/livingroom/set/70/percent . I’m assuming openHAB is using a specific format already?

[EDIT] It seems THIS page could be a start

Well in my opinion everything should be pretty generic:

You would need something like:

  • a dictionary of locations
  • a dictionary of objects (maybe referencing locations, maybe as description & id)
  • a dictionary of actions and maybe valid parameters

If the voice assistant is able to recognize these elements as something like this:

  • sepia/action = "set to"
  • sepia/actionparameter = "70"
  • sepia/location = "livingroom"
  • sepia/object = "light"
  • sepia/capturedcommand = "set light in the living room to 70 percent"

every Home Automation could work out the details to perform the action.

Regarding openHAB, all these dictionaries could be read from the Rest-API, the actions are defined via the object-tags, the location and the object names (including ids) are well defined.

1 Like

This is what SEPIA already does and it translates it to openHABs REST API calls (the service reads the openHABs device list, finds the correct device and then constructs the REST API call) :slight_smile:
I’ve all the intent data in a well structured JSON object, but as far as I understand MQTT protocol this should result in one specific URI (aka channel path) or? I mean a combination of the above mentioned 5 (action, actionparameter, location, …) in one message.

That’s perfectly fine. Just output your structure to mqtt, every sytem is able to parse those messages.

Just output your structure to mqtt

oh you mean just add the JSON object as MQTT message payload?

Yes.

1 Like

Thanks to some feedback to the new test version above I’ve found a bug that might lead to problems in certain setups of openHAB preventing control via the chat interface (Control HUB works though). Here is a new version that should fix this: DOWNLOAD. (I’ve also updated the link above). :v:

1 Like

Hey,
can i also setup Openhab with nginx reverse proxy? Because i dont know where to put in the Username and password.

Hi David,

do you mean an openHAB server protected by Nginx via Basic-Auth. or similar? The official version doesn’t have support for this yet but the test build one post above has :slight_smile:.
If you use Basic Auth. for example you can add:

smarthome_hub_auth_type=Basic
smarthome_hub_auth_data=dXNlcjp0ZXN0MTIzNDU=

with ‘dXNlcjp0ZXN0MTIzNDU=’ being the base64 encoded ‘username:password’, in this case ‘user:test12345’.

I hope that answers your question?