Alexa Voice Command Room Awareness via OH

Tutorials & Examples
,
1

Currently, the Alexa room awareness feature (aka Alexa-enabled groups) is derived from the majority of the current display categories. So, until Amazon adds new categories, there is no way to address devices not part of that list (e.g. fans) via that feature.

Thanks to the Amazon Echo Control binding lastVoiceCommand channel and a proxy Alexa-linked activity item, we can mimic that feature from the OH side for unsupported categories.

The definitions below set the room awareness portion. This should handle any permutations of the Alexa activity capability commands. To add a new category, you just need to add an Alexa-enabled activity item, the category name lower-cased in the category regex ".*\\b(fan|<newCategory>)\\b.*", and the relevant devices to the mapping file using format <CATEGORY>-<Location>=<DeviceName>. If a specific room doesn’t have a device for a given category, specify the mapping with an empty device name. That way a voice error message will be sent back.

In order to prevent misfiring on specific named device voice commands (turn off bedroom fan from another room vs turn on the fan in the current room), the associated Alexa-linked activity item state is checked to determine if it was triggered or not. That way, only relevant voice commands that occurred within a second the activity item was triggered will generate a command, otherwise, they will be ignored.

Keep in mind that the fan control definition below is just indicative here and can be more complex if necessary but wanted to simplify that portion to focus on the room awareness logic.

items

Group gAlexaVoiceCommand "Alexa Voice Command"

String BedroomEchoDotLastVoiceCommand "Last Voice Command" (gAlexaVoiceCommand) {channel="amazonechocontrol:echo:account:echodot_bedroom:lastVoiceCommand"}
String BedroomEchoDotTextToSpeech "Text to Speech" {channel="amazonechocontrol:echo:account:echodot_bedroom:textToSpeech"}

Switch BedroomFanPower "Bedroom Fan" {alexa="Switch"}

Switch AlexaActivityFan "Fan" {alexa="Activity", expire="1s"}

rules

import org.eclipse.smarthome.model.script.ScriptServiceUtil

rule "Alexa Voice Command Room Awareness"
when
  Member of gAlexaVoiceCommand received update
then
  // Exit if voice command is "alexa"
  if (triggeringItem.state == "alexa") return;
  // Determine room location based on triggering item name
  val location = triggeringItem.name.split("Echo").get(0)
  // Determine device category and voice command based on triggering item state regex matching
  val category = transform("REGEX", ".*\\b(fan)\\b.*", triggeringItem.state.toString)
  val command = transform("REGEX", ".*\\b(activate|deactivate|disable|enable|on|off|start|stop)\\b.*", triggeringItem.state.toString)
  // Exit if category or command is null
  if (category === null || command === null) return;
  // Determine activity item state based on device category
  val activityState = ScriptServiceUtil.getItemRegistry.getItem("AlexaActivity" + category.capitalize).state.toString
  // Exit if activity item not triggered (state not defined)
  if (activityState == "NULL" || activityState == "UNDEF") return;
  // Determine device name and state based on category-location mapping and voice command
  val deviceName = transform("MAP", "alexa-device.map", category.toUpperCase + "-" + location)
  val deviceState = if (command == "activate" || command == "enable" || command == "on" || command == "start") "ON" else "OFF"
  // Send command to device if name defined, otherwise generate text to speech error message
  if (deviceName != "")
    sendCommand(deviceName, deviceState)
  else
    sendCommand(triggeringItem.name.replace("LastVoiceCommand", "TextToSpeech"), "There is no " + category + " to control in this room")
end

alexa-device.map

FAN-Bedroom=BedroomFanPower
8 Likes
3

Below is a simplified version of the rule using the relevant Alexa-linked activity item state as the device state, removing the need of parsing the voice command for all the different permutations especially for non-English users that had to translate the regular expression.

import org.eclipse.smarthome.model.script.ScriptServiceUtil

rule "Alexa Voice Command Room Awareness"
when
  Member of gAlexaVoiceCommand received update
then
  // Exit if voice command is "alexa"
  if (triggeringItem.state == "alexa") return;
  // Determine room location based on triggering item name
  val location = triggeringItem.name.split("Echo").get(0)
  // Determine device category and voice command based on triggering item state regex matching
  val category = transform("REGEX", ".*\\b(fan)\\b.*", triggeringItem.state.toString)
  // Exit if category is null
  if (category === null) return;
  // Determine device name based on category-location mapping
  val deviceName = transform("MAP", "alexa.map", category.toUpperCase + "-" + location)
  // Determine device state based activity item category related state
  val deviceState = ScriptServiceUtil.getItemRegistry.getItem("AlexaActivity" + category.capitalize).state.toString
  // Send command to device if name and state defined, otherwise generate text to speech error message
  if (deviceState == "ON" || deviceState == "OFF")
    if (deviceName != "")
      sendCommand(deviceName, deviceState)
    else
      sendCommand(triggeringItem.name.replace("LastVoiceCommand", "TextToSpeech"),
        "There is no " + category + " to control in this room")
end
1 Like
4

I see that fans are now in the display category list. Does this mean that I should be able to use room awareness with fans? I tried, but it does not work for me. I have fans in different groups in the Alexa app, but Alexa asks me which fan I want if I ask to turn on the fan. Are these items correct?

Or am I misinterpreting what you are saying above and this isn’t yet possible with fans?

Switch cabinFan "Cabin Fan" <fan> (gPubItems) {alexa="PowerController.powerState" [category="FAN"]}
Switch Bathroom_Fan "Bathroom Fan" <fan> {alexa="PowerController.powerState" [category="FAN"]}
5

I think my original statement may not have been very representative of the current experience, especially after Amazon added a bunch of display categories at the end of last year. Therefore, it doesn’t seem there is actually a correlation between the two as to my knowledge, none of the latest categories added are supported by the room awareness feature. From my experience, I believe only LIGHT & TV categories are currently supported.

1 Like
6

Based on a question requested in this post, it got me going on adding duration support to this rule. It certainly increases the complexity and I am unsure how it can be easily translated to other languages. I also added multi-devices mapping support.

items

Switch BedroomLight1 "Bedroom Light 1" {alexa="Light"}
Switch BedroomLight2 "Bedroom Light 2" {alexa="Light"}

Switch AlexaActivityLights "Lights" {alexa="Activity", expire="1s"}

rule

rule "Alexa Voice Command Room Awareness Updates"
when
  Member of gAlexaVoiceCommand received update
then
  // Exit if voice command is "alexa"
  if (triggeringItem.state == "alexa") return;
  // Determine room location based on triggering item name
  val location = triggeringItem.name.split("Echo").get(0)
  // Determine device category based on triggering item state regex matching
  val category = transform("REGEX", ".*\\b(fan|lights)\\b.*", triggeringItem.state.toString)
  // Exit if category is null
  if (category === null) return;
  // Determine voice command duration based on triggering item state regex matching
  val interval = transform("REGEX", ".*\\b((for|in)\\s+[a-z ]+\\s+(second|minute|hour|day)[s]?)\\b.*", triggeringItem.state.toString)
  val duration = if (interval === null) 0 else Float::parseFloat(transform("JS", "wordToDuration.js", interval)).intValue
  // Determine device names based on category-location mapping
  val deviceNames = transform("MAP", "alexa-device.map", category.toUpperCase + "-" + location)
  // Determine device state based activity item category related state
  val deviceState = ScriptServiceUtil.getItemRegistry.getItem("AlexaActivity" + category.capitalize).state.toString
  // Send command to device if name and state defined, otherwise generate text to speech error message
  if (deviceState == "ON" || deviceState == "OFF")
    if (deviceNames != "")
      deviceNames.split(",").forEach[ deviceName |
        val newState = if (duration <= 0) deviceState else null
        val previousState = if (deviceState == "ON") "OFF" else "ON"
        val timerState = if (duration > 0) deviceState else if (duration < 0) previousState else null
        val timerDuration = Math::abs(duration)

        if (newState !== null)
          sendCommand(deviceName, newState)
        if (timerState !== null)
          createTimer(now.plusSeconds(timerDuration)) [|
            sendCommand(deviceName, timerState)
          ]
      ]
    else
      sendCommand(triggeringItem.name.replace("LastVoiceCommand", "TextToSpeech"),
        "There is no " + category + " to control in this room")
end

alexa-device.map

LIGHTS-Bedroom=BedroomLight1,BedroomLight2

wordToDuration.js

(function(text) {
  var unit = {
    zero: 0, one: 1, two: 2, three: 3, four: 4, five: 5, six: 6, seven: 7, eight: 8, nine: 9, ten: 10, eleven: 11,
    twelve: 12, thirteen: 13, fourteen: 14, fifteen: 15, sixteen: 16, seventeen: 17, eighteen: 18, nineteen: 19
  };
  var ten = {
    twenty: 20, thirty: 30, forty: 40, fifty: 50, sixty: 60, seventy: 70, eighty: 80, ninety: 90
  };
  var magnitude = {
    hundred: 100, thousand: 1000, million: 1000000, billion: 1000000000
  };
  var prefix = {
    in: 1, for: -1
  };
  var time = {
    second: 1, minute: 60, hour: 3600, day: 86400
  };
  var joiners = ["and", "for", "in"];

  var words = text.toLowerCase().split(" ").map(function(word) { return word.trim().replace(/s$/, ""); });
  var direction = prefix[words[0]] || prefix.in;
  var scale = time[words[words.length - 1]] || time.second;
  var number = 0;
  var index = 0;
  while (index < words.length) {
    var value = 0;
    if (joiners.indexOf(words[index]) != -1) {
      index++;
      continue;
    }
    if (words[index] in ten) {
      value += ten[words[index++]];
    }
    if (words[index] in unit) {
      if (unit[words[index]] <= 9) {
        value += unit[words[index++]];
      } else {
        value = unit[words[index++]];
      }
    }
    if (words[index] in magnitude) {
      number += value * magnitude[words[index++]];
    } else {
      number += value;
      break;
    }
  }
  return number * direction * scale;
})(input)

utterances

Alexa, turn on the lights for one hour
Alexa, turn off the lights in five minutes

2 Likes
7

I like your post and will give this a try. Thanks for sharing it.

I am curious though how Alexa responds to “Alexa, turn on bedroom switch 1 for an hour”. I see how your code handles this but doesn’t the Echo device respond on it’s own first?

In testing I’ve done, Alexa tries to respond herself until openhab text arrives to override. The trick is finding a way to delay Alexa long enough for openhab to be the only response. Have you had success accomplishing this?

8

My use-case is related to room awareness feature. You won’t be able to use named device with my rule. As I mentioned before, the idea is that you setup a dummy Alexa activity item which will receive the command when calling it exactly by name such “turn on the lights” and therefore prevent the actual command from being sent to the actual device allowing the OH rule to handle it.

If you are looking to use named device, I guess you can use proxy items instead of being directly exposed to Alexa and control the actual device via rules similar to the one I mentioned.

As far as for response, you should just get an acknowledgement if you made the proper request. Keep in mind, this is a hack and there is not much control over delay and latency.

9

Thank you for pushing this idea into my head; I have been tracking presence based on HUE Motion sensors and now I’ve added talking to Alexa in a room as another data point for current motion identification in rooms.

Best, Jay

1 Like
10

Doing some testing and ran into a few errors. Appreciate any advice.
I believe the code for timers is missing needed group names.
Ran into error: The name ‘ScriptServiceUtil’ cannot be resolved to an item or type

11

You need to add the below line at the top of your rule file. It is actually part of the initial rule I posted above.

import org.eclipse.smarthome.model.script.ScriptServiceUtil
12

You will need to include this import at the top of the rule file


import org.eclipse.smarthome.model.script.ScriptServiceUtil

More detail here


1 Like
13

I was just trying to make use of this rule/script but I noticed that the LastVoiceCommand is not working for me on OH 3.2. Is it an issue from my openhab or is it releated to something else ?

14

That’s an issue with the Amazon Echo Control binding. I would suggest searching for that issue in the forum.

1 Like
15

Found it in the meantime, now using the latest binding jar file

16

I have a Dummy Switch I use so Alexa can tell me the current Gas Price (Question about Amazon Echo Control Speak Feature - #8 by J-N-K). Currently my rule is setup so if the Dummy Switch is triggered only on my living room alexa the answer will be said.

What I am to achieve now is to detect which Alexa Device has received the question, that triggered the dummy switch and have the same Device answering speaking the answer/current gas price.

I have 3 different Alexa devices, named this:
Livingroom
Bedroom
Kitchen

I am now having difficulties to understand how the scripts and rules exactly work to determine the alexa device that was triggered. May I ask you to elaborate how that would work. I seem to be to stupid to understand it how it is wirtten right now.

17

You could trigger your rule by a change to the lastVoiceCommand channel of your Echo and check if “gas price” is part of the command. If so, answer with the command, if not, do nothing.

Best Regards,

Jan

18

Here’s how I know which Alexa was spoken to OR last spoke.

This is pieces of my logic below. All my items for the Alexa are standardized for each room.

i.e. Echo_SourceRoom_ItemDesc

Member of AlexaCmds changed

var String sourceRoom = triggeringItem.name.split("").get(1)
var String TheRoom_TTS = "Echo"+sourceRoom+"_TTS"

logInfo(“ECHO”,"TTS Device is " + TheRoom_TTS + “.”)
logInfo(“ECHO”,"Triggering Command is " + triggeringItem.state + “.”)

TheRoom_TTS.sendCommand(HouseAlarmMsg)

Best, Jay

3 Likes
19

Perfect. Thank you very much for this.
I solved it in the same way now!

20

Is it possible with the rule and scripts you @jeshab shared in your posts that alexa understands something like this correctly aswell ?

Alexa, turn on the lights for 10 minutes and 40 seconds ?

21

My rule fragment lines above is tied to a very long rule watching for last command and re-acting to it based on what is said. You’ll also have to create a rule in the Alexa app, so Alexa doesn’t try to react to what you said. I usually create a routine looking for those words and just have Alexa respond back either “Sure” or “Let me find out”.

Here’s just one example of it.

			if (systemStarted.state != ON && triggeringItem.state == 'loft temperature') { 
		
				logInfo("ECHO","Alexa Loft temperature Status via Voice has been triggered using " + triggeringItem.name)
					Thread::sleep(500)
				
				var String lofttemperature = '<speak><prosody rate="fast">Loft temperature, is unknown to me at this time.</prosody></speak>'
				if (systemStarted.state != ON && gLoftTemp.state !== null && gLoftTemp.state != NULL) {
					
					lofttemperature = '<speak>Loft temperature is, ' + gLoftTemp.state + ' degrees.</speak>'
				}
				
				TheRoom_TTS.sendCommand(lofttemperature)	
				return;
			}

Best, Jay