Voice control rules

Setup, Configuration and Use Scripts & Rules
1

Hello friends

I am testing the rules of openhab together voice recognition
At this moment everything works without problems, but I need to understant if have any way to perform the command in more “smarter” way

For example: If I say
Turn ON the Light of Kitchen

The idea is recognize de command (Turn ON), the thing (Light) and the room (Kitchen)
Then the script just concatenate the string and execute the command

  1. In the rule below I just detect command by command
    Works, but for me this can be improved

Someone can help me ?
Sorry, the strings of commands below are in Portuguese

  1. Another doubt, is possible execute the code more faster ?
    Sometimes if I say something, the system have one delay of 15 seconds or more to execute

My rules
import org.openhab.model.script.actions.*
import org.openhab.core.library.types.*
import java.util.*

//Controle de Voz
rule “Voice control”
when
Item VoiceCommand received command
then
var String command = VoiceCommand.state.toString.toLowerCase
logInfo(“Voice.Rec”,"VoiceCommand received "+command)

//Corredor
if (command.contains(“ligar luz corredor”) || command.contains(“acender luz corredor”))
{
Light_Corredor.sendCommand(ON)
}

else if (command.contains(“desligar luz corredor”) || command.contains(“apagar luz corredor”))
{
Light_Corredor.sendCommand(OFF)
}

//Escada
else if (command.contains(“ligar luz escada”) || command.contains(“acender luz escada”))
{
Light_Escada.sendCommand(ON)
}

else if (command.contains(“desligar luz escada”) || command.contains(“apagar luz escada”))
{
Light_Escada.sendCommand(OFF)
}

//Sala
else if (command.contains(“ligar luz sala”) || command.contains(“acender luz sala”))
{
Light_Sala.sendCommand(ON)
}

else if (command.contains(“desligar luz sala”) || command.contains(“apagar luz sala”))
{
Light_Sala.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador sala”))
{
Ventilador_Sala.sendCommand(ON)
}

else if (command.contains(“desligar ventilador sala”))
{
Ventilador_Sala.sendCommand(OFF)
}

//Sanca
else if (command.contains(“ligar luz sanca”) || command.contains(“acender luz sanca”))
{
Light_Sanca.sendCommand(ON)
}

else if (command.contains(“desligar luz sanca”) || command.contains(“apagar luz sanca”))
{
Light_Sanca.sendCommand(OFF)
}

//Cozinha
else if (command.contains(“ligar luz cozinha”) || command.contains(“acender luz cozinha”))
{
Light_Cozinha.sendCommand(ON)
}

else if (command.contains(“desligar luz cozinha”) || command.contains(“apagar luz cozinha”))
{
Light_Cozinha.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador cozinha”))
{
Ventilador_Cozinha.sendCommand(ON)
}

else if (command.contains(“desligar ventilador cozinha”))
{
Ventilador_Cozinha.sendCommand(OFF)
}

//Hack
else if (command.contains(“ligar luz hack”) || command.contains(“acender luz hack”))
{
Light_Hack.sendCommand(ON)
}

else if (command.contains(“desligar luz hack”) || command.contains(“apagar luz hack”))
{
Light_Hack.sendCommand(OFF)
}

//Escritório
else if (command.contains(“ligar luz escritório”) || command.contains(“acender luz escritório”))
{
Light_Escritorio.sendCommand(ON)
}

else if (command.contains(“desligar luz escritório”) || command.contains(“apagar luz escritório”))
{
Light_Escritorio.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador escritório”))
{
Ventilador_Escritorio.sendCommand(ON)
}

else if (command.contains(“desligar ventilador escritório”))
{
Ventilador_Escritorio.sendCommand(OFF)
}

//Banheiro
else if (command.contains(“ligar luz banheiro”) || command.contains(“acender luz banheiro”))
{
Light_GF_Banheiro.sendCommand(ON)
}

else if (command.contains(“desligar luz banheiro”) || command.contains(“apagar luz banheiro”))
{
Light_GF_Banheiro.sendCommand(OFF)
}

//Filho
else if (command.contains(“ligar luz filho”) || command.contains(“acender luz filho”))
{
Light_Filho.sendCommand(ON)
}

else if (command.contains(“desligar luz filho”) || command.contains(“apagar luz filho”))
{
Light_Filho.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador filho”))
{
Ventilador_Filho.sendCommand(ON)
}

else if (command.contains(“desligar ventilador filho”))
{
Ventilador_Filho.sendCommand(OFF)
}

//Filha
else if (command.contains(“ligar luz filha”) || command.contains(“acender luz filha”))
{
Light_Filha.sendCommand(ON)
}

else if (command.contains(“desligar luz filha”) || command.contains(“apagar luz filha”))
{
Light_Filha.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador filha”))
{
Ventilador_Filha.sendCommand(ON)
}

else if (command.contains(“desligar ventilador filha”))
{
Ventilador_Filha.sendCommand(OFF)
}

//Suíte
else if (command.contains(“ligar luz casal”) || command.contains(“acender luz casal”))
{
Light_Suite.sendCommand(ON)
}

else if (command.contains(“desligar luz casal”) || command.contains(“apagar luz casal”))
{
Light_Suite.sendCommand(OFF)
}

else if (command.contains(“ligar ventilador casal”))
{
Ventilador_Suite.sendCommand(ON)
}

else if (command.contains(“desligar ventilador casal”))
{
Ventilador_Suite.sendCommand(OFF)
}

//Lavanderia
else if (command.contains(“ligar luz lavanderia”) || command.contains(“acender luz lavanderia”))
{
Light_Lavanderia.sendCommand(ON)
}

else if (command.contains(“desligar luz lavanderia”) || command.contains(“apagar luz lavanderia”))
{
Light_Lavanderia.sendCommand(OFF)
}

//Garagem
else if (command.contains(“ligar luz garagem”) || command.contains(“acender luz garagem”))
{
Light_Garagem.sendCommand(ON)
}

else if (command.contains(“desligar luz garagem”) || command.contains(“apagar luz garagem”))
{
Light_Garagem.sendCommand(OFF)
}

//Jardim
else if (command.contains(“ligar luz jardim”) || command.contains(“acender luz jardim”))
{
Light_Jardim.sendCommand(ON)
}

if (command.contains(“desligar luz jardim”) || command.contains(“apagar luz jardim”))
{
Light_Jardim.sendCommand(OFF)
}

//Apagar todas as Luzes
else if (command.contains(“desligar todas as luzes”) || command.contains(“apagar todas as luzes”))
{
Lights.sendCommand(OFF)
}

//Desligar todos os ventiladores
else if (command.contains(“desligar todos os ventiladores”))
{
Heating.sendCommand(OFF)
}

end

// vim: syntax=Xtend

2

Hi

I have this working in Portuguese with a different approach. I’m at my cell phone now, but will post my code here tomorrow.

João

1 Like
3

OK João
Thank you very much

4

Having previously done the same thing and created rules to interpret speech (or more accurately process speech-to-text results) my recommendation would be to stop going down this route. As you can already see, using rules you are limited in ways to phrase your command and are having to explicitly code for such i.e. command.contains("…")

IMO a better approach is to remove the processing of the natural language from your rules and use either the Google Assistant or Alexa integration as they are better at understanding and interpreting natural language AND mean you don’t have to write many lines of rules to cope with it all.

That being said, if you’re determined to stick with doing your command processing in your rules(maybe you have some other constraints or reasons) I don’t see many better ways of doing it.

5

Olá Ricardo

As promissed, I post my code. As you can see it is not mutch different from yours, but I think I managed to make it more flexible a easy to modify.

rule "controlo por voz"

when
Item VoiceCommand received command
	then
	var String command = VoiceCommand.state.toString.toLowerCase
	//sendNotification("xxxxxxxxx@gmail.com", "Recebi comando " + command)
	if (Mensagens.state==ON)
		
		{
			say(command , "voicerss:ptPT", "sonos:PLAY1:sala")
		}
	if (MenTVSala.state==ON)
		{
			LG_TV0_Toast.sendCommand(command)
		}
	if (command.contains("acende")||command.contains("acender")||command.contains("liga")||command.contains("ligar"))
			{
				if (command.contains("biblioteca"))
				{
					LampadaTetoBiblio.sendCommand(ON)
				}
				else if (command.contains("quarto"))
				{
					LampadaQPrincipal.sendCommand(ON)
				}
				else if (command.contains("laura"))
				{
					LampadaLaura.sendCommand(ON)
				}
				else if (command.contains("sala"))
				{
					LampadaTetoSalaTV.sendCommand(ON)
				}
				else if (command.contains("escada"))
				{
					LampadaEscada.sendCommand(ON)
				}
				else if (command.contains("cozinha"))
				{
					LampadaCozinha.sendCommand(ON)
				}
				else if (command.contains("entrada"))
				{
					LampadaHallPatio.sendCommand(ON)
				}
				else if (command.contains("computador"))
				{
					ValorLampBiblio.sendCommand("13")
				}
			}
	else if (command.contains("apaga")||command.contains("apagar")||command.contains("desliga")||command.contains("desligar"))
		{
			if (command.contains("biblioteca"))
					{
						LampadaTetoBiblio.sendCommand(OFF)
					}

					else if (command.contains("quarto"))
					{
						LampadaQPrincipal.sendCommand(OFF)
					}

					else if (command.contains("laura"))
					{
						LampadaLaura.sendCommand(OFF)
					}
					else if (command.contains("sala"))
					{
						LampadaTetoSalaTV.sendCommand(OFF)
					}
					else if (command.contains("escada"))
					{
						LampadaEscada.sendCommand(OFF)
					}
					else if (command.contains("cozinha"))
					{
						LampadaCozinha.sendCommand(OFF)
					}
					else if (command.contains("entrada"))
					{
						LampadaHallPatio.sendCommand(OFF)
					}
					else if (command.contains("computador"))
					{
						ValorLampBiblio.sendCommand(OFF)
					}
					else if (command.contains("tudo"))
					{
						Lampadas.sendCommand(OFF)
					}
				}
			else if (command.contains("abre")||command.contains("abrir"))
			{
			if (command.contains("quintal"))
			{
				PortaoPatioAbre.sendCommand(ON)
			}
			if (command.contains("tudo"))
			{
				AbreTudo.sendCommand(ON)
			}
			if (command.contains("biblioteca"))
			{
				ValorEstBiblio1.sendCommand("100")
				ValorEstBiblio2.sendCommand("100")
				ValorEstBiblio3.sendCommand("100")
			}
			if (command.contains("quarto"))
			{
				ValorEstQPrincipal.sendCommand("100")
			}
		}	
		else if (command.contains("fecha")||command.contains("fechar"))
		{
			if (command.contains("quintal"))
			{
				PortaoPatioFecha.sendCommand(ON)
			}
			if (command.contains("tudo"))
			{
				FechaTudo.sendCommand(ON)
			}
			if (command.contains("biblioteca"))
				{
					ValorEstBiblio1.sendCommand("0")
					ValorEstBiblio2.sendCommand("0")
					ValorEstBiblio3.sendCommand("0")
				}
			if (command.contains("quarto"))
				{
					ValorEstQPrincipal.sendCommand("0")
				}
		}
			
	
		
end

I understand that using Amazon Echo or Google Assistent its easier, but I chose no to do it for two reasons ( I own a Amazon Echo Dot that sits in my daughter’s room, so she can turn on anf off some lights )
1 These devices can not interpret Portuguese. At home we speak Portuguese. We could use English to interact with Echo or Google Home (we use it at my daughter’s room if we want to use the Dot), but it does no feel natural, so we almost never do.
2 .I’m not ready to have some machine in the living room, always listen to every thing said at home, waiting for the “magic word”.

I know that I will buy more Echos or some similar equipment eventually, but this will not happens if they cannot understand my language.

João

6

NOTE: If you are on OH 2, as written, you need none of those imports.

I’ve not done this myself but I would try to break the problem into parts. For example, first I’d try to figure out the command:

import java.util.regex.Pattern
import java.util.regex.Matcher
import org.eclipse.smarthome.model.script.ScriptServiceUtil

val Pattern digits = Pattern.compile(".*([\d]+).*")

rule “Voice control”
when
    Item VoiceCommand received command
then

    val command = VoiceCommand.state.toString.toLowerCase
    val Matcher digitsM = digits.matcher(command)

    // get the new state from the command
    var newState= ""
    switch command {
        case command.contains("on"): newState = "ON"
        case command.contains("off"): newState = "OFF"
        case command.matches(".* to [\d]+.*") : newState = digitsM.find.get(0)
        case command.contains("open"): newState = "OPEN"
        // and so on
    }

    // get the room
    var room = ""
    switch command {
        case command.contains("kitchen"): room = "Kitchen"
        case command.contains("master bedroom"),
        case command.contains("our room"): room = "MBR"
        // and so on
    }

    // get the device
    var device = ""
    switch command {
        case command.contains("light"): device = "Light"
        case command.contains("lights"): device = "Lights"
        case command.contains("lamp"): device = "Lamp"
        case command.contains("tv"): device = "TV"
        // and so on
    }

    // With clever naming we should have everything we need to command the device
    var itemName = room+"_"+device

    // Special cases
    if(newState == "" || room == "" || device == "") {
        // List each special case that cannot be handled by the keyword searching above
        switch command {
            case command.contains("it's party time"): { itemName = "PartyTime" newState = "ON" }
            // and so on
        }
    }

    // error checking
    val itemRef = val override = ScriptServiceUtil.getItemRegistry.getItem(itemName)
    if(itemRef == null) {
        logError("Voice", "Cannot find Item " + itemName + ": " + command)
    }
    else itemRef(newState)
end

Rather than trying to match full phrases or sentences, look for keywords and name your Items so that you can build the Item name from the words in the voice command. Notice how there are two cases that result in “MBR”. This is how you can handle synonyms.

If you need to do things like control color you will probably need to create some more regular expression Patterns to search for common color names.

Like I said, I don’t do voice control so this is notional, but I think it should work and it is a bit more flexible than listing phrases separately and it is a little easier to update and follow with nested if else statements.

One case it doesn’t handle well is statements like “turn off the lamp on the coffee table” since it contains both “on” and “off” but I bet this case could be managed if it were a problem.

2 Likes
7

I get an error in the string:
val Pattern digits = Pattern.compile(".([\d]+).")

[5,45]: Invalid escape sequence (valid ones are \b \t \n \f \r " ’ \ )
[20,38]: Invalid escape sequence (valid ones are \b \t \n \f \r " ’ \ )

and:
val itemRef = val override = ScriptServiceUtil.getItemRegistry.getItem(itemName)
therefore ignoring it: [57,19]: no viable alternative at input ‘val’

what can be?

8

Hmmm, that’s off. \d is a standard regular expression value. Change the \d to 0-9

val Pattern digits = Pattern.compile(".([0-9]+).")

That should do it. If not, see https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html for the full documentation for Pattern.

It does say the \d should work so I don’t know why it would be complaining about that.

9

val Pattern digits = Pattern.compile(".([0-9]+).")
all is well,

but

[INFO ] [el.core.internal.ModelRepositoryImpl] - Validation issues found in configuration model ‘voice.rules’, using it anyway:
The operator ‘==’ should be replaced by ‘===’ when null is one of the arguments.

and
[WARN ] [el.core.internal.ModelRepositoryImpl] - Configuration model ‘voice.rules’ has errors, therefore ignoring it: [59,19]: no viable alternative at input ‘val’

it is about:
val itemRef = val override = ScriptServiceUtil.getItemRegistry.getItem(itemName)

10

it will be better this way:
val itemRef = override = ScriptServiceUtil.getItemRegistry.getItem(itemName)

11

That isn’t an error, just informational. When ever you have null on either side of an equals or not equals, you need to use an extra “=”.

This line makes no sense. You can’t define an assign a variable on the same line.

You need to use something like:

val itemRef = ScriptServiceUtil.getItemRegistry.getItem(itemName)
val override = itemRef
12

after the replacement, the rule works. Many thanks for your help.

13

Tanks a lot friends

14

Hello. :slight_smile:

Your reply here caught my attention. I see you’ve managed to make your system respond by voice in Portuguese but I’m wondering if it is possible to use Google Home as the voice receiver to send the orders to OpenHab instead of having them go directly to the service’s cloud (for example Philips Hue). Not sure if I’m explaining myself right here. :laughing: By the way, how do you send the voice commands? Using your phone and clicking the microphone icon on the app?

Thank you.

15

Hi.
I just use the microphone icon in the app. Don’t know how to send the commands using Google Home.
To be fair, I use this feature less than before. As my system grows I have less need to send commands to OpenHab as the rules are taking care of the automation.

João

16

Oh. Because I have a few devices and automations but they use Google and their own services and I control the lights by voice but wanted to do it all at local level the same way I do today, I just tell Google to turn them on lol. I’ll keep looking, thank you. ^^

17

Olá!

My current workaround is Automate (I use Android so not sure if it’s available on other platforms) and you can create a fairly simple rule to listen to your Google command and send it over to OH. This app let’s you use the generic speech recognition block or the Google Now voice command. After the STT block you just need to add a Plug-in action block to send over the command to your specific OH Item.

If you choose to use the Speech Recognition option the only different thing you need to do is to add a widget on you home screen to start recording the command each time you need.

Hope this helps.
Cheers!