OrangeAssist - Google Assistant Integration

[luckymallari (Lucky)]

If you cannot issue the command using your phone, this won’t work with OrangeAssist either.

[spicerackk]

So this could be exactly what I am looking for. I want to configure a group of all of my devices so that as I leave/arrive home, everything turns off/on as required. The issue with that is that Chromecast power controls aren’t available in OH.

Would this be able to be integrated into a rule so that once i leave home, a broadcast is sent to all CC that are on and they turn off?

[fabio_folino]

I cannot seem to get the device registered. I think other users are experiencing this as well, but I didnt see any resolve. Is the command I have below correct?

(env) $ googlesamples-assistant-devicetool --project-id OpenHabFab register --model ‘hab-orange1’ --type SWITCH --trait action.devices.traits.OnOff --manufacturer ‘Assistant SDK developer’ --product-name ‘Assistant SDK Speaker’ --description ‘Assistant SDK Speaker’ --device ‘hab-orange’ --nickname ‘My Assistant’ --client-type LIBRARY

Creating new device model
Error: Failed to register model: 400
Could not create the device model. Check that the request contains the required field project_id with a valid format in the request payload. See Device Model and Instance Schemas  |  Google Assistant SDK  |  Google for Developers for more information.

I’m wondering if the update to Google Actions console has anything to do with this.

[fabio_folino]

I don’t want to deter anyone from Lucky’s solution, but for those that may want it, I found this on GitHub. https://github.com/greghesp/assistant-relay

Was very easy to install and is under active development.

[rlkoshak (Rich Koshak)]

A separate thread with a quick tutorial would be most welcome. Especially how you manage to use it with openHAB.

[fabio_folino]

Google Assistant Relay

I was looking for an easy way to setup broadcast to our google homes from OH. I found this project on github and it was a breeze to setup. This tutorial assumes you have a project setup on the Google Developers console, and you’ve enabled the Google Assistant API, created credentials, and downloaded your client secret. Make sure you have Node installed from https://nodejs.org/en/ Install PM2 as a pre-requisite if not already installed sudo npm i pm2 -g Download a copy of the latest relea…

[kluszczyn]

Hi All, @luckymallari,

I successfully installed OrangeAssistant, registered with Google, I have it visible in Google Home app, but I can’t make it responding to requests:

OA config/debug info:

# orange assist v1 #
Server OrangeAssist with config:
{
    "credentials_file": "credentials.json",
    "delete_output_files_sec": 60,
    "device_id": "B51",
    "device_model_id": "ha-orange-b51-orange-b51-e31crv",
    "host": "0.0.0.0",
    "is_debug": false,
    "is_verbose": true,
    "on_success": null,
    "password": "oa",
    "port": "2828",
    "project_id": "ha-orange-b51",
    "screen_mode": "SCREEN_MODE_UNSPECIFIED",
    "username": "kluszczyn"
}
Ready!

0. OA installed on the same machine as Openhab, port 80 free, port 2828 free

Debug info:

1. when browsing from another computer in the same lan http://oh.lan:2828 (oh.lan is server where OA is installed):

192.168.1.92 - - [2020-01-03 18:42:27] "GET / HTTP/1.1" 200 271 0.001288

in browser I receive orange! message

2. when trying browsing http://oh.lan:2828/helpers/tester.html

192.168.1.92 - - [2020-01-03 18:44:50] "GET /helpers/tester.html HTTP/1.1" 404 490 0.000508

resulting in The requested URL was not found on the server. message in browser

3. when OA is started with config "is_debug": true it responds to commands issued in console.

4. I tested also REST response with following rule, based on one from tutorial:

val orangeassistPostURL = "http://kluszczyn:oa@oh.lan:2828/assist/ask?html=1"
val timeoutSec = 10

rule "Send OrangeAssist Command"
when
	Item GOA_orangeassistcmd received command
then
    logInfo("OA", triggeringItem.state.toString)
	var result = sendHttpPostRequest(orangeassistPostURL, "text/plain", GOA_orangeassistcmd.state.toString, timeoutSec*1000)
    logInfo("OA", result)
	postUpdate(GOA_orangeassistcmdResult, result)
	GOA_orangeassistcmdSwitch.sendCommand(ON)
end

rule "command OA"
when
    Item GOA_Command received command 
then
    GOA_orangeassistcmd.sendCommand("What time is it?")
end

and receiving logs:

2020-01-03 18:59:22.689 [INFO ] [rg.eclipse.smarthome.model.script.OA] - What time is it?

2020-01-03 18:59:22.693 [INFO ] [rg.eclipse.smarthome.model.script.OA] - {

    "text": "NO OUTPUT. DOH!"

}

and in OA console:

192.168.1.6 - - [2020-01-03 18:59:22] "POST /assist/ask?html=1 HTTP/1.1" 200 298 0.000651

5. I was changing "screen_mode" to OFF and PLAYING - no change in output

I’m running out of ideas. Any suggestion welcomed!

[luckymallari (Lucky)]

@kluszczyn

when browsing from another computer in the same lan http://oh.lan:2828 ( oh.lan is server where OA is installed):

You dont access the helper from a URL as it is not exposed. You simply open the html file on your browser.

It’s been a while since I worked on this and I have and still using this on a daily basis. I am contemplating on making this an actual Binding. The regular Google Assistant Actionwe currently have is basically what it is: It integrates our DIY devices and exposes them go Google Home ecosystem via the Actions SDK. A new Binding will be helpful to do it the other way around, i.e., have OpenHAB control other Google Assistant devices.

The Binding I have in mind will have:

1. Have an audio sink, so that users can register the binding as a sink and have the “say” command issue a broadcast command to GH. This will be helpful since it will to continue to play whatever was playing prior to the broadcast, as opposed to the Chromecast binding where it simply creates an audio from TTS, then streams it to the devices, which causes the currently playing media to stop and not resume.
2. Java based, no NodeJS/Python needed, just the Binding itself.

The trickiest part is to parse the result of the command since the “answers” from GA will vary. For example, asking what time it is sometimes yield to “It’s 2AM” or “It’s noon.” We can probably set the item as string and just have the user parse it through a rule or a map.

Once the Binding is done, I plan to expand Orange Assist to do same for Alexa, so that it’s both Google and Alexa assistant capable…

[kluszczyn]

@luckymallari

You dont access the helper from a URL as it is not exposed. You simply open the html file on your browser.

Thanks Lucky for explaining. One issue cleared.
For the second - not receiving responses via REST - I gonna start from the ground, second chance

[rlkoshak (Rich Koshak)]

@luckymallari

A new Binding will be helpful to do it the other way around,

Does it make sense to add this to the existing binding rather than needing to separate bindings to interact with the same service?

[luckymallari (Lucky)]

While it does, and I actually thought about it hard, I don’t think we’d want to complicate the existing binding. Setting up OH->GA will require the steps posted at page 1 of this thread: Setting Google project, enable assistant API, etc.

[kluszczyn]

@kluszczyn

For the second - not receiving responses via REST - I gonna start from the ground, second chance

Fixed!
I had to change http request from text/plain to application/json and format text as json string:

val orangeassistPostURL = "http://kluszczyn:oa@oh.lan:2828/assist/ask?html=1"
val timeoutSec = 10

rule "Send OrangeAssist Command"
when
	Item GOA_orangeassistcmd received command
then
    logInfo("OA", triggeringItem.state.toString)
	var result = sendHttpPostRequest(orangeassistPostURL, "application/json", '{"request":"' + GOA_orangeassistcmd.state.toString +'"}', timeoutSec*1000)
    logInfo("OA", result)
	postUpdate(GOA_orangeassistcmdResult, result)
	GOA_orangeassistcmdSwitch.sendCommand(ON)
end

rule "command OA"
when
    Item GOA_Command received command 
then
    GOA_orangeassistcmd.sendCommand("openHAB")
end

and:

2020-01-05 03:07:28.535 [INFO ] [rg.eclipse.smarthome.model.script.OA] - openHAB

2020-01-05 03:07:30.563 [INFO ] [rg.eclipse.smarthome.model.script.OA] - {

    "request": "openHAB",

    "text": "OpenHAB (Software). open Home Automation Bus is an open source home automation software written in Java. It is deployed on premise and connects to devices and services from different vendors. As of 2019, close to 300 bindings are available as OSGi modules.\nWikipedia ( https://en.m.wikipedia.org/wiki/OpenHAB )."

}

[rlkoshak (Rich Koshak)]

I don’t know, maybe if it can be made optional. It just seems confusing and complicated from the user’s perspective to have two bindings. But if that’t the way it has to be s be it. It would be better to have the capability whether or not we have two bindings than not to have it.

[luckymallari (Lucky)]

@rlkoshak

It would be better to have the capability whether or not we have two bindings than not to have

Totally agree. Still unsure if a separate binding is optimal. Consider that I am planning to do the same for Alexa. So choices are:

1. New Binding (will have Google Home and Alexa)
2. Extend current Bindings
   a. Alexa
   b. Google Assistant

To me, #2 could even be more confusing. I am open to suggestions

[luckymallari (Lucky)]

@kluszczyn

I had to change http request from text/plain to application/json and format text as json string:

Awesome! I updated the OP

[rlkoshak (Rich Koshak)]

I don’t see 2 being confusing from a user’s perspective. People already all why they can’t tell GA to do stuff with the existing bindings. I’m not positive, but I think that the Alexa binding already does have Orange Assist type capabilities.

Users will expect that if they install a GA binding, they will get two way integration. Having two bindings to connect to GA would be more confusing.

Similarly, having one binding that supports both services would be unexpected. In OH 2, the trend had been to move away from such bindings (see weather 1 versus all the separate v2 weather service bindings). I don’t know if this is official policy or just how things developed. But it does establish expectations for the users.

[Doxlike (Pierre Peters)]

Hi,

does anyone know why it didnt fire everything? So i have a few “Abläufe” i think its called rules in Ghome us. Some of them are triggered when i send the configured command and some are not.
I would like to send the command “Starte Tims abend” and it should fire 2 things in the assistant 1:“spiele die Playlist playlistname im büro von google play music ab” 2:“Stelle die Lautstärke im büro auf 20%”

So here are the screenshots:

When i fire it with a voice command in front of an google home mini or when i type it in the assistant on my mobile its working.

So did i miss something?

br
Doxlike

[Doxlike (Pierre Peters)]

i can only add one picture

[Doxlike (Pierre Peters)]

Answer in the activitys on google.com

[luckymallari (Lucky)]

It is not a full GHome, it is limited with what the API can provide