OrangeAssist - Google Assistant Integration

Follow steps for configuring audio devices

https://developers.google.com/assistant/sdk/guides/service/python/embed/audio

I went through the guide and tested the speakers as described. I can hear the voice playing.
I then launched orangeassist and ran my query again, same error.
:confused:

# speaker-test -t wav

speaker-test 1.1.3

Playback device is default
Stream parameters are 48000Hz, S16_LE, 1 channels
WAV file(s)
Rate set to 48000Hz (requested 48000Hz)
Buffer size range from 512 to 65536
Period size range from 512 to 65536
Using max buffer size 65536
Periods = 4
was set period_size = 16384
was set buffer_size = 65536
 0 - Front Left
Time per period = 0.374149
 0 - Front Left
Time per period = 1.372050
 0 - Front Left
Time per period = 1.707663
 0 - Front Left
^CTransfer failed: Bad address
(env) [11:51:45] root@openHABianPi:/home/openhabian/orange-assist/tmp# python -m or
Cred file: /root/.config/google-oauthlib-tool/credentials.json
 + Found!
# orange assist v1 #
Server OrangeAssist with config:
{
    "credentials_file": "/root/.config/google-oauthlib-tool/credentials.json",
    "delete_output_files_sec": 60,
    "device_id": "OpenHAB",
    "device_model_id": "celeste-openhab-orangeassist",
    "enable_speaker": true,
    "host": "192.168.1.80",
    "is_debug": false,
    "is_verbose": true,
    "on_success": null,
    "password": null,
    "port": "2828",
    "project_id": "celeste-openhab-orangeassist",
    "screen_mode": "SCREEN_MODE_UNSPECIFIED",
    "username": null
}
Cred file: /root/.config/google-oauthlib-tool/credentials.json
 + Found!
Ready!
ERROR:flask.app:Exception on /assist/ask [POST]
Traceback (most recent call last):
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/openhabian/orange-assist/tmp/or.py", line 98, in decorated
    return f(*args, **kwargs)
  File "/home/openhabian/orange-assist/tmp/or.py", line 126, in create_task
    t, h = answer(r)
  File "/home/openhabian/orange-assist/tmp/or.py", line 82, in answer
    text_response, html_response = orange.assist(r)
  File "/home/openhabian/orange-assist/tmp/orangeassist.py", line 173, in assist
    flush_size=DEFAULT_AUDIO_DEVICE_FLUSH_SIZE
  File "/home/openhabian/orange-assist/tmp/helpers/audio_helpers.py", line 190, in __init__
    blocksize=int(block_size/2),  # blocksize is in number of frames.
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/sounddevice.py", line 1264, in __init__
    **_remove_self(locals()))
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/sounddevice.py", line 681, in __init__
    samplerate)
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/sounddevice.py", line 2490, in _get_stream_parameters
    info = query_devices(device)
  File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/sounddevice.py", line 488, in query_devices
    raise PortAudioError('Error querying device {0}'.format(device))
sounddevice.PortAudioError: Error querying device -1
192.168.1.143 - - [2019-01-31 11:52:32] "POST /assist/ask HTTP/1.1" 500 412 0.025555

Can you paste what’s in your .asoundrc file

.asoundrc

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
  type plug
  slave {
    pcm "hw:0,0"
  }
}
pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
  }
}

result of aplay -l

**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 7/7
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

arecord -l lists no devices so I have tried the .asoundrc file with the pcm.mic section and the reference removed but it didn’t make any difference.

speaker-test -t wav

speaker-test 1.1.3

Playback device is default
Stream parameters are 48000Hz, S16_LE, 1 channels
WAV file(s)
Rate set to 48000Hz (requested 48000Hz)
Buffer size range from 512 to 65536
Period size range from 512 to 65536
Using max buffer size 65536
Periods = 4
was set period_size = 16384
was set buffer_size = 65536
 0 - Front Left
Time per period = 0.381968
 0 - Front Left
Time per period = 1.362102

And I can hear the test voice

Did you install portaudio

sudo apt-get install portaudio19-dev libffi-dev libssl-dev

The only other thing I could think of is that your audio devices is busy. Do you know how to setup snooping for pi? If not, let me know and I will give you instructions

Edit: looks like I did here

That will allow your audio devices to be used concurrently by any process

Yes, but I did it again just to be sure:

 sudo apt-get install portaudio19-dev libffi-dev libssl-dev libmpg123-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libffi-dev is already the newest version (3.2.1-6).
libmpg123-dev is already the newest version (1.23.8-1).
libssl-dev is already the newest version (1.1.0j-1~deb9u1).
portaudio19-dev is already the newest version (19.6.0-1).
0 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.

Tested and same issue.


I updated my .asoundrc file to match yours in the thread you linked (my HW IDs are the same):

pcm.snoopedmic {
    type dsnoop
    ipc_key 2241234
    slave {
        pcm "hw:0,0"
        channels 1
    }
}
pcm.mixedspeaker {
    type dmix
    ipc_key 2241234
    slave {
        pcm "hw:0,0"
        channels 1
    }
}

pcm.!default {
        type asym
        playback.pcm {
                type plug
                slave.pcm "mixedspeaker"
        }
        capture.pcm {
                type plug
                slave.pcm "snoopedmic"
        }
}

Still the same issue:

...
File "/home/openhabian/orange-assist/tmp/env/lib/python3.5/site-packages/sounddevice.py", line 488, in query_devices
    raise PortAudioError('Error querying device {0}'.format(device))
sounddevice.PortAudioError: Error querying device -1
192.168.1.143 - - [2019-01-31 15:57:17] "POST /assist/ask HTTP/1.1" 500 412 0.024712

Yep. It looks like this project is probably going to replace that one, if I can just get some time. I’m also trying to get Shinobi set up and having some “fun” reading the H.264 stream from one of my RPi cams. MJPEG works just fine so I may just give up and move on. I only have three cameras so it’s not like I need that many resources anyway.

I’ll get back to this this weekend I hope.

@luckymallari, I never touched python before.
Now I followed your tutorial and it worked up to this point:
image
I’m able to talk to my assistant and it switches lights for example.

But I don’t understand how to connect to this python-machine via http.
Which port/url do I use?

And I had to choose a device-type - there I selected “speaker”.
Is this relevant?

EDIT: It works now :slight_smile: After setting debug to false I reached the host/port from the config.json

EDIT2:
WORKS: Switch on < light >
WORKS: Remind me to < something > at < time >
DOESN’T WORK: play < movie > on netflix on shield
DOESN’T WORK: play < song > on < device or group >

Go to config.json and set debug to false. You will more configuration stuff there. Use the html tester so you know what to send to the url when you configure OH

@luckymallari technically it works now - but many commands don’t get a result. Especially the “play music on device” commands. Neither in english nor in german.

Did you register your device using the device tool (part of the SDK) and you can confirm your instance shows under your Google Home app?

I am having problem to create an instance of the device.

googlesamples-assistant-devicetool register-device --model "rpi-assistant-7cf1b-rpi-assistant-o137ha"

i just receive Error: Missing option “–project-id”.
Have tried with

googlesamples-assistant-devicetool register-device --project-id "rpi-assistant-7cf1b"  --model "rpi-assistant-7cf1b-rpi-assistant-o137ha"

Same error.

How did you do it

EDIT I did not success with the command but now i see it in my Google Home :grinning:

/Mike

Is it possible to run tester from a remote computer?

I am running this on a pi without browser.

/Mike

Yes. It’s a rest api

What is the url to tester.html.

/Mike

When i tigger how’s my day it does not respond in my Google Home.

/Mike

Well of course. It won’t trigger your GH. The code itself is the assistant. You get the response through the API. You need to trigger routines if you have routines set in your GH app. I had mine to broadcast “my day” to the kitchen GH device.

Spent all night trying to figure out why this won’t run… Was using this -

root@ubuntu1:/orange-assist# python3.7 -m orangeassist

when i should be using this incase someone needs it

root@ubuntu1:/orange-assist# python3.7 -m or

And in general, you shouldn’t be using root :wink: