Switches occasionally do not react / are not updated

Wolfgang_S · January 10, 2020, 11:19pm

I am running 2.5.0 together with a custom myopenHAB cloud.
Switches occasionally do not react in the alexa skill on my android phone. Alexa via voice command also sometimes tells me that it looks like there would be a problem with my network.
So far I did not find the root cause. I think the network cannot be the problem.
The problem only occurs using the skill ( voice and panel with switches in the skill ). Using paper UI is ok.
What I observed by logging the connections on the smartphone is that the skill tries to connect to us-east1, us-west2 hosts in the amazon cloud in case of problems while the skill is hosted in eu-west1.
The configured language is german and distribution region is germany.

Is there any other location that needs to be configured to use the correct lambda server endpoints ?

jeshab · January 11, 2020, 4:19am

You deployed your Lambda function in the correct region. Have you checked your Lambda function logs for errors? Also, add the environmental variable LOG_LEVEL=info to that function to get additional log information.

Wolfgang_S · January 11, 2020, 11:15am

Lambda function is deployed to eu-west-1. The ARN on the lambda function page in the console also contains eu-west-1. The same ARN that is shown on the lambda function page is used as default endpoint as well as endpoint for europe.

I set environment varialbe LOG_LEVEL to info.
Extract from the log:

10:52:05 REPORT RequestId: b4ed0ceb-87ba-4c89-9890-dac1dc3cb1b5 Duration: 468.89 ms Billed Duration: 500 ms Memory Size: 128 MB Max Memory Used: 103 MB
10:52:15 START RequestId: 7c7ddaa9-58cf-4a1d-8a67-6a331235d482 Version: $LATEST
10:52:15 2020-01-11T10:52:15.619Z 7c7ddaa9-58cf-4a1d-8a67-6a331235d482 INFO INFO: Input: {"directive":{"header":{"namespace":"Alexa","name":"ReportState","payloadVersion":"3","messageId":"82d4c763-139a-4e74-ac00-dd0f60c401dd","correlationToken":"AAAAAAAAAQCr+evDB0I23OvBeG4UqOxKDAIAAAAAAACvMqeg3Q2llGl8StoSpTWSv3zgz7B1IMGiKYC2UAt54+8L6T3aXHFyd8x5AlvYeULZGbLPtOWN8aVJoflSfRbblQAcfutK7TklcZnyASEzJ1W3YwxQwP/OxAx
10:52:15 2020-01-11T10:52:15.621Z 7c7ddaa9-58cf-4a1d-8a67-6a331235d482 INFO WARN: RequestError: socket hang up {"directive":{"namespace":"Alexa","name":"ReportState","endpointId":"Rule_TV_einschalten","payload":{},"propertyMap":{"PowerController":{"powerState":{"parameters":{"category":"SWITCH"},"item":{"name":"Rule_TV_einschalten","type":"Switch"},"schema":{"name":"powerState"}}}},"token":"vFG2qrz2dkJYLm0
10:52:15 2020-01-11T10:52:15.621Z 7c7ddaa9-58cf-4a1d-8a67-6a331235d482 INFO INFO: Response: {"event":{"header":{"namespace":"Alexa","name":"ErrorResponse","payloadVersion":"3","messageId":"9da96676-edec-4bdc-80de-492d908491f4","correlationToken":"AAAAAAAAAQCr+evDB0I23OvBeG4UqOxKDAIAAAAAAACvMqeg3Q2llGl8StoSpTWSv3zgz7B1IMGiKYC2UAt54+8L6T3aXHFyd8x5AlvYeULZGbLPtOWN8aVJoflSfRbblQAcfutK7TklcZnyASEzJ1W3YwxQwP/OxA
10:52:15 END RequestId: 7c7ddaa9-58cf-4a1d-8a67-6a331235d482

It says: WARN: RequestError: socket hang up

Do you have any advice what I need to check next ?

jeshab · January 11, 2020, 2:54pm

A socket hang up could be a configuration issue with your reverse proxy server where idle sockets aren’t staying up long enough. The skill requires a minimum of 45 seconds idling between two requests before it should timed out.

Also, make sure to use the latest skill version available on the master branch repository. We made some changes back in September related to idle socket handling.

github.com/openhab/openhab-alexa

Request socket hang up error

opened 08:21PM - 01 Sep 19 UTC

closed 07:01PM - 03 Sep 19 UTC

jsetton

bug

Due to the change in #265, adding request keep alive http agent support (`foreve…r: true`), we have been getting sporadically a socket hang up error causing relevant requests to fail. This seems to be caused by the native nodejs http agent not handling gracefully keep-alive free sockets that have timed out on the server side. Unless defined in the myopenhab cloud connector nginx configuration, the default server [keep-alive timeout](https://nginx.org/en/docs/http/ngx_http_core_module.html#keepalive_timeout) is currently set to 75 seconds; while an AWS lambda execution context can be inactive for up to 30 minutes between two invocations. The options to resolve this issue are one of the following: - Remove the keep-alive support - Add retry logic when getting this error - Use a better suited http agent such as [agentkeepalive](https://www.npmjs.com/package/agentkeepalive) which allows to specify a free socket timeout that we could set to something like 45 seconds to ensure that these sockets are cleanup prior to server-side socket timeout. The reason for that time value is that based on the logs, most of the consecutive invokes for a given execution context happens within that timeframe. I think that, with the first option, we would lose the response time benefit for not reusing existing connections during peak times. With the second one, we would add some overhead and therefore some added response time going through the motion of first getting the initial reuse connection request rejected and then creating a new connection. While the last one seems to be the most efficient but would rely on an additional module. ``` RequestError: Error: socket hang up at new RequestError (/var/task/node_modules/request-promise-core/lib/errors.js:14:15) at Request.plumbing.callback (/var/task/node_modules/request-promise-core/lib/plumbing.js:87:29) at Request.RP$callback [as _callback] (/var/task/node_modules/request-promise-core/lib/plumbing.js:46:31) at self.callback (/var/task/node_modules/request/request.js:185:22) at emitOne (events.js:116:13) at Request.emit (events.js:211:7) at Request.onRequestError (/var/task/node_modules/request/request.js:881:8) at emitOne (events.js:116:13) at ClientRequest.emit (events.js:211:7) at TLSSocket.socketCloseListener (_http_client.js:331:9) ``` [Lambda log reference](https://eu-west-1.console.aws.amazon.com/cloudwatch/home?region=eu-west-1#logEventViewer:group=/aws/lambda/openhab-alexa-v3;stream=2019/09/01/%5B$LATEST%5D747f2c5cc0d2488cb230e04db67134ba;refid=34953075651410008503219671030792499089368180731301265409;reftime=1567350119480)

Wolfgang_S · January 11, 2020, 5:44pm

Perfect. Thank you.

On Dec 16th I updated to latest version of the skill. Since that time I had the problem.
Changing the keepalive_timeout configuration parameter of the nginx server solved the problem.

JornHermans · January 12, 2020, 8:17pm

I think i have a simulair problem, first i tought it was my connection to the custom openhab cloud from my local instance, buth now i have that resolved…

I can succesfully control my items in alexa, buth the state is not staying and sometimes i get the notice device unreachable. Can’t seem to figure out what i did wrong…

I have the latest version and my reverse proxy is configured right…

jeshab · January 15, 2020, 7:56pm

How many items do you have setup on your server? You could be hitting the other outstanding issue potentially related to the reverse proxy cache buffering. Maybe you could disable that functionality based on my comment listed in the issue below.

github.com/openhab/openhab-alexa

Discovery items type error failure

opened 04:10PM - 19 Aug 19 UTC

jsetton

bug

Per the skill logs, several users discovery requests are failing due to the erro…r below ``` error: discover failed with error:items.forEach is not a function { "stack": "TypeError: items.forEach is not a function\n at Promise.all.then (/var/task/alexa/v3/discovery.js:45:13)\n at <anonymous>\n at process._tickDomainCallback (internal/process/next_tick.js:228:7)" } ``` After enabling debug mode, the issue seems to be related to the items data value being a string because the request call seems to only have read/received a partial response preventing the expected JSON object from being parsed properly. The example based on the reference below isn't that considerably big in size, although it's just partial data, and the Lambda execution time for that request was about 5 seconds. [Lambda request log reference](https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEventViewer:group=/aws/lambda/openhab-alexa-v3;stream=2019/08/16/%5B$LATEST%5D64be4d08d373457fa4a1b2468f58d794;refid=34922537345580123117038527377609257753437664721776476160;reftime=1565980734486)

JornHermans · January 19, 2020, 4:01pm

Got it almost right the largest server has 140 items, buth they al do the same.
Now it is always working this on both the EU lambda server and the US one.

Buth still it forgets it status so you see light is on buth it dissapears after 5-10seconds…

EDIT: Also sometimes i get device not reachable: and this error in acceslog

18.206.45.79 - - [19/Jan/2020:17:12:08 +0100] “POST /rest/items/Relay_03_04 HTTP/1.1” 499 0 “-” “-”

dastrix80 · March 6, 2020, 10:20am

I too have this issue, running the latest skill. Mind you, it was happening randomly on a very old skill too. It comes and goes but is around more often than not.

I had alexa working with private openhab for over 1 year without issues and recently its started to break.

My openhab cloud is definitely not disconnecting, so this is some kind of alexa related issue.

I use nginx, but again, nothing changed there so I cant see why this would be an issue

I have 65 items, not a huge amount

Any suggestions? Cloudwatch doesnt show the error in much detail. I’ve turned off proxy_buffering in nginx so ill see how that goes.

Thanks