Disconnected from the openHAB Cloud service

Are the servers a bit more unstable the last weeks? I’ve noticed the last month (estimate) that google will say that ‘openhab is unreachable’ (not often, one or two times a week). Usually after a few minutes it responds again. Before this happened rarely…

(edit: ah, seems I wasn’t looking at the last post … so let me rephrase the above: also experience this :wink: )

My monitoring flagged 17 issues with the cloud connector starting after 19:30 EDT on 2022/10/28 and ending before 00:45 on 2022/10/29. During that time, I was able to login to myopenhab.org - it indicated that I was offline. Most of the issues were timeouts (timeout was five seconds) - my Internet monitor showed no issues from my end. There were some error 500 “Server Error: openHAB is offline for url: https://myopenhab.org/rest/items/…” issues, which I saw when I disabled the cloud connector at my end during testing.

I monitor every 15 minutes. Of the 20 samples, three succeeded without errors, two after a single error attempting to reach myopenhab.org, and eight after one restart of the cloud connector. In seven attempts, two restarts were attempted - there were four consecutive double restart attempts between 22:15 and 23:00. The issues disappeared sometime after 00:30 - there was nothing in my openhab logs between 00:30 and 00:45 that suggested anything changed at my end. Since then, no further issues have been detected.

I am not able to change the log level to DEBUG. From openhab-cli console:

openhab> log:get org.openhab.io.openhabcloud
INFO
openhab> log:set DEBUG  org.openhab.io.openhabcloud
Error executing command: Unable to set level for logger

I am running OpenHAB 3.3.0 upgraded from OpenHAB 3.2.0. It might be related to log:set doesn't work when manually edited log4j2.xml exists · Issue #1217 · openhab/openhab-distro · GitHub (possibly fixed in OH 3.4.0-SNAPSHOT) or a permissions issues (have not had the time to dig further).

Yesterday and overnight I didn’t see any further problems, so the bundle restart hasn’t had a chance to be tested.

What I currently experience / what leaves me puzzled:

  • I experienced the same reported disconnects and reconnects from openHAB Cloud between October 8th and 10th and since October 22nd.
  • Even though the last entry showed „reconnect“
2022-10-31 12:43:42.240 [INFO ] [io.openhabcloud.internal.CloudClient] - Disconnected from the openHAB Cloud service (UUID = xxx, base URL = http://localhost:8080)
2022-10-31 12:44:09.197 [INFO ] [io.openhabcloud.internal.CloudClient] - Connected to the openHAB Cloud service (UUID = xxx, base URL = http://localhost:8080)

Alexa still does not work / myopenhab says that my server is offline

  • I normally also lean towards this…

… but, since the connection appears to be down now for almost 9 hours, I suspect it won’t come up again? I have not yet tried to reconnect the cloud connector, because I‘d like to understand why my server does not reconnect? Assuming I would be away (not being able to do the reconnect myself), the server should manage to get online all by itself.

Any ideas?

Correct. Your server doesn’t know that it’s disconnected, so there’s no need to reconnect. You can confirm this by going to myopenhab.org, which will show that your system is “Offline”.

Also, correct. Per the various ongoing discussion threads, this is happening to a lot of people. Something’s wrong, but it’s not your system. So the maintainers are trying to fix it.

The short-term fix is to restart the cloud connector. The easiest way to do this is to save the settings via MainUI, which forces a disconnect/connect.

I know it’s frustrating. What I find strange is that some are more affected than others. I’ve only experienced this once or twice in the past few weeks, and I don’t know why I would be luckier than anyone else.

3 Likes

Yes.

Though, funny twist: My server did just manage to go back online all by itself. So I can go to bed now with a smile. :slight_smile:

2 Likes

Apart from the openhab Server that is kind a „down“ I experianced the issue, that my logview is down as well. I cant resch ip:9001 any longier. Even a restart did not fixed the Problem. Is someone facing the same problem?

That’s probably not related to this discussion, but oddly enough a similar thing happened to me. At 1am this morning, the log froze and MainUI became inaccessible. However, my sitemap and rules still worked.

I’m chalking it up to an old SD card that needs to be replaced.

Restarting the binding “org.openhab.io.openhabcloud” seems to work.

I’m using Javascript/JSR223

RestartBinding("org.openhab.io.openhabcloud");

export//
function RestartBinding(sBindingID)
{
  var Exec = Java.type("org.openhab.core.model.script.actions.Exec");
  var Duration = Java.type("java.time.Duration");
  // @ts-ignore
  Exec.executeCommandLine(Duration.ofSeconds(6), "sudo", "/usr/bin/ssh", "-p", "8101", "-i", "/var/lib/openhab/.ssh/openhab.id_rsa", "openhab@localhost", "bundle:restart", sBindingID);
}

The way I detect that the cloud has disconnected is by creating an item Cloud_Heartbeat_TimeStamp, which I update by calling the cloud API from rules every 10 minutes. If it stops getting updated, it probably means the cloud connection is down.

4 Likes

Seeing the same here since at least a week or so. My girlfriend starts getting annoyed by Alexa saying the hub is not responding :).

Hello, for some time I have had the problem that the connection to myopenhab.org is interrupted once a week and I cannot access it from on the road. If I then edit the settings for the openhab cloud in Openhab, it works again for a week.
Since a few days I have to do this almost every day.
Does anyone have similar problems and an idea for a solution?
Good, I can still access it via my private VPN - but that’s always just plan B.
Greetings, Ansgar

I’ve had this issue a year ago. Reinstalling OpenHAB somehow fixed it. Just untill two weeks ago, when this started to be a problem again. XHR Polling Error in the Logs.

I just reinstalled OpenHAB again and voila … it still goes offline periodically.

Can someone help me with this?

Right now https://home.myopenhab.org/ is giving me: 504 Gateway Time-out

Been having this regularly the past several days. Becoming annoying when the Mrs asks Google to turn the light off in the nursery for putting the little one down. Even a rule restarting it a couple times a day isn’t enough…

Maybe it’s not relevant but I just checked my OH 2.5 logs and I can’t see any disconnect for more than a month… Wether in my local logs nor in in the myopenhab.org logs

Hi LordLiverpool
This sounds like something I need, I was trying to find a way to extract the online/offline status from myopenhab. My plan was to change the colour of some led striplights.
can you explain a bit more how you use the timestamp?

First I create a “cloud heartbeat” item in my items file:

DateTime Cloud_Heartbeat_TimeStamp "Cloud access heartbeat date-time [%1$tY.%1$tm.%1$td %1$tH:%1$tM:%1$tS]" (gAllDates)

Then I have a function that I run every ten minutes that updates the item via the cloud, using the REST API. That way, if cloud access is interrupted, the “heartbeat” stops. I don’t know how much detail you need. I’ll post a couple of the relevant functions.

// Update this date-time item via the cloud. If the cloud is down, it will not get updated
ZombificationCheck.prototype.SendCloudHeartbeat = 
function SendCloudHeartbeat() 
{
  // logInfo("SendCloudHeartbeat");

  try
  {
    var headers = GetOpenHabCloudCallAuthorisationHeaders();
    // logInfo("Cloud heartbeat headers: " + headers);

    var dateNow = new Date();

    DoHttpPutRequest("https://myopenhab.org:443/rest/items/Cloud_Heartbeat_TimeStamp/state", "Set cloud heartbeat", "*/*", dateNow.toISOString(), headers);     
  }
  catch(exception)
  {
    logInfo("Exception caught in SendCloudHeartbeat. Message: " + exception);   
  }
}

export//
function GetOpenHabCloudCallAuthorisationHeaders()
{
  var headers = [];

  headers['Content-Type'] = '*/*';
  headers['accept'] = '*/*';
  headers['Authorization'] = 'Basic xxxxxxxxxxxxxxxxxxxxxxxxxxxxx'; // Cloud user and pw encoded
  headers['WWW-Authenticate'] = 'Basic';

  return headers;
}

What I’m trying to do, and what may be working - but I’m not sure because lately the cloud disconnections have stopped, is restart the relevant bundle when the cloud heartbeat stops for too long.


ZombificationCheck.prototype.CheckIfAutomateAndCloudOffline = 
function CheckIfAutomateAndCloudOffline() 
{
   // Try restarting cloud client if down for a short amount of time
  if (dgReboot.GetMinutesSinceReboot() > 10 && nCloudHeartbeatAge > this.CLOUD_MAX_HEARTBEAT_AGE && (nCloudHeartbeatAge < this.CLOUD_MIN_AGE_ALERT_THRESHOLD || nCloudHeartbeatAge > dgReboot.GetMinutesSinceReboot()))
  {
    SendPushNotification("Attempting to restart cloud binding because cloud down");
    RestartBundle("org.openhab.io.openhabcloud");
  }
}

export//
function RestartBundle(sBundleID)
{
  var Exec = Java.type("org.openhab.core.model.script.actions.Exec");
  var Duration = Java.type("java.time.Duration");
  // @ts-ignore
  Exec.executeCommandLine(Duration.ofSeconds(6), "sudo", "/usr/bin/ssh", "-p", "8101", "-i", "/var/lib/openhab/.ssh/openhab.id_rsa", "openhab@localhost", "bundle:restart", sBindingID);
}

Getting the bundle restart to work was a bit involved, I recall. I can dig into the details if you want - I’d have to refresh my memory.

3 Likes

I can now definitely confirm that restarting the bundle from rules re-establishes the connection. Why the connection doesn’t re-establish itself is a different question.

Me and probably others are still having the following issue. Anyone know of any news on this issue?
Going to the “dashboard” on myopenhab returns a blank white page but e.g Google Assistant is connected fine.

OpenHAB retured empty sitemap list - Apps & Services / openHAB Cloud - openHAB Community