openHAB Cloud setup on Google Cloud Platform

Hello,

This is a guide on how to setup a openHAB Cloud instance on the Google Cloud Platform. I was initially following the instructions on the Quick start guide, and while it was mostly successful, it still has some issues to be resolved. I am recreating the VM there (basically because I want to make the disk smaller, and CGP does not allow one to easily do this), so I will be rewriting and improving the instructions.

VM setup

Sizing: micro (1 vCPU, 0.6 GB memory) - it may give you occasional performance warnings, but it could run mostly well enough
Disk: 10 GB HDD - minimum size required by the OS image, it should be more than sufficient
Firewall: Allow HTTPS traffic
Preemptibility: Off (recommended)
OS: Ubuntu 17.10

Before starting anything else, after connecting to the VM via SSH, get the latest packages:

sudo apt-get update
sudo apt-get upgrade

DNS configuration

A record created, pointing “mydomain.com” to the public IP of the VM instance. One option is to assign a static IP to the instance (but then you must keep it constantly running, otherwise you will be charged). To avoid that, you may want to use ddclient and configure it to regularly push your ephemeral IP to your DNS configuration:

sudo apt-get install ddclient
sudo vi /etc/ddclient.conf

openHAB Cloud install

sudo apt-get install build-essential redis-server mongodb nginx python git
cd /opt
sudo git clone https://github.com/openhab/openhab-cloud.git
cd openhab-cloud
sudo apt-get install npm
node --version
sudo npm install (lots of warnings while installing the dependencies)
redis-cli ping

Redis answered “PONG”, so let’s move to the web server setup.

Nginx configuration

sudo cp /opt/openhab-cloud/etc/nginx_openhabcloud.conf /etc/nginx/sites-available/default
cd /etc/nginx/sites-available
sudo sed -i 's/opt\/openhabcloud/opt\/openhab-cloud/g' default
sudo sed -i 's/your-openhab-cloud-host/mydomain.com/g' default

Here we have defined the domain name, and replaced /openhabcloud with /openhab-cloud, as the nginx default configuration does not match the installation directory name. If you don’t do this, the login page will load, but without any formatting:

Before restarting the web server, we can generate the SSL certificate. To keep it simple for now, you may generate a self-signed cert (or use Let’s Encrypt as described further below) and then restart the service:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/nginx/ssl/openhabcloud.key -out /etc/nginx/ssl/openhabcloud.crt
sudo service nginx restart

Final configuration and service startup

You first need to configure the application:

sudo cp config-production.json config.json
sudo vi config.json

    "system": {
      "host": "mydomain.com",
     [...]
    "mongodb": {
        "hosts": ["127.0.0.1"],
        "db": "openhab",
    },
    [...] Plus SMTP settings for "mail" and "mailer"

<ESC> :wq!

Please notice that mongodb come with a (blank) user and password, you need to remove these in order for the connection to work. You may also want to remove the password from redis, as leaving the blank entry could cause a warning in the logs. In a nutshell, besides removing these credentials, I only changed the domain name and defined SMTP configuration.

It is finally time to try to start openHAB Cloud as follows:

sudo node app.js

After trying to restart openHAB Cloud again, got an error, “Cannot find module ‘…/build/Debug/addon’”. After some additional searching on the web, this is how I could fix the issue and finally start it successfully:

npm rebuild (you may also need to upgrade nodejs to +7.10.1) 
sudo node app.js 
2018-01-04T05:38:43.539Z - info: openHAB-cloud: Backend service is starting up...
2018-01-04T05:38:43.548Z - info: openHAB-cloud: Backend logging initialized...
2018-01-04T05:38:44.071Z - info: openHAB-cloud: Initializing XMPP connection to GCM
2018-01-04T05:38:44.286Z - info: openHAB-cloud: Connecting ro Redis at 127.0.0.1:6379
2018-01-04T05:38:47.103Z - info: openHAB-cloud: Mailer will emulate sending in development environment
2018-01-04T05:38:47.304Z - info: opneHAB-cloud: Trying to connect to mongodb at: mongodb://127.0.0.1/openhab?poolSize=100
2018-01-04T05:38:47.361Z - info: openHAB-cloud: Scheduling a statistics job (every 5 min)
2018-01-04T05:38:48.220Z - info: openHAB-cloud: IFTTT is configured, app handling IFTTT capabilities...
Option polling duration is not valid. Please refer to the README.
2018-01-04T05:38:48.272Z - info: openHAB-cloud: express server listening on port 3000
2018-01-04T05:38:48.280Z - info: openHAB-cloud: Redis connect response: OK
2018-01-04T05:38:48.281Z - info: Redis is ready
2018-01-04T05:38:48.459Z - info: openHAB-cloud: Successfully connected to mongodb
2018-01-04T05:38:48.462Z - warn: openHAB-cloud: GCM XMPP error: XMPP authentication failure
4 Likes

Location directives in the nginx config file are in the form of /opt/openhabcloud and your openHAB cloud folder is /opt/openhab-cloud. I think the dash could be the problem.

Matthias, right on the spot, thank you for your reply! This indeed fixed the problem:

sudo vi /etc/nginx/sites-available/default
Occurrences of alias /opt/openhabcloud/... replaced with alias /opt/openhab-cloud/...
sudo service nginx restart

The static content now loads fine:

2018-01-04 10_00_03-openHAB Cloud - Home

As I still have warnings some warnings on the logs, I will try to fix them and implement some improvements such as making openHAB Cloud start as a service. I will report the steps here to serve as a reference for other users.

1 Like

I think I have enough meat to close this topic, as I’ve finally got a fully working system - here is how I got there:

Google Cloud Messaging

To get rid of the “GCM XMPP error: XMPP authentication failure” warning, I went ahead to generate Google Cloud Messaging credentials via Firebase:

  1. https://console.firebase.google.com/
  2. [+] Add project
  3. Select an existing Google Cloud project where Firebase will be added - as per the current rules, billing should be disabled on that project so you can use Firebase’s free tier
  4. Project settings > Cloud Messaging tab

There you will find the values you will need to update on config.json, Sender ID and Server key. You should copy these and add to openHAB Cloud’s config.json as shown below. We will also make another change that will be useful later on:

  "gcm": {
    "jid": "<Sender ID>@gcm.googleapis.com",
    "password": "<Server key>"
  },
[...]
  "registration_enabled": true

Further changes to have notifications actually working would be another topic, but once the process is restarted with the configuration in place, that will get rid or the warning:

info: openHAB-cloud: GCM XMPP connection is online

SSL Certificate powered by Let`s Encrypt

This is also a good opportunity to replace the self-signed certificate, which is actually not that difficult to do for free with Certbot:

sudo apt-get install -y certbot
sudo apt install python-certbot-nginx
sudo certbot certonly --nginx -d mydomain.com

 - Congratulations! Your certificate and chain have been saved at:
   /etc/letsencrypt/live/mydomain.com/fullchain.pem
   Your key file has been saved at:
   /etc/letsencrypt/live/mydomain.com/privkey.pem
   Your cert will expire on 2018-04-05. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot
   again. To non-interactively renew *all* of your certificates, run
   "certbot renew"

If you skipped the self-signed certificate and want to start with the Let’s Encrypt certificate, running the commands above would generate errors due to the missing certificates being referenced in the nginx config. You should comment out the line ssl on and the lines for ssl_certificate and ssl_certificate_key, before executing the commands above. You will also need to Allow HTTP traffic on the GCP firewall - you may disable it after the certs are generated.

Now we need to point Nginx configuration to where these certificates are located, and then restart the web server:

sudo vi /etc/nginx/sites-available/default (and then update the lines below)
ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;
<ESC> wq!
sudo service nginx restart

Since Let`s Encrypt certificates expire every few months, you may want to check on Certbot documentation how to use cron to have certificates updated automatically.

openHAB Cloud Connector

Next step is to configure the openHAB Cloud Connector on our local openHAB instance. Go to Paper UI > Add-ons > MISC > openHAB Cloud Connector > Install. After the service is installed, go to Configuration > Services to adjust the settings:

  • Base URL for the openHAB Cloud server should point to https://mydomain.com
  • Select items you want to expose to external integrations (e.g. IFTTT)

We now need to connect to your local server via SSH to retrieve the “UUID” and “Secret” credentials that will be used to configure the remote openHAB Cloud instance:

cat /var/lib/openhab2/uuid
cat /var/lib/openhab2/openhabcloud/secret

You should copy the output of these commands that will be used to configure our Cloud instance. Now go to https://mydomain.com/ on your browser and create a new user, supplying the “UUID” and “Secret” you just copied from your local openHAB. If you don’t plan creating additional users, you can now disable registration on config.json:

  "registration_enabled": false

Production readiness

One requirement for a stable production-ready system is that it doesn’t generate too many logs, so we should adjust the openHAB Cloud configuration’s logging level from the default “debug” to a less verbose “info”:

sudo vi /opt/openhab-cloud/logger.js
 transports: [
        new (winston.transports.Console)({json: false, timestamp: true, level: "info"})
    ],

I am leaving the exceptionHandlers definition as debug, since I may want more details in the logs in case something goes wrong.

We now need to keep openHAB Cloud running even if we hit ^C or if we close the SSH session. The powerful PM2 process manager can be installed on our VM to accomplish this goal:

sudo npm install pm2 -g
pm2 start app.js --name="myopenhab"

You may refer to the PM2 documentation to see how to stop, restart and see logs openHAB Cloud and any other Node.js applications that PM2 is asked to keep running as a daemon.

One of the last steps to finalize the configuration is to restart your local openHAB instance, so it tries to connect to openHAB Cloud using the previously configured UUID and Secret credentials:

sudo systemctl restart openhab2

Connection should be working now! Finally, you may want to configure the Remote URL on your mobile clients (Android, iOS).

  • Remote URL: https://mydomain.com
  • Username and Password: email and password for the user created on openHAB Cloud (it’s not the UUID and Secret!)

This completes the initial configuration of openHAB Cloud on a Google Compute Engine VM and the adjustments needed on the client-side to connect to it.

1 Like

@Flavio_Costa

What is the purpose of this integration vs. the Official Google Assistant Integration for openHAB?

Thanks.

Mike

This is for those users who want to run their own openHAB Cloud instance instead of using myopenhab.org.

Hi All

I understand under the PaperUI configuration you configure https://domain name.

Problem is, when I do this OpenHab Cloud reports OpenHAB offline and sends me a GCM alert to tell me so.

When I configure http://192.168.0.3:3000 it works just fine.

The domain is reachable via the internet. Any suggestions? racking my brain here.

2019-02-27 21:22:46.152 [INFO ] [io.openhabcloud.internal.CloudClient] - Shutting down openHAB Cloud service connection

Cheers

@dastrix80, I don’t remember having faced any issue like this. The steps below show exactly what I have done over one year ago, and it has been working well since then.

There are so many small steps related to domain name that is probably easy to forget something and have problems. I would suggest checking each step in this configuration to make sure everything has been properly adjusted to your domain name. Also make sure you have the correct “UUID” and “Secret” credentials in place.

Other than this, my suggestion would go exactly around what you have suggested yourself - making sure the system is reachable over the Internet. In my case, I have the cloud instance running on port 443, the DNS is pointing to the right IP and the firewall rules on Google Console allow the public IP to receive connections from the Internet. My local openHAB instance is running on a Raspberry Pi that does have access to the Internet, so it all works.

Hopefully this might give you any ideas that could help find the issue on your setup.

Good luck,

Flavio

1 Like

Hi Flavio, I suspect its DNS/LAN related because I host the OH2 Cloud on 3000 on the same machine as OH2, on my LAN. When I use the HTTPS://Domain name, it would go out to the internet and back in again

I might try placing the OH2 Cloud and NGINX on a VM on the internet and see if the issue persists.

Thanks for the response, I didn’t think it was really anything wrong with your instructions more a specific setup on my network. I’d be keen to know if anyone has this locally.

Cheers

Hi @Flavio_Costa

I managed to fix the issue.

a combination of NGINX and DNS :slight_smile:

Question: are you using the mailer to send emails for password resets?

Good that you got this working for you! As I always had this setup with openHAB Cloud on the Internet, it just worked for me from the beginning.

I am the only registered user and I never really use SMTP regularly, so while this is something I did add to my configuration it is not something that is actually tested.

1 Like

Thanks Flavio, can you test it and see if it works or even just provide what you used?

Just some learnings from someone who has recently endeavoured to get this working:

  • GCP doesn’t list the Ubuntu 17.10 image any longer. You need to use the Ubuntu 18.04 LTS image (19.04 did not work for me).

  • Installing node on Ubuntu 18.04 LTS using “sudo apt-get install npm” results in Node 8.x.x. This doesn’t work with openhab-cloud from my experience. You need to downgrade to 7.10.1. To do this, after running “sudo apt-get install npm”, you need to run:

$ npm install -g n
$ n 7.10.1

1 Like

@Flavio_Costa

Hi! I have a personal openhab cloud hosted on an amazon server, but I have troubles setting up SMTP settings for “mail” and “mailer”. I am new to openHAB and I can’t figure out how to make the cloud send emails when there is a new registration or when the “Forgot your password?” option is used. Can you help me with this? Also, I have a Gmail email with custom domain.

Hello all, as I have messed up my server by trying to update Ubuntu (17.10 has reached end-of-life), I redid this procedure so I am documenting here how it went for anyone trying to setup openHAB Cloud now.

These are the steps I followed:

  1. Created a new micro VM, with both HTTP and HTTPS enabled in the firewall and choosing Ubuntu 18.04 LTS Minimal (note: it does not come with vi or nano, so if you don’t want to edit the configuration with sed you may need to install a text editor).

  2. Executed apt-get update and upgrade

  3. Installed ddclient (service other: “domains.google.com”, protocol: dyndns2, username and password retrieved from Google Domains, use: web, ssl: yes, and then your domain).

  4. Configured a 1GB swap file, since the server has limited memory and apt-get install for the openHAB Cloud requirements was getting stuck on “extract → gunzTarPerm” (this is the only new/different procedure I had to follow)

  5. Executed the openHAB Cloud install (git, redis, nginx, mongodb, npm, etc.).

  6. Setup Certbot to generate certificates with Let’s Encrypt (as mentioned above, 3 lines had to be commented out from /etc/nginx/sites-available/default before running Certbot for the first time). Then I disabled plain HTTP on the firewall.

  7. Configured the domain name and certificate in Nginx.

  8. Configured openHab Cloud (setting the domain, removing certain passwords, setting up GCM, adding SMTP settings, enabling user registration, on config.json).

  9. Retrieved the UUID and Secret from my local openHAB server and created a user on openHab Cloud with those values; then I disabled registration. Afterwards, the local openHAB server had to be restarted to “synch” with the cloud.

  10. Used PM2 to start app.js as a daemon.

It essentially followed the instructions above, that did not change significantly since they were first written. The following, however, is what I did not do:

  • Node worked with the version provided by default, I did not need to downgrade it to 7.10.1.

  • I did not generate any self-signed certificates, as the process described for Let’s Encrypt worked well.

  • I did not get the “Cannot find module ‘…/build/Debug/addon’” error as in the first time, so no npm rebuild was needed.

  • No change was needed on logger.js, as the log level was already set to “warn” by default.

I have not received any emails at any point (e.g. during the user registration). I don’t know if it is not supposed to send any emails or if I have wrong settings somewhere (SMTP, firewall, etc.). It is not a critical feature to me, so I didn’t spend time troubleshooting that.

With this new setup, I have a bit better performance, maybe because I created the server in a geographically nearer location, because of the swap file, or both. By the way, this is how I did the swap configuration to have a more robust service (all commands with sudo):

fallocate -l 1G /swapfile
chmod 600 /swapfile
mkswap /swapfile
swapon /swapfile
sed -i '$ a /swapfile   none    swap    sw    0   0' /etc/fstab

This is the key to getting openhab cloud working on ubuntu 18!

Hi there, greetings, I tried it now in 2020 and still having issues. Can someone please post details on how the config file looks , and on starting app.js console just goes to new line with the prompt as if I pressed an enter key. No log, error, warning, just new line. SO more confused. Also please can we have a documentation on openhab-cloud installation. If possible a video tutorial showing all the files and setup will be even more helpful.
Thanks in advance if anyone is still reading. Any help appreciated.

I just saw the log files and it says xyz module not found like winston,winston-daily-rotate-file,mongoose,bcrypt, etc. When i try “npm install xyz” , then a new module throws same error like something else not found. It never stops saying so always a new module name comes. Can someone tell me how to fix this so that I can debug further. Thanks in advance.
EDIT:
1 UPDATE - I tried “npm install -g n” as well but no stone moved.

@Flavio_Costa thanks for the detailed post. I did try the steps you outlined in this post and found that I had to make some slight modifications in order to get it to work. Below are the steps I took to get openHAB Cloud running on Google Cloud Platform.

These steps do not include instructions to pointing a domain at a Google Cloud Compute Engine as these steps can be different depending on your domain provider. YouTube has some good videos describing the process. I watched this one https://youtu.be/cG9kv5-5bPI and it helped me figure out how I could redirect my IONOS Domain Name Server setting to Google Cloud DNS. I’m also using a static IP address for this setup and may investigate dynamic DNS once my free trial begins to run out.

I will try to add some screenshots later.

I should also note that I’m still waiting on some hardware to build a openHAB client so I don’t know if anything works.

1. Setting up the Virtual Machine

Log on to Google Cloud Platform Console by going to https://cloud.google.com/.

In the navigation menu (hamburger bars on the top left) go to Compute Engine and click on VM instances.

In the VM instances click Create New Instance.

This will open up the Create an instance menu. Below are the settings i used.
Name: I used openhab-cloud but I’m pretty sure you can name it what ever you like
Labels: left blank
Region and Zone: I choose us-central1 and us-central1-a. This was the default setting. As long as you don’t choose an international or us-east4 you should be in a free tier machine.
Machine family: General Purpose
Series: N1
Machine type: f1-micro
Deploy a container image to this VM instance: not checked
Boot Disk: Change this to a Public Image, Ubuntu, Ubuntu 18.04 LTS, Standard persistent disk, Size 30 (GB). This was the only image that worked for me.
Identify and API access: Left as default
Firewall: Allowed both HTTP and HTTPS traffic
Click on Management, security, disk, networking, sole tenancy
Click on Security
Check Turn on Secure Boot
Click on Networking
Hostname: Enter the domain name for your openHAB Cloud Server. This will be based on the domain name that your currently own or a subdomain in my case (openhab.somedomain.net)
Edit the Network Interfaces
External IP: Create IP address
Name: I used the domain name for the server with -'s but you can probably use anything.
Network Service Tier: Standard
With all these settings I click Create to create the Virtual Machine Instance.

2. Logging in to the Virtual Machine

Once the Virtual Machine has been set up you will be taken to the VM instances page. In a few seconds there will be a green check box next to the machine you just created. Once there on the right hand side there will be a SSH button. I selected Open in browser window to create a terminal window to the Virtual Machine.

3. Creating a swap file

I created a swap file based on @Flavio_Costa post on Oct’19. Below are the commands.

sudo  fallocate -l 1G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo sed -i '$ a /swapfile   none    swap    sw    0   0' /etc/fstab

4. Updating the Virtual Machine

With the swap file created I updated the Virtual Machine using apt-get update and upgrade

sudo apt-get update
sudo apt-get upgrade

5. Install the prerequisite software

I executed the following commands to get the prerequisite software.

sudo apt-get install build-essential redis-server mongodb nginx python git nodejs npm

This should result in node version 8.10.0. This can be conformed by runing the following command

node --version

I also found that I had to command npm to update itself or else I would get errors when I tried to install. I did this by by running the following command.

sudo npm install -g npm

With that completed I setup nginx for SSL

6. Getting an SSL certificate for your domain and server

Prior to completing this step you need to make sure you can access your Virtual Machine via it’s domain name. By default a simple webpage is created when nginx is installed. If you can’t access your Virtual Machine via the domain name you assigned it you will be unable to proceed.

For this I used Certbot. I’m not going to put the instructions here as I used the instructions on the Certbot website https://certbot.eff.org/lets-encrypt/ubuntubionic-nginx. I ran in to errors the first time and that was due to that fact that I could not access the web server via my domain name. I selected the redirect to SSL option.

If this is successful make note of the location of the certificate and private key as this will be important as we configure the nginx server later

7. Installing openHAB Cloud

I also installed openHAB Cloud to the /opt directory. This requires that you get the git repository and place it in the /opt directory. This is done by doing the following.

cd /opt
sudo git clone https://github.com/openhab/openhab-cloud.git

Now we can run npm to do the install. I found that I had to set additional flags in order for it to work.

cd openhab-cloud
sudo npm install --unsafe-perm=true --allow-root

This will generate a lot of warning but you should not see any errors. At the end it found 61 vulnerabilities (20 low, 12 moderate, 18 high, 2 critical)

Now with that completed I setup the nginx server for openHAB Cloud

8. Configuring openHAB Cloud on nginx

For this I pretty much followed the instructions from @Flavio_Costa on Jan’18. The only deviation was that I saved a copy of the default file. Below are the commands.

sudo  cp /etc/nginx/sites-available/default /etc/nginx/sites-available/default.old
sudo cp /opt/openhab-cloud/etc/nginx_openhabcloud.conf /etc/nginx/sites-available/default
cd /etc/nginx/sites-available
sudo sed -i 's/opt\/openhabcloud/opt\/openhab-cloud/g' default
sudo sed -i 's/your-openhab-cloud-host/mydomain.com/g' default

I did not do these exact commands as I used vim to edit default and change the values directly.

You will also need to update the ssl_certificate and ssl_certificate_key with the information you received in step 6. It should look something like this.

ssl_certificate /etc/letsencrypt/live/yourdomain/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/yourdomain/privkey.pem;

I also restarted nginx by executing the command below.

sudo service nginx restart

You will have to update your domain accordingly. Note at this point if you try to access your openHAB Cloud website nothing should happen except maybe a 404 error.

The next step is to configure openHAB Cloud

9. Configuring openHAB Cloud

You will need to configure openHAB Cloud’s configuration. I should note I’m still waiting on a openHAB client so I don’t know if I configured everything.

Using the instructions from @Flavio_Costa I copied the configuration file and named it config.json

cd /opt/openhab-cloud
sudo cp config-production.json config.json

With that copied I used vim to edit it

sudo vim config.json

I updated the host name to be my domain name for the openHAB-Cloud server. I added a Google Cloud Messaging API key following @Flavio_Costa instructions on Jan’18. I also updated the mailer configuration to be aligned with one that my domain name company provided.

This this completed I started openHAB-cloud

10. Starting openHAB Cloud

For this I just did the following command and received general warnings.

sudo node app.js

After running this command and waiting a few seconds I was able to see the openHAB Cloud log in page when I sent to my domain. I also used pm2 to run the openHAB Cloud

cd /opt/openhab-cloud
sudo npm install pm2 -g
sudo pm2 start app.js --name="myopenhab"

Conclusion

This resulted in me having a openHAB Cloud server that I can see on my domain. I haven’t tried to configure it to a openHAB client yet but that will be my next step.

I hope this information was helpful and that it resolves any issues you may of had in trying to setup openHAB Cloud of Google Cloud Platform.

Thank you @Flavio_Costa for the post as the information your provided was extremely helpful.

3 Likes

Thank you dear. I will certainly try in and will try to share my update. Thanks a lot for sharing. Later . . .
UPDATE : I tried your instruction above and it worked. The openhab cloud server is up and running on GCP. Thanks a lot.

However I was still not able to config mail mailer but I think that might just be me. Why can the names be simple like incoming and outgoing , anywas thanks to @Flavio_Costa and @J_Cat, together with both the input by you two, I was able to setup my openhab cloud server. The new instructions by @J_Cat worked hasslefree.
:beers::beer: