Long story short - Backup often, backup soon and write tutorials

How?
I suspect you mean, you can restore a backup with one command, which means it let your system back to the latest backup, right?

Ansible playbooks are designed for automating complex tasks. If all the configuration is setup in ansible, one command could just run all the commands to re-create the configuration.

Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.

https://docs.ansible.com/ansible/latest/index.html

Yack!! :astonished:

I just took at quick look… Started the quick start video…
13 minutes just to tell what Ansible is?? I can create an image backup and restore it again faster than that. And I have not even done anything regarding Ansible, which I understand may be very good for restoring, but there seem to be pretty much setup behind.

he told you

google is yer bud

And I gave them a link to the documentation and they read it.

What point are you trying to make here? Verifying you do not read the posts in a thread before posting?

Here’s my “super simple” backup script, good if you have a NAS at home or other sort of device running FTP server. Uses openhab-cli backup command within a custom script to get backups off to alternate storage. Hasn’t missed a beat since I’ve set it up. Saved my arse twice (although now on SSD HDD, coz SD flash cards suck)

pound sand Bruce!

No, I mean set up a new machine with one command.

With Ansible, everything that you do to a machine gets coded in playbooks. Every installed or uninstalled program, every change to every config, any files that need to be moved to the machine get captured in the playbook.

So yes, you do need to keep a backup of sorts for some things (e.g. database contents, openHAB configs) but Ansible can be used to both take those backups and restore them. For openHAB configs I use git as a backup for the configs.

Then I just run ansible-playbook -i <path to my inventory file> homeautomation.yml. The inventory tells Ansible what machine is the home automation machine(s) and homeautomation.yml has all of the roles and playbooks to build up my home automation machine from a base Linux with nothing configured to my preferred configuration running openHAB, Mosquitto, InfluxDB, Grafana, and Shinobi with the most recent database backup restored to InfluxDB and the most recent openHAB configs checked out from my git server.

It’s not a backup in the traditional sense of the word. It’s a complete rebuild from scratch. But the playbooks are all also checked in to git and because everything is encoded in the playbooks, all of the configs are also self documented. And because I use git, I also have a history of changes I’ve made to my machines.

And upgrades are a breeze. I just spin up a new VM with the new OS (e.g. moving from Ubuntu 18.4 to 20.4) and run the playbook and I’m up and running.

It’s not about backup and restoring. It’s about infrastructure as code. You don’t set up your machines. You code how you want the machines set up. Once you’ve done that you’ve created a configuration that is repeatable, self documenting, and traceable.

What would happen if your images got lost or the restore didn’t work? What will happen when you need to move from stretch to buster and doing an inplace upgrade of the OS fails? Do you remember how you set everything up? Do you remember every little tweak and change you’ve made? You’re gonna have to redo them in either of these scenarios. That’s why the OP mentioned tutorials and why I mentioned Ansible as an alternative to tutorials.

If you are only dealing with one machine, one probably set up with openHABian in the first place, it’s probably no big deal and Ansible is way overkill. I have 5 VMs, 6 RPis, and well over a dozen services installed and running not to mention scores of tweaks and minor changes to all of these machines. I’d never ever remember how I set up and configured them all. With Ansible I don’t have to.

@Bruce_Osborne and @Andrew_Rowe, there is no need to get angry here. We are all friends.

4 Likes

Hmm… In a short term, there is no difference :smiley:

Does it get recorded, or do you have to manually create scripts?
( I admit, I only looked for 7 minutes at the quick start video. I simply felt this it was way beyound any decent backup system).

This is a usual problem with backup systems… Often you´ll need to do something, before you can restore your backup. (like having a clean OS up running). Only exception is, if you do drive/image copy, or if you have a backup procedure which can run from boot.

Hmm so Ansible doesnt touch the main OS (core system) ??

I agree, these are all fall groups in a backup system, speciellt when using image copy.
A image copy which fails is fatal, like any other backup systems, where the restoring fails.
The best way to do image backups, is to do it regulary (once a day).
If I need to upgrade the OS, and it fails. There is a totally new situation. A typical backup cant helt here either. But I do understand the advantages when using Ansible or simular. I just cant see how a very complexed system would be easy to code in Ansible.

But I guess you´ll have to spend alot of time coding them first time?

If you need additional software installed in addition to the basic install, this can be done in the ansible playbook too. One command does it all.

It can if you wish (your choice), The place I work uses it to manage the OS on Linux servers.

It is probably not too bad if it was done while he was building the configuration.

Does Ansible “record” then?

Ansible is text file based. At work I have my text configurations for network switches stored in a Git repository that also acts as a configuration backup.

So you´ll need to enter the code in text files while you´re setting up your system?

1 Like

You have to manually create the scripts, or download them from Ansible Galaxy or GitHub. Most common software has had some developer already create a Role for you so all you have to do is download it and run it. Once you get some experience, I find it takes less time to code it than to do it from the command line, especially for things that need to be done to more than one machine.

For example, I recently discovered that ssmpt is no longer supported on Raspbian Buster and msmtp is recommended. So I figured out how to install it and configure it from Google. Coded up the role to do all the steps (including uninstalling ssmtp). Then I tested it on one machine, fixed the bugs if there are any, and verify it worked. Then I run it again on all the machines. It took about 20 minutes to switch all 11 of my machines from ssmtp to msmtp. And now when I need to set up a new machine or VM, msmtp will just get installed and configured and I don’t even need to remember that I’m using msmtp.

It’s not a backup system. It’s not intended to be a backup system and I didn’t bring it up to be used as a backup system. I only brought it up to address the “tutorials” mention in the OP. You don’t need tutorials if everything you’ve done is already captured in an ansible playbook.

It absolutely does touch the core OS. But as I’ve said before, the whole idea of the OS being this separate thing on Linux is not correct. The “core OS” is really just the version of the Linux kernel + the thousands of base libraries + a pre-selected set of software that have been tested and known to work well together. All of that, including the kernel, can be updated by apt-get. But only within a given release of the distro. When they release a new version of the OS it’s a much bigger deal. There can be many breaking changes between the versions and the software may be wired together differently. Therefore they treat it as a wholly new and separate thing. Running apt-get update will not migrate you from Ubuntu 18.4 to Ubuntu 20.4.

and it is not always possible and pretty much never advisable to upgrade a distro like that in place. They always recommend a fresh install of the OS because an in place upgrade leaves a lot of cruft, always takes longer (for an RPi we are talking hours), and it often fails leaving you with a broken OS that needs to be reinstalled from scratch anyway.

So in the upgrade case using Ansible, you would install a fresh Ubuntu 20.4 (when it comes out), create your user (which is usually part of the installation process) and configure ssh (Ansible works through ssh). Then run the ansible playbook and it rebuilds the machine on the new OS.

It’s like openHABian only it supports anything, not just openHAB stuff.

If you only have a disk image, you are potentially stuck forever on that old and potentially unsupported version of Raspbian (or whatever) if for some reason the in place upgrade fails to work.

There is a little bit of up front learning of the syntax. Once you have that, it really takes no longer to code it in Ansible than it does to just type it by hand.

For example, here is my msmtp role discussed about above:

---
- name: Install msmtp
  apt:
    name: [msmtp, msmtp-mta, mailutils]
  become: yes

- name: Create config
  template:
    dest: /etc/msmtprc
    mode: u=rw,g=r
    src: msmtprc
  become: yes

The first task installs msmtp and related packages. The second task copies the config file I have as part of my role (I’m not going to show it as it has me password in it) to the destination machine.

That’s it.

Even something really complex like cloning a git repo and building the source code isn’t much more work than that.

Far less time than setting it up by hand on all of those machines and, in my experience, once you learn Ansible, less time than doing it by hand the first time. It’s really not that complex.

Probably my most complex role is my Mosquitto role, but that’s because I have it generate the self signed certificates and set up a user and password as part of the role. It combines around three different tutorials from three different websites. But as you can see, it’s a little long but very straight forward to understand what it’s doing.

---

- name: Create mosquitto user
  user:
    comment: 'Mosquitto'
    createhome: no
    name: mosquitto
    shell: /bin/false
    state: present
    system: yes
  become: yes

- name: Mount mosquitto from file share
  include_role:
    name: mount-cifs
  vars:
    mount_mode: '0660'
    cifs_user: "{{ share_user }}"
    cifs_pass: "{{ share_pass }}"
    cifs_domain: "{{ workgroup }}"
    mount_user: "mosquitto"
    mount_path: "{{ mosquitto_data }}"
    mount_src: "{{ mosquitto_mount }}"

- name: Create mosquitto directories
  file:
    path: "{{ item }}"
    state: directory
    mode: u=rwx,g=rwx,o=rx
  become: yes
  become_user: mosquitto
  with_items:
    - "{{ mosquitto_data }}/config"
    - "{{ mosquitto_data }}/data"
    - "{{ mosquitto_data }}/log"

- name: Copy the prepared mosquitto.conf
  copy:
    src: mosquitto.conf
    dest: "{{ mosquitto_data }}/config/mosquitto.conf"
    mode: u=rw,g=rw,o=r
  become: yes
  become_user: mosquitto

- name: Install mosquitto cients, temporarily install mosquitto
  apt:
    name: "{{ item }}"
    update_cache: no
  become: yes
  with_items:
    - mosquitto
    - mosquitto-clients
    - openssl

- name: Install pexpect
  pip:
    name: pexpect
  become: yes

- name: Generate passwd file
  expect:
    command: mosquitto_passwd -c {{ mosquitto_data }}/config/passwd {{ mosquitto_user }}
    responses:
      'Password\:' : "{{ mosquitto_passwd }}"
      'Reenter password\:' : "{{mosquitto_passwd }}"
  become: yes
  become_user: mosquitto

- name: Remove mosquitto
  apt:
    name: mosquitto
    update_cache: no
    purge: yes
    state: absent
  become: yes

- name: Recreate mosquitto user
  user:
    comment: 'Mosquitto'
    createhome: no
    name: mosquitto
    shell: /bin/false
    state: present
    system: yes
  become: yes

- name: Does the CA already exist?
  stat:
    path: "{{ mosquitto_data }}/config/ca.crt"
  register: mosquitto_ca

- name: Create mosquitto certificate authority
  expect:
    command: openssl req -new -x509 -days 1461 -extensions v3_ca -keyout {{ mosquitto_data }}/config/ca.key -out {{ mosquitto_data }}/config/ca.crt
    responses:
      'Enter PEM pass phrase\:' : "{{ mosquitto_ca_passwd }}"
      'Verifying \- Enter PEM pass phrase\:' : "{{ mosquitto_ca_passwd }}"
      'Country Name \(2 letter code\) \[AU\]\:' : "{{ mosquitto_ca_country }}"
      'State or Province Name \(full name\) \[Some\-State\]\:' : "{{ mosquitto_ca_state }}"
      'Locality Name \(eg, city\) \[\]\:' : "{{ mosquitto_ca_city }}"
      'Organization Name \(eg, company\) \[Internet Widgits Pty Ltd\]\:' : "{{ mosquitto_ca_org }}"
      'Organizational Unit Name \(eg, section\) \[\]\:' : "{{ mosquitto_ca_unit }}"
      'Common Name \(e\.g\. server FQDN or YOUR name\) \[\]\:' : "{{ mosquitto_ca_fqdn }}"
      'Email Address \[\]\:' : "{{ mosquitto_ca_email }}"
  when: mosquitto_ca.stat.islnk is not defined
  become: yes
  become_user: mosquitto

- name: Create server key and cert
  include_role:
    name: create-cert
  vars:
    key: "{{ mosquitto_data }}/config/server.key"
    csr: "{{ mosquitto_data }}/config/server.csr"
    crt: "{{ mosquitto_data }}/config/server.crt"
    cert_country: "{{ mosquitto_ca_country }}"
    cert_state: "{{ mosquitto_ca_state }}"
    cert_city: "{{ mosquitto_ca_city }}"
    cert_org: "{{ mosquitto_ca_org }}"
    cert_unit: "{{ mosquitto_ca_unit }}"
    cert_fqdn: "{{ mosquitto_ca_fqdn }}"
    cert_email: "{{ mosquitto_ca_email }}"
    ca_crt: "{{ mosquitto_data }}/config/ca.crt"
    ca_passwd: "{{ mosquitto_ca_passwd }}"
    ca_key: "{{ mosquitto_data }}/config/ca.key"
  become: yes
  become_user: mosquitto

- name: Get uid
  command: id -u mosquitto
  register: mosquitto_uid

- name: Get gid
  command: id -g mosquitto
  register: mosquitto_gid


- name: Run the eclipse-mosquitto docker container
  docker_container:
    detach: True
    exposed_ports:
      - 1883
      - 9001
      - 8883
    hostname: mosquitto.koshak.net
    image: eclipse-mosquitto
    log_driver: syslog
    name: mosquitto
    published_ports:
      - "1883:1883"
      - "9001:9001"
      - "8883:8883"
    recreate: True
    restart: True
    restart_policy: always
    state: started
    user: 999 # TODO why can't I use the variable created above?
    volumes:
      - /etc/passwd:/etc/passwd:ro
      - /etc/localtime:/etc/localtime:ro
      - /usr/share/zoneinfo:/usr/share/zoneinfo:ro
      - "{{ mosquitto_data }}/config:/mosquitto/config"
      - "{{ mosquitto_data }}/log:/mosquitto/log"
      - "{{ mosquitto_data }}/data:/mosquitto/data"

NOTE: There is some deprecated syntax in the above I’ve not bothered correcting yet.

Stuff in {{ }} are variables defined in another file. The only think unusual here is that I mount the mosquitto folder from a samba share using another role called mount-cifs. Over time I’m slowly replacing samba with NFS. When I do, I’ll have an mount-nfs role and call it instead.

Anyway, you should be able to scroll through that role and figure out what each one of the roles does. In fact, you could probably figure out the exact command line(s) that you would execute by hand for each task. It’s not that hard. In fact it’s far easier than developing openHAB Rules.

It does take some discipline though. No manual changes allowed. All changes, tweaks, config updates, etc must be done through Ansible. If not, then they will be lost when you need to rebuild that machine. That can be frustrating if you only want to change one word in a config file or the like. So you either need to do the change in Ansible or after you make the change remember to update the Ansible roles.

No. But does looking up the user task in the Ansible reference and typing:

- name: Create mosquitto user
  user:
    comment: 'Mosquitto'
    createhome: no
    name: mosquitto
    shell: /bin/false
    state: present
    system: yes
  become: yes

take more time than looking up a tutorial for creating a new user (I forget, which is better adduser or useradd), looking at the man page to remember the arguments to make a system user with no shell or home directory and typing:

adduser --system --no-create-home --disabled-password --disabled-login mosquitto

And if you add in the work of writing that down somewhere so you don’t have to look it up again and remember what you’ve done then you’ve actually created more work for yourself in not using Ansible.

Yes.

Here’s the general flow if you want to install something new.

  1. Create a new role directory structure of your Ansible folder. I use ansible-galaxy init roles/newprogram for that. This is usually on some other machine.
  2. Research the new program so you understand what needs to be done to install it (you have to do this anyway).
  3. Edit roles/newprogram/tasks/main.yml.
  4. Look at the Ansible reference docs to the Ansible equivalent for all of the commands that you need to run as discovered in step 1. Write the tasks as necessary into main.yml. Ansible is usually idempotent meaning you can rerun the same tasks over and over again and nothing will happen if there is nothing to change. This will let you build up the role step by step if necessary, letting you check that everything is working before moving on to the next lines.
2 Likes

Speaking to the original post, I back up my text configuration files and my org.ops4j.pax.logging.cfg whenever I make changes to them by copying them to Google Drive. Only takes a minute to do that.

When my SD card crashed in August, it was a simple task of restoring the last full backup and then copying the most recent text files over to get up and running.

2 Likes

Even though I´m only able to understand part of the scripts (I guess it would be easier if I was used to Linux). I cant help thinking, that this may not seem like a huge job, but insted it looks more like doing a setup twice. Ofcouse there is the benefit of only doing this once and remember to add changes. And I do understand the smart task in just having a bunch of scripts to set up a machine from. But as you said, for one machine, this would be way overkill.

Ansible is based on Python which is OS agnostic just like Java . It uses standard YAML syntax which is used by other software too.

Yep, exactly the files that I also backup regulary. Put them in a freh install and you are fine again :wink:

Why? How? You code it once in the playbook and then execute it. You don’t go and make the changes by hand and then update the playbook. The whole point is that all changes to all the machines under Ansible management are only ever made through Ansible. You only do it once.

I cheat a little bit for very minor changes but that’s bad and I shouldn’t do it.

Please don’t think that you log into a machine and start hacking on the keyboard to, for example install and configure Mosquitto and then go back to Ansible and convert what you did to a role. No no no. You start with the Ansible Role. You may never need to log into the other machine directly at all.

1 Like

I thought you were setting up your “real” machine, and afterwards was coding the Ansible scripts?

Okay, thats what I thought, which is why I said it looks like doing the same twice.