openHABian: add some additional default security and reliability settings

@ThomDietrich and other openHABian users.

I don’t run openHAB on a Pi so do not use openHABian myself, but I do run a bunch of Pis and have been experimenting with a number of features and settings to make it secure and reliable. Some things I’ve done so far:

  • set up the ability for the system to email me to my email address for alerts (e.g. cron job failures, tripwire reports, etc)
  • installed and configured tripwire to run and send me a report of changed files nightly, can indicate a failing SD card or compromise of the system
  • set up a read-only file system with the cron scripts to deal with the fact that certain parts of the system are read only (e.g. hw-clock) which is probably less useful for a stock openHABian install
  • remove the pi user and create a new user (I don’t like “default” users from hardware)
  • configured to syslog to a remote centralized syslog server (probably not needed in something like openHABian as an option, anyone who knows how to do this can figure out how to do it themselves)
  • configured ssh to only allow login with a cert (not sure how to do this through openHABian because of the chicken and the egg problem)
  • updated sysctl.conf to disable some networking features which are not needed and make the Pi a nice target for network based attacks
  • set up UFW (much easier to use than IPTables) with deny all allow only used ports enabled
  • set up Nginx as a reverse proxy with a LetsEncrypt cert
  • Set up fail2ban on sshd and Nginx with an aggressive and persistence blacklist (these two are probably not necessary if the assumption is all openHABian users will be using my.openhab

Most of the above is scripted in Ansible so it should be relatively easy to transfer it to openHABian.

Some of the things I’m looking into:

  • patching the kernel with GRSecurity and/or Apparmor
  • enabling the hardware watchdog using either systemd or watchdog to reboot the Pi if it ever becomes unresponsive
  • enabling a network watchdog to reboot the networking interface if it ever loses connection to the gateway
  • adding support for systemd software watchdog on my custom services (would require changes to openHAB so probably not an option right now for openHABian), but might be scriptable instead of using one of the watchdog services.

Many if not most of these changes could be done transparently to the end user and scripted as part of the openHABian setup.

So, is there any interest in this sort of thing?

4 Likes

Been reading some of your posts, very interesting and informative. This one is especially good. Being a “noob” and just getting my feet wet with the Pi, linux and openhab, I’m always afraid that I missed something especially when it comes to security. Making a system more secure and resilient with some failsafe such as the watchdog would be welcomed in my book. This may or may not apply, but also some sort of a cron job that would redo the correct permissions on folders would be great, maybe this could be done with a script. In all my efforts, I may have given extra permissions to a folder that should be more restricted. Resetting permission and allowing users to override would be a good addition as well, I would think.

sure! That would be great! :thumbsup:

OH 1.x used to do this every time openHAB started. I don’t know if that behavior is retained in OH 2.

My main goal with the above suggestion is to include some transparent security related settings to make their deployment more secure out of the box without causing extra problems which security often can.

I think it’s a great idea, I hope that you get to collaborate. I’m keeping an eye on this thread!

This one’s already in openHABian, we setup a script that automatically sets up a given username, password and generates a valid certificate using certbot. If you don’t have a usable domain handy, it’ll use a openssl cert until you do. It’s in the latest openhabian-setup

@rlkoshak that’s a great list. After step #1 (email setup) I would also add the following two applications to the list:

  • apticron - stay informed about the list of new package updates (apt upgrade)
  • logwatch - receive a summarized overview on activities in your log

I’m leaning towards No.

Most of what you listed is also part of my setup of publicly available webservers or of a fileserver in a company I help out at. These hardening steps make the system more secure, yes. Do we need all this in a RPi with the sole purpose of serving openHAB in a private network behind a router/NAS/Firewall? (to only mention the normal use case)

Setting up system mail is a tricky thing to get right (already discussed here). Tripwire/logwatch/… are needed if you are paranoid, finding real meaning in the output of these tools is actually not as easy as one might think. I can already hear users complain about problems because of the read-only filesystem or being in the fail2ban blacklist. And so on and so forth.

All mentioned steps are great recommendations for an IT administrator but seem to me like over-complications for openHABian. I am not against a routine in the openhabian-config menu to set up a mail account and install and configure tripwire, logwatch, apticron, fail2ban. I just don’t think it’s needed for the average openHABian user and the few that actually care will have their own preferences.

The username is actually something I chose. I could have used whatever I wanted but why not use the default one everyone knows. Reminds me of “Security by Obfuscation” :slight_smile:

At last I want to encourage everyone interested in increasing the security or transparency of his/her system to try out a few of these recommendations, especially if it is in an insecure environment.

That’s right. But you know what, I still need to write the update posting introducing this feature! :dizzy_face: I’ll do that this weekend, thanks again for contributing this nice addition. :wink:

Which is why I didn’t just start making Feature Issues. However, I would recommend at least the watchdog stuff so the Pi autoreboots when it gets stuck.

But is this the normal use case? Given the number of people I’ve warned off of blindly punching holes in their firewall with no further protections because they are either unaware of or unwilling to use My.openHAB I start to wonder, which is what was the impetus for this original post in the first place.

I am amazed and dismayed by the number of people who take no extra precautions after exposing their system on the Internet with “who would care about little old me?” as their security plan.

I was mainly referring to ssmtp which, as you mention in that thread, is pretty simple. My primary intent was to get informed of failed cron jobs, fail2ban triggers, and tripwire reports.

Indeed but I have already discovered and replaced a failing SD card due to Tripwire reports. It’s not just useful for security. And my thoughts would be that the policy would be set up and finely tuned so that the only thing in the reports would be changes to the system that are highly unlikely to be changed by the average joe user. So nothing in logs or dev or the like but watching some of the other files in /etc /bin /user, etc. And even then script the updates and upgrades to regenerate the TW db when updated. So the only thing they would have in their reports would be system changes they didn’t initiate.

It’s less security by obfuscation and more avoiding all those automated scripts that know that when they see a Raspberry Pi 99.9% of the time there will be a Pi user and to commence the known and default password attacks. Using a non-default user sidesteps that very easily. Obviously it isn’t going to protect against a determined attacker.

It is roughly of the same value as using some other port than 22 for ssh or not broadcasting your wireless SSID. It helps you avoid the automated attack scripts which has value. But you clearly shouldn’t rely on them for your only security. It mainly cuts through a lot of potential noise.

Anyway, since the demand may not be that high to make this part of openHABian, I’ll at least post my current set of Ansible playbooks for those who may want to apply them or at least reverse engineer them for their own systems. It will take a bit to get it packaged up and posted to a github repo. In the meantime:

securitySetup.yml - disable some redirect and networking relays

---
- name: Configure ssh to only allow logins with certs
  lineinfile:
    dest: /etc/ssh/sshd_config
    line: 'PasswordAuthentication no'

- name: Set net.ipv4.conf.all.accept_redirects
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv4.conf.all.accept_redirects = 0'

- name: Set net.ipv6.conf.all.accept_redirects
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv6.conf.all.accept_redirects = 0'

- name: Set net.ipv4.conf.default.rp_filter
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv4.conf.all.rp_filter=1'

- name: Set net.ipv4.conf.all.send_redirects
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv4.conf.all.send_redirects = 0'

- name: Set net.ipv4.conf.all.accept_source_route
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv4.conf.all.accept_source_route = 0'

- name: Set net.ipv6.conf.all.accept_source_route
  lineinfile:
    dest: /etc/sysctl.conf
    line: 'net.ipv6.conf.all.accept_source_route = 0'

ufw.yml - these settings are not for openHAB’s ports since I use these scripts on my other devices.

---
- name: Install ufw
  apt: name=ufw

- name: ufw deny incoming
  ufw:
    direction: incoming
    policy: deny
    proto: any

- name: ufw allow ssh on LAN
  ufw:
    rule: allow
    from_ip: 192.168.1.0/24
    proto: tcp
    to_port: 22

- name: ufw allow ssh on VPN
  ufw:
    rule: allow
    from_ip: 10.8.0.0/24
    proto: tcp
    to_port: 22

- name: ufw allow http
  ufw:
    rule: allow
    from_ip: any
    to_ip: any
    proto: tcp
    to_port: 80

- name: ufw allow https
  ufw:
    rule: allow
    from_ip: any
    to_ip: any
    proto: tcp
    to_port: 443

- name: ufw allow MQTT
  ufw:
    rule: allow
    from_ip: 192.168.1.0/24
    to_ip: 192.168.1.0/24
    proto: any
    to_port: 1883

- name: ufw allow MQTT tls
  ufw:
    rule: allow
    from_ip: 192.168.1.0/24
    to_ip: 192.168.1.0/24
    proto: any
    to_port: 8883

- name: ufw allow gmail for ssmtp
  ufw:
    rule: allow
    from_ip: any
    to_ip: any
    proto: any
    to_port: 587

- name: ufw allow rsyslog
  ufw:
    rule: allow
    from_ip: 192.168.1.0/24
    to_ip: 192.168.1.200
    proto: any
    to_port: 514

- name: Enable ufw
  ufw:
    state: enabled

tripwire.yml - also sets up ssmtp, requires tripwire to already have been installed since I couldn’t figure out how to automate that install. the ssmtp is set up for gmail.com. Setting up the policy is an exercise for the student. I use a pretty default policy.

---
- name: Install ssmtp
  apt:
    name: ssmtp

- name: Remove root
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    regexp: '^root='
    state: absent

- name: Set root email
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'root={{ ext_email }}'

- name: Remove mailhub
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    regexp: '^mailhub='
    state: absent

- name: Set mailhub to gmail
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'mailhub=smtp.gmail.com:465'

- name: Configure rewriteDomain
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'rewriteDomain=gmail.com'

- name: Remove hostname
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    regexp: '^hostname='
    state: absent

- name: Configure hostname
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'hostname=localhost'

- name: Configure From Line Override
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'FromLineOverride=YES'

- name: Configure AuthUser
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'AuthUser={{ ext_email }}'

- name: Configure AuthPass
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'AuthPass={{ ext_email_pass }}'

- name: Enable TLS
  lineinfile:
    dest: /etc/ssmtp/ssmtp.conf
    line: 'UseTLS=YES'

- name: Check for presence of Tripwire
  stat: path=/usr/sbin/tripwire
  register: tripwire_bin

- name: Fail if tripwire isn't present
  fail:
    msg: "You must install tripwire manually"
  when: tripwire_bin.stat.exists == False

- name: See if the DB already exists
  stat: path=/var/lib/tripwire/{{ host_name }}.twd
  register: tripwire_init

- name: Install pip
  apt:
    name: python-pip
  when: tripwire_init.stat.exists == False

- name: Install pexpect
  pip:
    name: pexpect
  when: tripwire_init.stat.exists == False

- name: Copy Tripwire policy file
  copy:
    dest: /etc/tripwire/twpol.txt
    src: twpol.txt
    owner: root
    group: root
    mode: u=rw
  when: tripwire_init.stat.exists == False

- name: Initialize Tripwire Policy
  expect:
    command: /usr/sbin/twadmin -m P /etc/tripwire/twpol.txt
    responses:
      'Please enter your site passphrase:' : "{{ tw_site_pw }}"
  when: tripwire_init.stat.exists == False

- name: Initialize Tripwire Database
  expect:
    command: /usr/sbin/tripwire --init
    timeout: 600
    responses:
      'Please enter your local passphrase:' : "{{ tw_site_pw }}"
  when: tripwire_init.stat.exists == False

- name: Remove Tripwire policy file
  file:
    path: /etc/tripwire/twpol.txt
    state: absent
  when: tripwire_init.stat.exists == False

readonly.yml - move directories that must be rw to a tempfs, set up mounts to mount the rest as ro by default, add some aliases to the default shell to easily see when in rw/ro mode on the shell, update scripts that need to write to things not in a tempfs to remount rw while they are working…

---
- name: Add aliases and fancy prompt to show status of FS
  blockinfile:
    state: present
    dest: /etc/bash.bashrc
    block: |
      # set variable identifying the filesystem you work in (used in the prompt below)
      set_bash_prompt(){
          fs_mode=$(mount | sed -n -e "s/^\/dev\/.* on \/ .*(\(r[w|o]\).*/\1/p")
          PS1='\[\033[01;32m\]\u@\h${fs_mode:+($fs_mode)}\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
      }

      alias ro='sudo mount -o remount,ro / ; sudo mount -o remount,ro /boot'
      alias rw='sudo mount -o remount,rw / ; sudo mount -o remount,rw /boot'

      # setup fancy prompt"
      PROMPT_COMMAND=set_bash_prompt

- name: Set up /tmp, /var/log, and /var/tmp as tempfs
  blockinfile:
    state: present
    dest: /etc/fstab
    insertafter: "#   use  dphys-swapfile swap[on|off]  for that"
    block: |
      tmpfs           /tmp            tmpfs   nosuid,nodev         0       0
      tmpfs           /var/log        tmpfs   nosuid,nodev         0       0
      tmpfs           /var/tmp        tmpfs   nosuid,nodev         0       0

- name: Set permissions on /tmp
  file:
    mode: a+rwx
    path: /tmp
    state: directory

- name:  Mount /tmp
  mount:
    name: /tmp
    src: /tmp
    fstype: tmpfs
    state: mounted

- name: Mount /var/log
  mount:
    name: /var/log
    src: /var/log
    fstype: tmpfs
    state: mounted

- name: Mount /var/tmp
  mount:
    name: /var/tmp
    src: /var/tmp
    fstype: tmpfs
    state: mounted

- name: Remap folders to /tmp
  script: relink.sh

- name: Waiting for {{ inventory_hostname }} to come back from reboot
  local_action: wait_for host={{ inventory_hostname }} state=started delay=30 timeout=300
  become: false

- name: Configure boot command line
  replace:
    dest: /boot/cmdline.txt
    regexp: 'otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait'
    replace: 'otg.lpm_enable=0 console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait fastboot noswap ro'

- name: Move dhcpd lock file to temp
  replace:
    dest: /etc/systemd/system/dhcpcd5
    regexp: '\=/run/dhcpcd.pid'
    replace: '\=/var/run/dhcpcd.pid'

- name: update fake-hwclock cron job to remount / rw and then mount it back
  copy:
    dest: /etc/cron.hourly/fake-hwclock
    src: fake-hwclock

- name: Check for presence of Tripwire cron job
  stat: path=/etc/cron.daily/tripwire
  register: tripwire_cron

- name: Update cron job to remount rw before running check
  copy:
    dest: /etc/cron.daily/tripwire
    src: tripwire-cron
  when: tripwire_cron.stat.exists == True

- name: Update apt daily script to remount rw before running
  copy:
    dest: /etc/cron.daily/apt
    src: apt

- name: Update dpkg daily script to remount rw before running
  copy:
    dest: /etc/cron.daily/dpkg
    src: dpkg

- name: Update logrotate daily script to remount rw before running
  copy:
    dest: /etc/cron.daily/logrotate
    src: logrotate

- name: Update man-db daily script to remount rw before running
  copy:
    dest: /etc/cron.daily/man-db
    src: man-db

- name: Update passwd daily script to remount rw before running
  copy:
    dest: /etc/cron.daily/passwd
    src: passwd

- name: Remove some start scripts
  shell: /sbin/insserv -r bootlogs; /sbin/insserv -r console-setup

- name: Set boot FS as readonly
  replace:
    dest: /etc/fstab
    regexp: '/dev/mmcblk0p1  /boot           vfat    defaults          0       2'
    replace: '/dev/mmcblk0p1  /boot           vfat    defaults,ro          0       2'

- name: Set root as readonly
  replace:
    dest: /etc/fstab
    regexp: '/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1'
    replace: '/dev/mmcblk0p2  /               ext4    defaults,noatime,ro  0       1'

- name: Reboot as read only
  include: tasks/reboot.yml

The above copies over some scripts (apt, dpkg, logrotate, man-db, passwd. In all of these cases the only difference between the defaults is a mount -o remount,rw before the body of the script and a mount -o remount,ro after.

That is all I have automated in Ansible up to this point. The other stuff I’ve been applying to my main server where I actually have OH installed and I’ve not Ansibilized that server yet.

Side note: for those who have a hub and spoke like system like me (i.e. central openHAB server with remote Raspberry Pis which report to/take commands from OH) it makes creating and updating them all very easy.

Obviously I did all of the above before openHABian or even before I realized there was a way to script the build up of the OS in the approach openHABian uses.

1 Like

@rlkoshak I might need to add the above. I am of course not completely against the intent to tighten security. Together with @Benjy we already added nginx based auth+ssl and implemented the change of the default Karaf Console password (in combination with opening up the interfaces).

I chose Raspbians default username+password to make the start for every user as easy as possible. Changing the password is of course something a user should do and can easily by executing raspi-config… but yeah. That’s something we could promote more, maybe by forcing the user to set a password on first ssh login or first openhabian-config execution.

I’ll open an issue for that.

I’ll reopen the issue. I would prefer the steps to be in bash just as the rest.

I never trusted these kinds of measures. If your hardware/os/software ever runs into a problem that can only be solved by rebooting, you should analyze and fix the problem instead. Don’t you agree?

tl;dr, I’d rather the machine be available and working than broken and awaiting analysis and repair. If a reboot can clear the problem, even if only for a few hours, I’d choose that over having the device permanently offline for a bunch of days.

It depends on how it is deployed. For example, I need my garage door opener to run reliably. But it is the garage so the Pi controller is running headless, mounted to a wall, wired to some relays and sensors. In short, if the machine falls off the network for whatever reason my ability to log in and discover what is wrong is almost non-existent. I have to do one of three things:

  • set up a table, bring out a monitor, keyboard, and mouse to plug in and hope to see something
  • unplug all the sensor and actuator wires, unscrew it from the wall, and bring it upstairs to my desk
  • unplug the SD card and plug that card into another Pi to do the analysis, only now it is running on a separate device so hardware problems may not be made manifest.

I’ve done all three before and a watchdog is not going to solve every problem (e.g. I had a botched apt-get update of raspberrypi-kernel recently which hosed one of my devices).

And even it I could get in to diagnose a problem, I’d rather it reboot and start working again in the meantime than lose my garage door controller or door sensors or RFM69 gateway.

The watchdog lets the Pi make a best effort to maintain a high degree of uptime even when errors are occurring. I get alerts when the machines go down so I can use the logs to diagnose the problem after the fact but in the meantime it will be up and available without requiring manual intervention.

Before delving into watchdog and systemd’s watchdog features I had the Pi powered by a zwave plug and when it fell off the network for more than five minutes I would just powercycle the Pi (implemented in OH rules).

So in short, I’d rather have the system running and available than sit there idle and broken awaiting analysis, particularly since accessing and performing that analysis if the network/ssh goes down is a really big deal and not something I can just stop and take care of at any moment.

If it were not for the camera I have hooked up to the Pi, I’d probably replace the whole system with Arduinos or ESP8266 or the like which really are more appropriate for this sort of deployment. But in the meantime the watchdogs plus the RO file systems let my Pis work more like a microcontroller/embedded computer than a typical server.

Hello,

i used on my old OH1 Raspberry a deamon called [monit](https://mmonit.com/monit/documentation/monit.html) for watchdog. Thats something what i miss in my new OpenhabianPi OH2 installation.

1 Like

Hello,

Without speaking about the work that has to be done I think they are good proposals, and thanks @rlkoshak to give us so much ideas to improve our HA security.

Regarding the discussion about where to stop in term of security why not letting the choice for the Openhabian installer. By letting them several proposals depending on their architecture or their requirements in terms of security and adapt the necessary packages following the choice made. For example it could be:
Level 1 security is not managed at pi level, openhab is coupled with myopenhab
Level 2 pi is exposed to internet behind a firewall with the openhab secured port

The only thing that don’t let me move for my house to Openhabian right now are all the tweakings in terms of security that I have made on my pi (and also the emonhub setup but it’s another story).

Nevertheless I will test the great work made in Openhabian for a friend in a few weeks.

1 Like