Memory usage (docker) 2.5.9 pushing 1.5-1.6gb

Yep. But it’s a project in an of itself and not at all light weight. But it’s been worth the effort for me. I discovered one of my RPis was silently rebooting itself every couple of hours without my knowledge. Turned out it needed a better power supply.

It also showed me that about every couple of hours one of my VMs was hitting a sustained load of 8-10 about twice a day. I was able to narrow that problem down to crappy default PostgreSQL settings. Of course this was a case where everything was fine until I installed Zabbix and PostgreSQL started to get pounded a lot more. That helped bring the load down to a sustained 2 with spikes to 4-5 so I got a new RPi 4 and moved a few services (vaultwarden, heimdall, and guacamole) off of the VM to the RPI 4. I also move my virtual desktop to the new RPi 4 and now everything is much happier with loads under 1 with spikes to 2.

It’s still telling me that my HAProxy on my pfSense firewall is restarting sporadically and for no apparent reason which I need to track down. I’ve also some weird inefficiencies on my network

I’m able to set triggers that generate an email (can be just about any kind of notification like Telegram or SMS) so I’m immediately informed when something goes wrong. I have it running on all my VMs and all my RPis. Pretty much everything that is always running.

If it helps you or others, here is the Ansible playbook I wrote to set this up.

---
# tasks file for roles/zabbix

- name: Create the zabbix user and group
  include_role:
    name: create-user
  vars:
    uid: "{{ zabbix_uid }}"
    gid: "{{ zabbix_gid }}"
    user_name: zabbix
    create_home: False
    service: zabbix

- name: Check if docker group exists 
  shell: /usr/bin/getent group | awk -F":" '{print $1}'
  register: etc_groups
  changed_when: False

- name: Add secondary Groups to zabbix user
  user:
    name: zabbix
    groups: docker
    append: yes
  become: true
  when: '"docker" in etc_groups.stdout_lines'

- block:
  - name: Create the folders for logging and settings
    file:
      path: "{{ item }}"
      state: directory
      owner: zabbix
      group: zabbix
      mode: u=rwx,g=rwx,o=rx
      recurse: yes
    loop:
      - "{{ zabbix_home }}"
      - "{{ zabbix_home }}/alertscripts"
      - "{{ zabbix_home }}/externalscripts"
      - "{{ zabbix_home }}/modules"
      - "{{ zabbix_home }}/enc"
      - "{{ zabbix_home }}/ssh_keys"
      - "{{ zabbix_home }}/ssl/certs"
      - "{{ zabbix_home }}/ssl/keys"
      - "{{ zabbix_home }}/ssl/ssl_ca"
      - "{{ zabbix_home }}/snmptraps"
      - "{{ zabbix_home }}/mibs"
      - "{{ zabbix_home }}/export"
    become: true

  # Create database and user
  - name: Install psycopg2
    pip:
      name: psycopg2-binary
    become: True

  - name: Create postgres database for Zabbix
    postgresql_db:
      login_host: "{{ postgresql_host }}"
      login_password: "{{ postgresql_password }}"
      login_user: "{{ postgresql_user }}"
      name: "{{ zabbix_db_name }}"

  - name: Create zabbix user for zabbix database
    postgresql_user:
      db: "{{ zabbix_db_name }}"
      login_host: "{{ postgresql_host }}"
      login_password: "{{ postgresql_password }}"
      login_user: "{{ postgresql_user }}"
      name: "{{ zabbix_db_user }}"
      password: "{{ zabbix_db_password }}"
      priv: ALL

  # Server: config variables https://hub.docker.com/r/zabbix/zabbix-server-pgsql/
  - name: Pull/update the zabbix server docker image
    docker_container:
      detach: True
      env:
        DB_SERVER_HOST: "{{ postgresql_ip }}"
        POSTGRES_USER: "{{ zabbix_db_user }}"
        POSTGRES_PASSWORD: "{{ zabbix_db_password }}"
        POSTGRES_DB: "{{ zabbix_db_name }}"
        ZBX_STARTVMWARECOLLECTORS: "5"
        ZBX_STARTDISCOVERERS: "5"
        ZBX_HISTORYCACHESIZE: "128M"
        ZBX_HISTORYINDEXCACHESIZE: "4M"
        ZBX_HOUSEKEEPINGFREQUENCY: "1"
        ZBX_MAXHOUSEKEEPERDELETE: "5000"
  #      ZBX_LOADMODULE: dummy1.so,dummy2.so # modules are in /var/lib/zabbix/modules
  #      ZBX_DEBUGLEVEL: 3 # 0-5, 5 is TRACE
  #      ZBX_TIMEOUT: 4 # timeout for processing checks
  #      ZBX_JAVAGATEWAY_ENABLE: false
      hostname: "{{ ansible_fqdn }}"
      image: zabbix/zabbix-server-pgsql:latest
      log_driver: "{{ docker_log_driver }}"
      name: zabbix-server
      published_ports:
        - "10051:10051"
      pull: True
      restart: False
      restart_policy: always
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - /etc/timezone:/etc/timezone:ro
        - "{{ zabbix_home }}/alertscripts:/usr/lib/zabbix/alertscripts"
        - "{{ zabbix_home }}/externalscripts:/usr/lib/zabbix/externalscripts"
        - "{{ zabbix_home }}/modules:/var/lib/zabbix/modules"
        - "{{ zabbix_home }}/enc:/var/lib/zabbix/enc"
        - "{{ zabbix_home }}/ssh_keys:/var/lib/zabbix/ssh_keys"
        - "{{ zabbix_home }}/ssl/certs:/var/lib/zabbix/ssl/certs"
        - "{{ zabbix_home }}/ssl/keys:/var/lib/zabbix/ssl/keys"
        - "{{ zabbix_home }}/ssl/ssl_ca:/var/lib/zabbix/ssl/ssl_ca"
        - "{{ zabbix_home }}/snmptraps:/var/lib/zabbix/snmptraps"
        - "{{ zabbix_home }}/mibs:/var/lib/zabbix/mibs"
        - "{{ zabbix_home }}/export:/var/lib/zabbix/export"

  # Web interface
  - name: Pull/update the zabbix web docker image
    docker_container:
      detach: True
      env:
        ZBX_SERVER_HOST: "{{ zabbix_server_ip }}"
        DB_SERVER_HOST: "{{ postgresql_ip }}"
        POSTGRES_USER: "{{ zabbix_db_user }}"
        POSTGRES_PASSWORD: "{{ zabbix_db_password }}"
        POSTGRES_DB: "{{ zabbix_db_name }}"
  #      ZBX_HISTORYSTORAGEURL: elasticsearch config
  #      ZBX_HISTORYSTORAGETYPES: elasticsearch config
        PHP_TZ: America/Denver
        ZBX_SERVER_NAME: Zabbix
      hostname: "{{ ansible_fqdn }}"
      image: zabbix/zabbix-web-nginx-pgsql:latest
      log_driver: "{{ docker_log_driver }}"
      name: zabbix-web
      published_ports:
        - "9090:8080"
      pull: True
      restart: False
      restart_policy: always
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - /etc/timezone:/etc/timezone:ro
  when: zabbix_server

# TODO Make this more dynamic, see if there is a way to look up the latest release number on GitHub or something
- name: Add zabbix repo ubuntu
  apt:
    deb: https://repo.zabbix.com/zabbix/5.4/ubuntu/pool/main/z/zabbix-release/zabbix-release_5.4-1%2Bubuntu20.04_all.deb
  when: (not 'pis' in group_names) and (not 'muninn' in inventory_hostname)
  become: yes

- name: Add zabbix repo rpi
  apt:
    deb: https://repo.zabbix.com/zabbix/5.4/raspbian/pool/main/z/zabbix-release/zabbix-release_5.4-1%2Bdebian10_all.deb
  when: ('pis' in group_names) or ('muninn' in inventory_hostname)
  become: yes

- name : Install the zabbix-agent2
  apt:
    name: zabbix-agent2
    update_cache: yes
  become: yes

- name: Install the config
  template:
    dest: /etc/zabbix/zabbix_agent2.d/custom-zabbix.conf
    mode: u=rw,g=r,o=r
    src: custom-zabbix.j2
    force: true
  become: True
  notify:
    - Restart the zabbix agent to pick up config

# this task is safe because of the validate
- name: Allow zabbix to have passwordless sudo
  lineinfile:
    dest: /etc/sudoers
    state: present
    regexp: '^zabbix'
    line: 'zabbix ALL=(ALL) NOPASSWD: ALL'
    validate: 'visudo -cf %s'
  become: True

- name: Add cron job to create log directory on reboot for RPis
  block:
    - name: Add the cron shell variable
      cron:
        name: SHELL
        env: true
        job: /bin/bash

    - name: Add cron job to create the directory at boot
      cron:
        name: "Zabbix log directory"
        job: mkdir /var/log/zabbix && chown zabbix:zabbix /var/log/zabbix
        special_time: reboot
  become: True
  when: "'pis' in group_names"

This installs the server stuff if zabbix_server is true and the agent stuff if zabbix_server is false so this same role can be used on all the machines. It also needs to add a different apt repo for the RPis as it does for my Ubuntu VMs so zabbix-agent2 can be installed which is required for good Docker integration. Also, on my RPis I put my logs into a RAM disk so I need to create and set the permissions on the directory Zabbix logs to before Zabbix starts which I do with a cron job that runs at boot.

My custom-zabbix.j2 file is:

Hostname={{ ansible_hostname }}
LogFileSize=1024
Server={{ zabbix_conn_str }}
ServerActive={{ zabbix_conn_str }}

The handler is

---
# handlers file for roles/zabbix

- name: Restart the zabbix agent to pick up config
  systemd:
    name: zabbix-agent2
    state: restarted
  become: True

Everything in {{ }} is a variable that will need to be set (except for the ones that start with “ansible” which gets set for you. See Ansible Revisited for more on using Ansible.

Using the role to install the server:

---

# Prerequisites:
# - install ssh
# - set up ssh for certificate only logins
# - install python3
# - all hosts added to the inventory under [homeauto]

# TODO set up root with ssh certs for git interactions

- hosts: homeauto
  vars:
    - update: False
  roles:
    - common
    - { role: vm, when: not update }
    - firemotd
    - mount-data
    - nut-client
    - msmtp
    - fail2ban
    - { role: multitail, openhab_logs: '/srv/openhab/userdata/logs/' }
    - { role: docker, when: not update }
    - role: zabbix
      vars:
        - zabbix_server: True
    - openzwave
    - mosquitto
    - openhab
    - docker-prune
    - tripwire

Example showing using the role to install the agent.

- hosts: media
  vars:
    - update: False
  roles:
    - common
    - { role: vm, when: not update }
    - firemotd
    - mount-data
    - nut-client
    - msmtp
    - fail2ban
    - { role: multitail, openhab_logs: '/srv/openhab2/userdata/logs/' }
    - { role: docker, when not update }
    - gitlab
    - plex
    - { role: calibre, calibre_type: 'docker' }
    - postgresql
    - redis
    - elasticsearch
    - nextcloud
    - { role: zabbix, zabbix_server: False }
    - docker-prune
    - tripwire

Note the above shows two different ways to set variables when including a role into a playbook.