Move tmp and cache folders to tmpfs a good idea?

KevinI · December 30, 2017, 11:48am

Hi,

Just wondering if moving the following folders to tmpfs is a good idea, my thinking is they would get cleared on a Pi3 reboot and also further reduce IO from the SD card.

/var/lib/openhab2/tmp
/var/lib/openhab2/cache

Just wondering if anybody else has done this and if there’s a downside apart from slower restart of OH after the reboot when reloading the cache.

Kevin

travellingkiwi · December 30, 2017, 9:29pm

Cracking idea…

I just tried it… A 200MB cache and 100MB tmp both on temps. Start-up wasn’t any slower than normal really (You can probably attribute that to the vastly increased speed of the tmpfs vs the ext4fs)

KevinI · January 1, 2018, 2:39pm

Hi,

Well at least that’s two of us trying this out and I guess we will see if there are any downsides of which I have had none so far.

Happy New Year
Kevin

travellingkiwi · January 3, 2018, 6:28pm

There is one issue on an upgrade, currently it looks like one of the scripts tries to remove /var/lib/openhabd2/cache (And probably /var/lib/openhab2/tmp). And can’t because it’s a mounted filesystem.

e.g.

root@piman-202:/home/hamish# apt-get install openhab2=2.3.0~20180102110829-1
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  fonts-dejavu-extra libasyncns0 libatk-wrapper-java libatk-wrapper-java-jni libdrm-amdgpu1 libdrm-freedreno1 libdrm-nouveau2 libdrm-radeon1 libelf1 libflac8 libgconf2-4 libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libgnome2-0 libgtk2.0-0 libgtk2.0-bin libgtk2.0-common libice-dev libllvm3.9
  libnspr4 libnss3 libpthread-stubs0-dev libpulse0 libsm-dev libsndfile1 libtxc-dxtn-s2tc0 libvorbisenc2 libx11-dev libx11-doc libx11-xcb1 libxau-dev libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-sync1 libxcb1-dev libxdmcp-dev libxshmfence1 libxt-dev libxxf86vm1 rlwrap
  tzdata-java x11proto-core-dev x11proto-input-dev x11proto-kb-dev xorg-sgml-doctools xtrans-dev
Use 'apt-get autoremove' to remove them.
The following packages will be upgraded:
  openhab2
1 upgraded, 0 newly installed, 0 to remove and 2 not upgraded.
Need to get 68.6 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 https://openhab.jfrog.io/openhab/openhab-linuxpkg/ unstable/main openhab2 all 2.3.0~20180102110829-1 [68.6 MB]
Fetched 68.6 MB in 1min 46s (641 kB/s)
(Reading database ... 79456 files and directories currently installed.)
Preparing to unpack .../openhab2_2.3.0~20180102110829-1_all.deb ...
rm: cannot remove ‘/var/lib/openhab2/cache’: Device or resource busy
dpkg: error processing archive /var/cache/apt/archives/openhab2_2.3.0~20180102110829-1_all.deb (--unpack):
 subprocess new pre-installation script returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/openhab2_2.3.0~20180102110829-1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)
root@piman-202:/home/hamish#

The work-around at present is the pretty basic stop, remove the filesystem, then install, mount the filesystems and start

KevinI · January 4, 2018, 7:35am

Hi,

Thanks for the heads up, as I only have a single system running the house I tend to stay on the stable version so this wouldnt have shown up until the next release, as you mention it’s not a show stopper as the original cache and tmp folders still exist they are just hidden by the tmpfs mounts so a simple process for updates would be.

Stop openhab
unmount cache/tmp tmpfs
Patch openhab
Remount cache/tmp
Restart openhab

I bet I forget to do this next time but it’s not a big issue.

Just for further I/O reduction information ive also done the following, some based on other posts.

Moved /var/log/openhab2 to tmpfs as well - I dont really care if old logs get removed after a reboot as it’s only the current state thats of interest to me.
Replaced rsyslog with busybox-syslogd - logs no-longer written to a file just stored in memory, again Im not too worried about historical data, this has only proven to be an issue if PI reboots for no reason as theres no logs to go back to - could send logs to s central server but I really dont care to much.
Set “vm.swappiness = 1” to reduce swaping to a minimum, not wanting to turn this off completely as openhab is still a bit or a memory hogger and it’s a constant upwards trend, going to start to capture this info for the openhab process so I can track the usage.
Backups of everything to a NAS via rsync scripts and also use “rpi-colne” to keep a copy on a new SD card.

Kevin

kintrupf · February 16, 2018, 5:21pm

I cobbled together this script from bits and pieces from this forum. It will relocate the log-, cache- and tmp-folders to a tmpfs. During a proper reboot the contents of these folders is preserved by copying them back and fourth to their original location.

This is for my Synology Diskstation 718+, but I see no reason why it should not work for other Linux systems.

#!/bin/sh
# On startup create a tmpfs and link some openHAB2 folders to it.
# This is to allow the DS to power down the hard disks.
# Cobbled together from bits and pieces from community.openhab.org - thanks!
# The contents of the log- and temp folders is copied back to their original
# location on stop and back to the tmpfs on start, so the contents is preserved
# during a proper reboot.
#
# For execution during system start and system shutdown put it into the following folder: 
# /usr/local/etc/rc.d/
# When executing it manually make sure openHAB is NOT running!

# Original openHAB2 folder
OPENHAB_ROOT="/volume1/@appstore/openHAB"

# where the tmpfs should be created
TMPFS="/volume1/SmartHome/openHAB/tmpfs"

# Size of tmpfs
TMPSIZE="200M"

# Must be subfolders of $OPENHAB_ROOT
FOLDERS_FOR_TMPFS="userdata/persistence/rrd4j userdata/logs userdata/tmp userdata/cache"

LOG_NAME=$(basename $0)".log"
LOG_FILE=/tmp/$LOG_NAME
LOG_PREFIX=$(date +%Y-%m-%d' '%T)": "

# Copy old logfile to temp folder
if [ -f "$OPENHAB_ROOT/userdata/logs/$LOG_NAME" ]; then
  cp -p "$OPENHAB_ROOT/userdata/logs/$LOG_NAME" -t /tmp
fi

echo $LOG_PREFIX"-----------------------------------------------------------------" >> $LOG_FILE
echo $LOG_PREFIX"$0 called with "$1  >> $LOG_FILE

case $1 in
start)
  # create a tmpfs 
  if $(mount | grep "none on $TMPFS type tmpfs" >> /dev/null); then
    echo $LOG_PREFIX"start: $TMPFS is mounted => start unmounting $TMPFS" >> $LOG_FILE
    umount $TMPFS
  fi
  if [ -d "$TMPFS" ]; then 
    echo $LOG_PREFIX"start: directory $TMPFS exists => removing" >> $LOG_FILE
    rm -rf $TMPFS
  fi
  mkdir -p $TMPFS
  mount -t tmpfs -o size=$TMPSIZE none $TMPFS 
  chmod 777 $TMPFS
  chown -hR openhab:users $TMPFS

  # for each folder in $FOLDERS_FOR_TMPFS: delete in $OPENHAB_ROOT, create in $TMPFS, create symbolic links  
  for f in $FOLDERS_FOR_TMPFS
  do
    echo $LOG_PREFIX"start: processing $OPENHAB_ROOT/$f" >> $LOG_FILE
    if [ ! -d "$TMPFS/$f" ]; then
      echo $LOG_PREFIX"start: creating $TMPFS/$f" >> $LOG_FILE
      mkdir -p $TMPFS/$f
    fi

    if [ -d "$OPENHAB_ROOT/$f" ]; then 
      echo $LOG_PREFIX"start: copy contents from $OPENHAB_ROOT/$f to $TMPFS/$f" >> $LOG_FILE
      cp -pr $OPENHAB_ROOT/$f/. -t $TMPFS/$f
      echo $LOG_PREFIX"start: deleting $OPENHAB_ROOT/$f" >> $LOG_FILE
      rm -rf $OPENHAB_ROOT/$f
    fi

    echo $LOG_PREFIX"start: creating $OPENHAB_ROOT/$f" >> $LOG_FILE
    mkdir -p $OPENHAB_ROOT/$f
		
    echo $LOG_PREFIX"start: linking $TMPFS/$f to $OPENHAB_ROOT/$f" >> $LOG_FILE
    mount --bind $TMPFS/$f $OPENHAB_ROOT/$f 

    chmod 777 $OPENHAB_ROOT/$f
    chown -hR openhab:users $OPENHAB_ROOT/$f
    chmod 777 $TMPFS/$f
    chown -hR openhab:users $TMPFS/$f
  done

  # For ZWAVE USB stick:
  chown -R root.uucp /run/lock
  chmod -R g+w /run/lock
  chown -R root.uucp /var/lock
  chmod -R g+w /var/lock  
  chown openhab.dialout /dev/ttyA*
  ls -l /dev/ttyA* >> $LOG_FILE

  ;;

stop)
  for f in $FOLDERS_FOR_TMPFS
  do
    if $(mount | grep "none on $OPENHAB_ROOT/$f type tmpfs" > /dev/null); then
      echo $LOG_PREFIX"stop: $OPENHAB_ROOT/$f is mounted => start unmounting $OPENHAB_ROOT/$f" >> $LOG_FILE
      umount $OPENHAB_ROOT/$f
      rm -rf $OPENHAB_ROOT/$f
      mkdir -p $OPENHAB_ROOT/$f
      echo $LOG_PREFIX"stop: copy contents from $TMPFS/$f to $OPENHAB_ROOT/$f" >> $LOG_FILE; 
      cp -pr $TMPFS/$f/. -t $OPENHAB_ROOT/$f
    fi
  done

  if $(mount | grep "none on $TMPFS type tmpfs" > /dev/null); then
    echo $LOG_PREFIX"stop: $TMPFS is mounted => start unmounting $TMPFS" >> $LOG_FILE
    umount $TMPFS
  fi
  if [ -d "$TMPFS" ]; then 
    echo $LOG_PREFIX"stop: directory $TMPFS exists => removing" >> $LOG_FILE
    rm -rf $TMPFS
  fi
  ;;

*)
  echo "Usage: $0 [start|stop]"
  ;;
esac


# Copy log file back
if [ -d "$OPENHAB_ROOT/userdata/logs" ]; then
  cp -p "/tmp/$LOG_NAME" -t "$OPENHAB_ROOT/userdata/logs"
fi