I want to share some experience with which I achieved to get a high performance Openhab plattform leveraging Raspberry. You may are interested into this.
A challenge with Raspberry is the slow microSD. Therefore, i decided to circumvent the SD when ever possible. My solution:
Buy an above average-speed microSD and use it only for booting the system.
Have the rest of the system on a very fast above-average USB stick.
(There may is some potential to save some further money as I am not 100% sure if with my solution also a slow microSD can be used, hence, I wanted to go secure and compared to a Windows server I anyway save an awful lot of money & time that i rather invest in some further automation HW)
Hardware:
Raspberry 3 Model B
Corsair Flash Voyager GTX 2015 - 128GB
Toshiba EXCERIA microSDHC 16GB, 95 MB/s, UHS Class 3
Installation:
On my system I am using the absolute minimum required:
Nettalk and avahi-daemon ** (sudo apt-get install netatalk ; sudo apt-get install avahi-daemon)
That’s it. The installation time is less then an hour, including some configuration. Afterwards you are ready to go.
** required to access with OpenHAB designer the files on your raspberry in a very comfortable manner.
Some security thoughts:
If you go with Raspbian Jessie Lite, than you have already a minimum of SW which from a security perspective is great
Re-configure your SSH leveraging certificates instead of passwords. This is more comfortable and more secure
use a networkscanner and shutdown any not requried service. This has the nice side effect that you make more ressources to the software you really need available
run from time to time sudo apt-get update ; sudo apt-get upgrade ; sudo apt-get clean (I have a cron job, letting the system doing this work for me)
Having the system split on cheap HW you can add another USB stick and dump your active USB stick every night to the 2nd one. This allows you to restore your system within minutes if your usb stick faces an issue. Alternative, use a remote server. I use a NAS that has a unix based OS. Again, very simpel, leverage a cron job and you haven’t any time to invest for back up.
Thats it. Happy to ready some of your experience in how you tuned your Raspberry but also how you improved security.
NOTE, THIS IS MY EXPERIENCE AND THERE IS ABSOLUTELY NO GARANTEE THIS WORKING IN YOUR ENVIRONMENT TOO. IF YOU START TO BUY HARDWARE BECAUSE OF THIS POST, THAN YOU DO THIS ON YOUR ONWN RISK. I WILL PROVIDE NO SUPPORT. THE ABOVE IS EXPERIENCE EXCHANGE ONLY.
Install and configure ufw which is way easier to use and configure than iptables.
sudo apt-get install ufw
# Deny all by default
sudo ufw default deny incoming
# Allow ssh from LAN
sudo ufw allow from 192.168.1.0/24 to any port 22 proto tcp
# Open ports for openHAB and any other service you need access to
# Allow web access for updates
sudo ufw allow from any to any port 80 proto tcp
sudo ufw allow from any to any port 443 proto tcp
Install and configure a Host Intrusion Detection package. I use Tripwire.
sudo apt-get install tripwire
sudo tripwire --init
sudo sh -c 'tripwire --check | grep Filename > checkResults.txt'
# Comment out the entries in checkResults.txt in /etc/tripwire/twpol.txt
# Then comment out:
/var/lock
/var/run
/proc
/etc/rc.boot
# Add under /proc
/proc/devices -> $(Device) ;
/proc/net -> $(Device) ;
/proc/tty -> $(Device) ;
/proc/sys -> $(Device) ;
/proc/cpuinfo -> $(Device) ;
/proc/modules -> $(Device) ;
/proc/filesystems -> $(Device) ;
/proc/interrupts -> $(Device) ;
/proc/ioports -> $(Device) ;
/proc/self -> $(Device) ;
/proc/kmsg -> $(Device) ;
/proc/stat -> $(Device) ;
/proc/loadavg -> $(Device) ;
/proc/uptime -> $(Device) ;
/proc/locks -> $(Device) ;
/proc/meminfo -> $(Device) ;
/proc/misc -> $(Device) ;
# Add under /dev
/dev/pts -> $(Device) ;
# Add under /etc
!/etc/fake-hwclock.data ;
sudo twadmin -m P /etc/tripwire/twpol.txt
sudo tripwire --init
sudo tripwire --check
# Make sure there are no more errors and no violations
# delete the twpol.txt
sudo rm /etc/tripwire/twpol.txt
# To regenerate twpol: sudo sh -c 'twadmin --print-polfile > etc/tripwire/twpol.txt'
# To incrementally update the DB: sudo tripwire --check --interactive
# To blindly accept all the changes: sudo tripwire --update -a
- configure tripwire to send daily reports (replace stuff in < > with appropriate values for your setup
sudo apt-get install ssmtp
# Edit /etc/ssmtp/ssmtp.conf to look like this:
root=<youremail>@gmail.com
AuthUser=email
AuthPass=<gmail app pass as set up in Google Accounts>
mailhub=smtp.gmail.com:587
UseTLS=YES
UseSTARTTLS=YES
rewriteDomain=gmail.com
hostname=localhost
FromLineOverride=YES
# Edit /etc/ssmtp/revaliases to contain:
root:<myaddress>@gmail.com:smtp.gmail.com:587
root@<hostname>:<myaddress>@gmail.com:smtp.gmail.com:587
<user>:<myaddress>@gmail.com:smtp.gmail.com:587
<user>@<hostname>:<myaddress>@gmail.com:smtp.gmail.com:587
# Test with: echo -e "Subject: `uname -n` test \n\n This is a test" | /usr/sbin/ssmtp -s <youremail>@gmail.com
# Add a cron job to run daily
sudo crontab -e
55 11 * * * bash -c `echo -e "Subject: Tripwire report for `uname -n` \n\n `/usr/sbin/tripwire --check`"' | /usr/sbin/ssmtp <youremail>@gmail.com >> /dev/null 2>&1
Where possible, run your HA devices on a segregated network. This can mean setting up a vlan or a separate guest network in your router and putting your HA devices on one and your more sensitive computing on an another. You can bridge between the two using an always on VPN or putting one of your devices on both networks (the OH server perhaps). Allow your HA devices to reach out to the Internet only through a proxy running on your bridge server.
On a related note, where possible use TLS, ssh tunnels, or VPN to communicate between your devices (e.g. set up Mosquitto to require TLS) to encrypt your network traffic, particularly if they are on wifi.
I admit I’ve not yet done 6. And there are other things you can do to secure your HA device but these are the ones I’m currently working on.
The Raspberry PI has USB 2.0 interfaces which have a practical maximum throughput of about 35 MB/s for the USB stick. From what I’ve seen, the SD Card throughput is roughly the same with a UHS card.
However, I’d be interested in knowing some specifics about the performance improvements you saw and in what contexts (what was using the disk, how frequently, etc.).
USB 2.0 (in my opinion) is indeed an issue with Raspberry. I understand that they do not move to USB 3 because of power consumption. Pity there is no USB 3 Raspberry available.
Regarding performance, I just can tell you that without the measures above, the performance of my Raspberry was for Charts very slow. Looking to the system I saw that Memory was nearly used to 100% so the CPU. You could say well, thats using the HW efficient, but it made it slow for charts. I observed, that both, the OpenHAB Java process as well as the mysql process did take all ressources. I talked to a electronic semiconductor and he idicated this may has to do with the threats waiting for each other because of the mySQL DB cannot read fast enough. I am really not a HW specialist, but what I can say is that the above turned into an amazing performance improvement. On the CPU / Memory side, both are bored and rarely go above 50% now.
Btw. I going to add to my original posting, that this is my experience and there is NO GARANTEE this working in your environment too.
Thanks for the additional information. It does seem quite odd though for several reasons. Assuming by “Memory” you mean RAM, that would typically not be affected by using a USB stick vs an SD card for persistent storage. I suppose one possibility is that you allocated more swap space for the larger capacity USB stick and that allowed the OS to manage the RAM usage better.
If threads were blocking because of disk contention, you’d see relatively low CPU utilization rather than high. The system would seem sluggish because the threads would not be running code most of the time.
I’m wondering if there might have been some other OH configuration changes that were done at the same time as the switch to a USB stick. I’ve run openHAB on both an RPi 2B and RPi 3B and haven’t had the RAM or CPU issues you describe. CPU usage, for example, is almost always 9-10%.
My sitemap has almost 70 charts. However, I use rrd4j instead of MySQL for the data storage. When I’ve run MySQL on the Pi in the past I have also noticed that it consumes a significant amount of system resources.
Don’t misunderstand me. I’m not questioning the performance improvements per se, but they don’t seem like they can be explained by the switch to the USB stick.
What you write makes a lot of sense. I however cannot explain this nor agree or disagree. I would not call myself Hardware specialist. Most important to me, with the above performance increased considerable.
Also, have invested some time and I observed 2 elements of system changes of which I think they did have no considerable impact. Tell me if I am wrong with this:
old:
OpenHAB 8.1.1
oracle-java7-jdk - Java™ Platform, Standard Edition 7 Development Kit
new
OpenHAB 8.1.2
oracle-java8-jdk - Java?? Platform, Standard Edition 8 Development Kit
Based on your above I had another idea which i did test. I moved the mysql db to an old raspi 2b. Now running openhab on the fast rapi 3 and mysql on the old raspi 2. Experience so far is positive
Again, i would not call myself hardware or software specialist, nor specialist in how the two work together. So i checked with an electronic semiconductor again. He said, it is likely, that due to the lower performance the previous solution potentially turned into operations that wait for further mySQL queries / data, which than potentially turned into openhab/java using a lot of memory to store up until the full data is available. Since in the same time mySQL does need a lot of memory, they are in conflict. This to me somehow makes sense… The other thing, there seem to be a difference in performance between SD cards and SD cards. The same applies for the USB sticks. As a result, leveraging the more powerful USB sticks and SDHC cards could indeed make a difference. Unfortunately i do not have the old SD card anymore, but I remember, it was one of the cheaper. The Toshiba is different, also its Class 3. Maybe this made also some impact…
