Just for completeness it turned out my problem Pi wasn’t caused by any of the above. Instead it was caused by a botched upgrade or raspberry-kernel (I run my updates through Ansible and it appears that particular package took too long to deploy and Ansible gave up on it before the upgrade could complete). This caused the networking to get hosed.
Since this Pi is in a closet, headless, and far way from the nearest monitor yet wired up to the house I just thought it was dead. After unplugging everything and moving it to a monitor I could see it actually did boot, it just couldn’t bring up wlan0.
ok sorry but now I’m confused. Aren’t you recommending mounting the persistence folder to tmpfs because it makes so many writes to the SD? Maybe I’ve confused the folder hierarchy, but I thought the persistence folder was inside the etc folder you suggested I mount, and with your statement about the frequency of writes I thought that was the reason you suggested it…?
@rlkoshak very happy to hear you worked out your issue!
He recommends putting the persistence folder (/var/lib/openhab/persistence in openHAB 1 or /var/lib/openhab2/userdata/persistence in openHAB 2 I think) in a tempfs with a periodic cron job to write the contents of it back to the SD card. This limits the amount of writing that gets done on the SD card (i.e. once an hour rather than hundreds a minute) but it does pose the risk of losing data (up to an hour’s worth) upon a Pi restart.
@rlkoshak ok it is how I suspected. Now that I re-read @Spaceman_Spiff comment, I think the issue was my wording. I should’ve but the “JUST” after the word “mount”. I was trying to ask why I shouldn’t mount JUST the persistence folder and not the entirety of etc. And was when I (incorrectly) remembered the persistence folder as being inside etc. SO really it sounds like the suggestion should be to mount /etc, /log and /var/lib/openhab/persistence to tmpfs and create a script to back them up to SD card regularly.
@mckar Thanks so much for that suggestion. Any further color on how best to accomplish this? Is the flash on a USB stick that much more reliable?
Not /etc/ Nothing really changes much in /etc unless you physically do it yourself. On a Pi there are a couple of exceptions (e.g. fakehardwareclock gets written to once an hour by a cron job, resolv.conf gets written to once a day, etc) but for the most part etc is pretty static by design. It is intended to be configuration data, not runtime data.
I’ve posted my Ansible playbook I use to set up my Pis as readonly here.
If you use Ansible you should be able to use this as is. If not you should be able to figure out the steps to make it work.
This works fine, too (and is how I did it)!
Because I use rrd4j and mapdb so there are multiple folders in /etc.
With mounting the complete folder I do not need to mount the sub folders individually.
Sooo I know it’s been long time but I FINALLY got around to implementing this and realized I didn’t 100% understand how to initially set these folders in Openhab. Is it a change to openhab.in.sh in /usr/share/openhab/bin/ that I need?
I think i can figure out the cron job that copies it to the SD card once an hour…
Edit: found this https://www.howtoforge.com/storing-files-directories-in-memory-with-tmpfs and used this: sudo mount -t tmpfs -o size=100M,mode=0755 tmpfs /home/seth/openhab/configurations/persistence/
and sudo mount -t tmpfs -o size=100M,mode=0755 tmpfs /var/log/openhab
to mount the files (I think this is right?) and added the following lines to fstab: tmpfs /home/seth/openhab/configurations/persistence/ tmpfs size=100M,mode=0755 0 0 tmpfs /var/log/openhab tmpfs size=100M,mode=0755 0 0
So now I just need to cron/copy the folders back once an hour…I thought this would be the easy part…but I’m not so sure how the mount command works. It feels like it creates like a symlink from that folder location to the tmpfs on ram. But if that’s true, do I just copy it once an over over the same folder path as it was before? or…?
Well I’m not so sure that worked the way I wanted it too. After those commands, my /var/log/openhab and ~/configurations/persistence are now empty as well as /dev/shm/ sooo I don’t think I quite have it yet…
On shutdown you have to copy the files to the sd card and on startup you have to restore them from the sd-card to tmpfs. There is a script in the pi-forums which does exactly that (google “varlog”) or I can post the script that I am using.
@Spaceman_Spiff first off, thanks so much for responding despite my long delay. I would love to see your example script, but I tried hard to find this script in the pi-forums but to no avail (sorry I really didn’t want to have to come back here for something as easy as googling ). I tried both the official RPi forums and the /hardware/server/ sections of this forum but nothing was immediately obvious.
Oh wow, thanks so much! That is more involved than I was anticipating. I’ll comb through this to try to understand it before implementation, then report back. Thanks again
Basicly it just does this file copy thing as I have described earlier.
It mounts the folders as tmpfs without the fstab and then copies the saved files to the new mount.
On shutdown it saves all files on the sd card.
Additionally (I think) I run a cron job once a day with “/usr/local/bin/openhab stop” to persist my data on my sdcard.
Edit6: I apologize. My post had degenerated into babbling nonsense. Short story: been hacking at this all day learning (and relearning) lots about a bunch of stuff that I’m sure a more experienced user just breezes through, but its all very useful.
I still can’t quite get it to work though. The services start w systemctl but varlog gives an error /bin/cp: cannot stat '/var/log_save/*': No such file or directory even though I do have that folder. One question is that it contains very different content from /var/log/. Do I need to do an initial copy over? Or is this maybe an issue with the asterix
I think i discovered that you probably did a manual install and I an apt-get one, so our folder structure is slightly different (I gather this from the /opt/Openhab folder locations). Let me know if that’s incorrect, but if not I assume I replace many of the references with my specific openhab installation’s folders. If I have an apt-get installation and my logs are located in /var/log/, that would already be included in the varlog script, right? So is it correct that I wouldn’t need the first 2 lines of this script?
You have to call /usr/local/bin/openhab stop the first time you want to use the script.
I am running a manual installation, so additionally you have to replace the folder names in /usr/local/bin/openhab.
Please leave the varlog untouched.
I have mounted the openhab-log folder via the fstab-file.
@Spaceman_Spiff can you elaborate on what exactly you added to the fstab file? is it tmpfs /var/log/openhab tmpfs size=70M,mode=0755 0 0 or something like that to /etc/fstab?
That last line confuses me because it was my understanding that the varlog script moved ALL of /var/log (in which openhab resides on my installation) to tmpfs with copies back and forth at boot, shutdown (and eventually regular intervals w cron).
Which actually leads me to another clarification that may be related. My current /var/log_save/ folder contains the following:
And obviously the contents are vastly different in /var/log/ (not least of which, /var/log includes a openhab folder). If I enable varlog.service and reboot my system, I am able to SSH in, but openHAB doesn’t open on my computer or the android app. And the /var/log/ folder will contain the above hierarchy that used to be in /var/log_save/ (and still is, both log/ and log_save/ contain that)
On a more positive note. I have a /home/openhab folder, which isn’t the real home of user openhab, /var/lib/openhab still is, /home/openhab just symlinks to the 4 main folders of openhab (in /var/log, /etc, /var/lib and usr/share/). In the openhabmount script (that’s what I renamed it) I have 3 “entries” for /var/lib/workspace/, /var/lib/persistance/, and /usr/share/webapps/ that all save to a respective folder in /home/openhab/openhab_save. When enabling openhabmount.service those folders in ~/openhab_save seem to be populated by the contents of their respective folders. So I’m obviously very close…I think there is maybe something small but pertinent about tmpfs shares that I don’t quite grasp…
edit: had a thought: do I need to call /usr/local/bin/varlog stop to initialize the process as well as the openhab one? I’m a little hesitant to try without confirm, because of the contents already in /var/log_save/…
Yes - of course. You have to call stop for both scripts. Sorry that I forgot to mention that. :S
I have to mount the openhab/logs/ because I do not use the apt-get installation.
ahhh, no my fault, I should’ve figured that out. Just was a little nervous trying because there was already stuff in /var/log_save, and you specifically didn’t mention it (I clearly hold your opinion in the highest esteem )
I SEEM to be up and running. I have the 4 shares the scripts set up show up when I type mount (for /var/log/ /var/lib/openhab/{persistence,workspace} and /usr/share/openhab/webapps), and the corresponding save folder seems to be populated by the hierarchy I’d expect… So thank you so much for all your help @Spaceman_Spiff!!!
2 final questions if you don’t mind?
@SS i know your answer to this, but just curious if anyone else has any other folders to recommend adding to tmpfs, both inside openhab and just linux in general
You mentioned putting /usr/local/bin/openhab stop into cron… which I will do, but I’m curious if I’d also have to have a cron entry to restart it? Or will the service continue to run via systemd (and the stop command, just copies the tmpfs folder over to the save location)?
). I tried both the official RPi forums and the /hardware/server/ sections of this forum but nothing was immediately obvious.
)