I wish I could’ve put more context in the title, because what I’m most perplexed with is not the actual crashesbut why the fix I enact works (cause I think that’s my best bet at fixing the problem).
With far more frequency than I’d prefer, I get a notification on my phone that OpenHAB (ver 1.8.1) is down. When I get back to the Raspberry Pi (ver 2 B+) its running on, both the red and green lights are on. From what I’ve read this indicates a bad bootcode.bin file, however, what that means, I don’t really know. I’m looking into this, but what’s odd is the fix. Pulling the plug doesn’t work, it just doesn’t boot up. What HAS worked several times now, is pulling the sd card, putting it into a DIFFERENT Raspberry Pi and booting it up. Then putting the card BACK into the Original Raspberry Pi running Openhab and it boots back up!
Full disclosure: this is a problem I’ve dealt with in the past, detailed here. I thought I’d licked it by installing the oracle version of Java and enabling tmp.mount to lower I/O load. And that DOES seem to have increased reliability, but these crashes are still happening.
Holy cow, I think I had the same problem! I’ve not had time to look into it but I had my Raspberry Pi 3 up and die on me. When I finally got around to rebuild the SD card it booted in the other Pi I stuck it into next to my monitor. I put it back in the original and I was good.
I currently have a Pi 1 that is inexplicably dead. I’ll try that fix when I get home and see if it works.
It’s odd, I’ve run without error or problems for years prior to this.
I don’t think I’m seeing any frequent crashes of anything on these Pis. I’m not running OH on any of them, just some Python scripts and ffmjpeg on one of them (the Pi 3).
I am running OH on a PI2 for about a year non stop without any creashes or issues.
Please make sure you do not over clock the pi2, since it does not work very well at all.
Also mount /etc and /log as a tmpfs (but make sure to make regular backups).
Same, I have 3-4 running around the house and none of them exhibit anything but 100% reliability. Granted they are all under much less use than this and have less peripherals attached to them (this one has a shield with IR blasters and a 433mhz antenna), but still it is only this one that consistently/inexplicably crashes.
Yes, in the previous thread I posted that @rikoshak chimed in on, undoing my overclocking was one of the changes I made. I think it helped but I chalk up my increased reliability more to enabling that tmp.mount command w/ systemctl. But I don’t think I’ve mounted /etc and /log as tempfs, how would I go about doing that?
I found various threads on the RPi forum basically saying a bad bootcode.bin is most likely caused by bad imaging (here that links to here) but I’ve tried multiple SD cards and seems awfully coincidental for 2 of us to be experiencing similar issues with this ONLY on RPi’s running openhab. I think the dismissiveness of the posters on the RPi forums are that this is usually a more novice problem (not that I’m an expert but I feel relatively confident I’ve moved past the imaging stage of my RPi advetures ) Maybe I’ll post there as well…
@Spaceman_Spiff I looked into mounting various other folders to tmpfs and it seems pretty straightforward, but you suggest /etc and that is where my configurations reside. Logs make perfect sense to me, but etc is where my configurations reside, wouldn’t I want that on something more permanent?
I would think that code and configuration should be fine on an SD card, but logs and persistence should be on some other volume(s). I fried a microSD card with a MySQL server pretty quickly!
I’m pretty sure this isn’t the cause of my crashes as my latest Pi that is failing to boot is running with a read only configuration. There are like three files that get written to on the SD card under normal operation with everything else running on a tempfs.
Overclocking and marginal microSD card would be the big two, but other possibilities include the Raspberry Pi itself being marginal, electromagnetic interference, extreme operating temperatures. I had a RPi2 in an attic booting from a portable mechanical USB disk and it would lose the filesystem randomly. Now that it’s in comfortable room temperature all the time, it seems to be reliable now.
Thanks @watou! Unfortunately, I don’t think any of those excellent suggestions are my issue.
My MicroSD is a Sandisk (which someone on a previous thread was ragging on, but in my subsequent research seems like a highly reliable brand…as I previously thought) and I’ve also tried multiple different kinds.
I did have it overclocked at one point but have since removed that (which I do believe helped).
And its operating with a shield but no case (for now) on my media center in my living room, so I don’t think temperature is an issue.
But maybe you’re last suggestion is the issue. It’s currently sitting between a HTPC and a Drobo (which has been off for the entire existence of the RPi), but maybe it only crashes when there is a power surge in my house or something? I have a lot of my components connected to a UPS but not the RPi. And every time I print something, or sometimes when my Main PC (which is nearby and on the same UPS/Outlet) comes to life, the UPS makes that loud clicking sound it does when it gets a surge. It happens so infrequently, and often when I’m away from the home, that maybe its happening on a power event.
If this is the answer, its disappointing, cause I’m not sure how to fix it. I was really hoping it could be some kind of observable/reproduceable issue, and still hoping its something specific with the OpenHAB installation, so that I can fix it:). If you think of anything else, please pass it on
Either or both of these could be my problem. This one is an original gen Pi with almost all of the GPIO pins wired to reed and vibration sensors around the house and it is located in the utility room next to the gas furnace. Not an ideal operating environment. The power might be marginal as well. At least I’m not overclocking.
Just for completeness it turned out my problem Pi wasn’t caused by any of the above. Instead it was caused by a botched upgrade or raspberry-kernel (I run my updates through Ansible and it appears that particular package took too long to deploy and Ansible gave up on it before the upgrade could complete). This caused the networking to get hosed.
Since this Pi is in a closet, headless, and far way from the nearest monitor yet wired up to the house I just thought it was dead. After unplugging everything and moving it to a monitor I could see it actually did boot, it just couldn’t bring up wlan0.
ok sorry but now I’m confused. Aren’t you recommending mounting the persistence folder to tmpfs because it makes so many writes to the SD? Maybe I’ve confused the folder hierarchy, but I thought the persistence folder was inside the etc folder you suggested I mount, and with your statement about the frequency of writes I thought that was the reason you suggested it…?
@rlkoshak very happy to hear you worked out your issue!
He recommends putting the persistence folder (/var/lib/openhab/persistence in openHAB 1 or /var/lib/openhab2/userdata/persistence in openHAB 2 I think) in a tempfs with a periodic cron job to write the contents of it back to the SD card. This limits the amount of writing that gets done on the SD card (i.e. once an hour rather than hundreds a minute) but it does pose the risk of losing data (up to an hour’s worth) upon a Pi restart.
@rlkoshak ok it is how I suspected. Now that I re-read @Spaceman_Spiff comment, I think the issue was my wording. I should’ve but the “JUST” after the word “mount”. I was trying to ask why I shouldn’t mount JUST the persistence folder and not the entirety of etc. And was when I (incorrectly) remembered the persistence folder as being inside etc. SO really it sounds like the suggestion should be to mount /etc, /log and /var/lib/openhab/persistence to tmpfs and create a script to back them up to SD card regularly.
@mckar Thanks so much for that suggestion. Any further color on how best to accomplish this? Is the flash on a USB stick that much more reliable?
) Maybe I’ll post there as well…