OS: Raspbian GNU/Linux 11 (bullseye), Kernel = Linux 5.15.61-v7l+
openHAB version: 3.3.0
I am having the problem that my the my backup sd-cards are not booting. This is the case for the backup that I made using option 53 (for periodic backup) and 54 (for the one-time raw-copy backup). Curiously, for both cases it worked the first time.
This is the order of events:
I setup the periodic backup (i.e. Amanda+system-backup on the same sd-card) during installation of OpenHABian on my RPi a few months back.
I tested the backup SD-card on the next day and it booted and worked as expected. I then switched back to the main card and again put the backup card in the USB reader to continue creating periodic backups.
Since then I had OpenHABian running without testing the backup SD-card.
I recently upgraded to OpenHAB 3.3 (~2 week ago) and tested the backup SD-card and it did not boot anymore.
To continue with my update, I switched to another SD-card for the backup and created the backup manually on that using the raw-copy option 54 in openhabian-config. I tested, if I can boot from that backup SD-card and it booted as expected. Then I again switched back to the main card and successfully updated it to OpenHAB 3.3.
Before making further modifications to OpenHAB I again ran option 54 to backup the current state. Testing again, if I can boot the backup SD-card, I found that this time the SD-card did not booting anymore. Since then, I have run option 54 twice (with intermitten reboots) and it continues not to work.
So I have been trying to figure out what the problem is:
This forum post descibes that the PARTUUID in /boot/cmdline.txt can end up not being correct:
I found this behavior in for my backup card. I tried fixing it as discussed in the post by correcting the PARTUUID as reported by sudo blkid as shown below, but it did not help:
blkid output with the backup sd-card in the USB reader of the RPi (as /dev/sda):
Corrected /mnt/cmdline.txt (which still does not boot):
Note: The forum post states that the PARTUUID of the /boot partition should be added to /mnt/cmdline.txt. But checking the /boot/cmdline.txt of my (functioning) main sd-card, I found that it actually contains the PARTUUID of the root partition (/). Anyway… I tried both PARTUUIDs in the cmdline.txt of the backup sd-card and neither worked …
I also checked that there is no file /etc/systemd/system/storage.mount (as suggested here here). I found none.
I reran option 54 with debugmode=maximum and recorded the terminal output like so:
sudo openhabian-config 2>&1 | tee /home/openhabian/01__debug/20220919_not_booting_from_sdcard_mirror/20220929_sdcard_backup.log
I have found that during the backup, I sometimes the following error in the logs:
Taking a raw partition copy, be prepared this may take long such as 20-30 minutes for a 16 GB SD card
FAILED (set random UUID)
mount: /storage/syncmount: wrong fs type, bad option, bad superblock on /dev/sda2, missing codepage or helper program, or other error.
sed: can't read /storage/syncmount/etc/fstab: No such file or directory
umount: /storage/syncmount: not mounted.
OK (dirty bit on fsck /dev/sda2 is normal)
When this happens, I am not able to boot from the backup micro-SD card.
However, I am still able to mount the created partitions after executing the backup script on my Pi. I then see that the partUUIDs are incorrect:
The partUUID of the partitions is not correctly set in /etc/fstab and /boot/cmdline.txt. Which makes sense, because looking at the backup script here in function mirror_SD(), the mounting is necessary to do the string-replace to set the partUUID.
However, I find it strange that I am able to mount the partitions after running the script, but mounting fails during execution of the backup-script. Is there an explanation for this?
Also, the micro-SD card boots without issues, when the above error does not occur …
I have quite the same issue.
In my backup, the cmdline.txt contains this wrong entry: root=PARTUUID=-02
Even when I set the UUID of the backup SD card partition (e.g. root=PARTUUID=4eaae159-02, I cannot boot from the backup card.
Thank you @mstormi for the fast reply!
Yes, two years is a long time.
However, I just setup my system with the latest openHABian (openHAB 4.1.3) system from scratch because my little bit older system (openHAB 4.1.1) got corrupted and the backup did not work. Maybe because of this UUID issue, at least the partition ID was also missing there.
Just today I enabled the automatic backup functionality, tried the backup SD, and it didn’t work again. So, I am looking for a solution.
openHABian and openHAB are different, unrelated things, the openHAB version is irrelevant to mirroring.
What openHABian image did you use for the new setup ?
How exactly did you setup mirroring ?
Set debugmode=maximum in /etc/openhabian.conf, make sure to capture the output of your terminal and re-setup mirroring, then post the log here.
Sorry for the late reply. I will also not give a final statement now, just a short interim report.
I assume the relevant version you were asking for is this one:Linux openhabian 6.6.31+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64
On my previous system, I only made manual raw copies (option 54). Unfortunately, I could not boot from the mirror SD card.
On the new system, I also used option 54 first and again could not boot.
Then I used option 53 to setup automated mirroring and also could not boot on the first try.
Yesterday, I tried again and was able to boot.
Before I say it works, I’d like to try it a few more times.
However, I’m not sure if automated mirroring is the right way to go. I’m worried that the system will slowly degenerate and using the mirroring won’t fix anything because it will run into the same issue again.
So, I’d like to read up on the Amanda backup first (How to backup your openHABian server using Amanda | openHAB).
My approach would be to create manually an SD mirror before making any major changes to the system (e.g. upgrading to openHAB 4.2 which is now the latest version) and use Amanda to go back to certain intermediate states.
Please let me know when I am on the wrong track.
Ah, I didn’t download the image directly. I used the Raspberry Pi Imager tool:
Unfortunately, I didn’t find the location where it stores the downloaded images. So, I cannot tell you file name.
I tried to use the backup from the automatic mirroring again and it seems to work fine.
So, I removed the automatic mirroring via openhabian-config option 53, set debugmode=maximum and wanted to start a manual raw copy. I tried it with
sudo openhabian-config | tee -a output.txt
and
sudo openhabian-config | script output.txt
but with both options, I was not able to operate the config tool because my key inputs were just somehow written onto the dialog box with no functional effect:
So, I just ran the tool using sudo openhabian-config as usual, performed the raw copy (option 54) and copied the text of the power shell box: raw_copy_debug_01.txt (41.2 KB)
I hope this output is sufficient for debugging?
The raw copy did not work, or at least openhabian / openHAB did not start.
Also after correcting the partUUID using my Windows machine, the SD card was not working.
I was only able to take some pictures (in sum, I tried to boot two times):
Sorry, currently I do not know how to bring the boot output into a text file. I will check the internet.
I know that screenshots are not good for debugging as you do not see everything and you cannot search for texts. I just thought that they might help you nevertheless.
I had issues with my system again and the backups didn’t work… again.
So, I spent some time to investigate the issue with SD Raw Copy backup (see attached file). Like half a year ago, sometimes it works (starting in line 60), but mostly it doesn’t (starting in line 1300 and 2640).
What I saw is that in cmdline.txt of the boot partition, the PARTUUID is missing / wrong:
console=serial0,115200 console=tty1 root=PARTUUID=-02 rootfstype=ext4 fsck.repair=yes rootwait
Additionally, it is also missing / wrong in /etc/fstab of the root partition (see line 4996): PARTUUID=-01 /boot/firmware vfat defaults 0 2 PARTUUID=-02 / ext4 defaults,noatime 0 1
Sometimes, it even shows different IDs (see line 2595): PARTUUID=88f9c798-01 /boot/firmware vfat defaults 0 2 PARTUUID=da168714-02 / ext4 defaults,noatime 0 1
Maybe you can find any issues in the logs?
Or do you have an idea how I can further investigate this issue?
I’m not gonna attempt to collect the information about your system setup it would take to
debug that for you but there’s some weird output you should look after.
You have several (fake?) sd devices. Those should not exist on a proper genuine Raspi
system. You should get rid of those first.
You didn’t tell what what you entered. Seems to be 54 raw copy? That now should not do anything to PARTUUIDs. Those become relevant when you setup mirroring, no sooner.
I’d advise you to re-set up your system from scratch, using latest openHABian image, rather to fiddle around making wild guesses about some messed up system of unknown age and state.
First, I have used two different mikro SD to USB adapters. Now I have attached a card reader with several slots. Probably that’s the reason why there are some unused sdx devices. But the result is the same as with the mikro SD to USB adapters.
Also with another SD card (64GB instead of 32) the issue is the same. The PARTUUID is missing in the cmdline.txt and fstab.
I noted “issue with SD Raw Copy backup”. This selection is also visible in the log file e.g. in line 970: + choice2='54 | Raw copy SD'
In fact, Raw Copy sets the PARTUUI (see e.g. line 1071):
And e.g. in line 1107: + sed -i 's|26552aeb|88f9c798|g' /storage/syncmount/cmdline.txt
In line 4861, it is even visible that Raw Copy exchanges the original PARTUUID with an empty string: + sed -i 's|26552aeb||g' /storage/syncmount/cmdline.txt
And in 4865 for fstab: + sed -i 's|26552aeb||g' /storage/syncmount/etc/fstab
In line 3646, there is an error noted because of that I assume the new PARTUUID cannot be read out and thus not set correctly:
+ origPartUUID=26552aeb
++ yes
++ set-partuuid /dev/sdd2 random
++ awk '/^PARTUUID/ { print substr($7,1,length($7) - 3) }'
mount: /tmp/set-partuuid-mnt: cannot mount; probably corrupted filesystem on /dev/sdd2.
dmesg(1) may have more information after failed mount system call.
+ partUUID=
I set up the system with the latest openHABian version from scratch a few days ago. The problem is always the same.
Is this Raw Copy a script where I can have a look at?
