I opened this topic in beginners forum because when I realized what I did I felt like using openhab for the first time…
Reading about the crashes of SD cards I tried cloning my SD card to have a backup of the tons of hours spent for items, rules, etc in openhab
Unfortunately I inserted my SD in a Windows PC. After finishing the copy process I tried booting my Pi as many times before with the old SD card. But now the following message appears:
end Kernel panic - not syncing : VFS : Unable to mount root fs on unknown-block(179,2)
Is there any way to get this solved without setting up the SD card again? Or do I have to run the creation process of openhabian again?
Did windows say that the device is being damaged and asks you to repair it?
If yes and you selected to repair it, I’m afraid that it’s gone. At least the boot sector. Maybe you still can mount it on another linux machine and save the data?
No, I did not repair anything. I was just opening the card in explorer and copying the content. I will try via live Linux to recover the data. Hopefully not everything is lost…
As far as I know windows can’t open a openhabian memory card I have too use a program like win32 disk image too read the memory card windows has probably done something too your card
You could whip up a VM of some live linux distro and have the card mounted by the VM so its in the Windows computer but its being directed to a linux VM for recovery purposes.
Going forward setting up samba so you can browse to the /etc/openhab2 folder to copy your work may be a better solution.
In Live Ubuntu I can see the folders overlays and System Volume Information. Don’t know if this is a good sign or not. I can’t find any Linux file system like the /etc or any other.
Is there any Ubuntu tool I can use for trying to recover the rules and items file or should I try to look for a general recovery tool?
Last I was aware System Volume Information is a Windows feature.
This software is OS independant and I have been able to recover files from a 2/3 disk failure of a RAID 5 setup, the larger the files got the less likely the recovery but I recovered numerous smaller files.
It is important to realize that there are two partitions on an RPi SD card, the boot partition that is FAT32 (or something like that) which Windows can read and the Linux partition that is EXT2 (maybe 4? O can’t remember) that Windows cannot read. So when you plug in the SD card it will show one drive named “Boot” and it will likely ask you to format or repair the other one.
Paragon makes an EXT file system software that will let Windows read the other partition on the SD card. But if you already have an Ubuntu, you should be able to see it there. If not you might be out of luck. Given the nature of how SD cards work, recovery options are not always as successful as they are with HDDs.
First of all thanks for the great support, as always here in this forum!
It took me some time to fully run test disk and photorec (which comes with test disk). With photorec I was able to find all rules and item files, as txt or h files but the content was undamaged. I saved what I found/remembered.
Also I found pieces of other stuff like habpanel configs, grafana stuff, logs and so on. So before saving just the rules and items file, do you think there is a chance to recover the SD fully again? Maybe with a tool that fixes the issue from the first post. Before I started this thread I read the article below, but the cmdline.txt on the SD did not contain this part:
root=/dev/mmcblk0p2
Sorry if this makes no sense, I am really not familiar with Linux. Maybe that’s the reason why I try everything to recover what I was able to install/configure.
Glad to hear the tools recovered the stuff you were after!
As for the other data you are after, what do you need it for? Asking from a point of view if its worth your time vs having OH offline for longer? I would think only the habpanel would be important, but it depends on how much time you have put into it so far.
Graphs and logs can be created new, so is the history something you need? I also have no experience with grafana.
Something to consider for the future, research how to use the fstab file to mount a windows file share (one example, you can mount other type of shares) and have the OS write the data to another system. But this also depends on how much complexity you may or may not want to add.
Well, it is not only the rules and items, though I am happy at least this could be saved.
I mean configuring user rights and the setup of additional components like influx, grafana, mosquitto is meanwhile part of the openhabian config, so this will be quick.
It is more the time consuming stuff like adding and naming the items, the configuration of the bindings, finding the Api (speech, weather, etc.) keys again.
Also all the things I did in habpanel and grafana will have to be done again.
If there were some other files apart from rules and items I could search for to recover that would save in some cases quite some time.
I was always thinking about backups but for some reason I couldn’t use the Amanda(?) solution in openhabian. Maybe I will also reconsider using the Pi or a device with better performance and a nonSD storage.
If you have Items or Things that are not in .items and .things files, you need to also recover at least /var/lib/openhab2/jsondb. That is where those get stored.
If you recover ALL of /var/lib/openhab2 and /etc/openhab2 you will have everything for OH.
For the rest you will need to research what needs to be backed up on a service by service basis.
For some things, like the InfluxDB files, you may not be able to tell if they are corrupted or not just by looking at them.
I doubt it will be worth your time to try to recover this SD card.
Photorec “creates” txt and other files. So basically I wasn’t seeing any paths or rules files. I let the search run thru the folder of recovered files. So for the files I knew the content of it was fine
OK, I will install everything again and let you know what I should have recovered, too.
Thanks for all posts and giving me the hints how to recover the items and rules. I really thought the probably more than 200h I spent were lost. I learned my lesson to regularly backup at least those two kinds of files.
Ironically the plan to backup my SD (on a Win PC with win32disk) led to all this mess. Next time I will just clone the SD from time to time with my Ubuntu stick
You can always leverage cron and a simple bash script to copy folders you want to retain, you can even tack in a date time stamp as part of the folder in the destination you back up.
I recently had to setup a curl execution for work to download a bank file and then copy it to three different servers and put the time stamp in the file name so the developer could write logic around his process to pull the correct file but also have a selection to choose from for troubleshooting reasons. If you want to see how what I did I can post just the copying parts.
This is very interesting. I haven’t seen it that way. I thought the easiest way would be to clone the SD card, so you have an image as backup. And if something goes wrong you can just use the other SD card with the latest image.
If you copy folders you would have to run the whole installation with openhabian again before you use the retained data, if I understood the process you are describing correctly. Does that work with all the files or are some files “linked” to the installation (I mean do they have a different name for different installations)?
For me as someone without much Linux knowledge a solution would be good where I could just insert the backup SD card and everything runs again as before (time of the image creation).
Update: In the end it was just 4-5h to recover everything (grafana charts not yet re-created). I was afraid of the habpanel settings but luckily there was a local snapshot available on my tablet . Also I should have written down all API keys (voice, openweathermap, etc.).
The best thing is that openhabian meanwhile has all the additional components like grafana + influxdb and user rights configurations included, so it is just a couple of clicks away. Initially I had to do that manually (with the instructions of this forum, of course).
Correct, that you would need to re-install openhab/openhabian again from scratch; for me its not a huge concern. I can always pull up the OH doco and install OH, what I can’t do is remember all the various rules or exact item/thing names that I used.
The backup script I was talking about copies the file that is specified in the script.
I’ve also switched hardware a few times in which case an image wouldn’t help. I have also on occasion shared my rules with co-workers and then modify things to work in their OH and provides something to build upon.
Glad you were able to recover majority of what you wanted! If you keep up with your sdcard image you could save your APIs on your PI in txt files and keep them with your image as a passing thought.
Today I finished the last part of restoring like trying to remember how I managed to set up my mqtt broker and things textually + revoking the token for my Telegram bot after having not written down the ID needed for the API…
Those are just some of the things I never took care of because I was never facing this scenario. I only remembered to look for the rules and items and totally missed the cfg, persist and things files. Lesson learned.
Concerning the backup. I would be very interested to see your solution. Your arguments about switching hardware, etc. make absolutely sense for using a script to copy files rather than having a system-bound backup as I had in mind. Also if you change one of the components in your smarthome and it comes with a new (device) ID or some components do not interact with each other properly, an old image would not help. And in the end a fresh installation always sounds better than using an old version (I realized I missed a lot of “new” features that are part of openhabian now).
When you say you have an own script I assume you did not like the backup solution that comes with openhabian (“Amanda”)?
So here is the script I did for work, I took out all of the work specific things but kept the curl command as it creates the source file that the following commands then copy to 3 files for Dev/Quality (QA)/Prod. There are more than likely a better way to go about this, but its currently not broken and was done with a deadline; later this year the systems that should be performing these downloads will get upgraded and negate this daily weekday job.
The first two sleeps are just to give the system time as the txt file is in the MB for size and its not a time critical process; then to avoid having to manually clean up txt files I have the 8 hour sleep in place incase the developer notifies me of an issue with the day’s filedump file.
In your case modify the ‘mv’ to ‘cp’:
cp -R /etc/openhab2/ /your/destination/
For the /media/ folders those are actually /etc/fstab windows share mounts, from the OS stand point it thinks it owns the folder and makes it easier to script things. I can’t include the fstab form my work as it would have most of the mount line omitted but included one from my house. Here is what I use to keep the logs from writing to the SD card:
An easy way to think of linux shell scripting is that if you can issue a command in putty/linux terminal you can put it into a .sh file, very similar to a windows batch file except you have to make the .sh file executable from the linux file octel side of things.
This is really cool. I think I will use that too. I also like the idea of writing the log files to an external device to avoid having to delete larger files once in a while.
Concerning the paths to copy I will use the ones @rlkoshak suggested: /var/lib/openhab2 and /etc/openhab2
. Also I should have written down all API keys (voice, openweathermap, etc.).