Sorry I do not know how to change the install script so I cannot test your new install. If you need someone to do this I am happy to act as a test case but you will need to tell me how to do it. You can PM me if you wish.
OK so I had a look at the README ( I will provide comments on it separately) and I then started a fresh openhabian install on my Rapberry PI 3B.
Now you stated that “There should not have been a backup user prior to running the Amanda installation menu option’. So to test this and after first boot had completed I executed sudo cat/etc/passwd to see if there was an existing ‘backup user”. There WAS a backup user
Details > backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
I ignored this.
I then prepared a share on my NAS and created a /mnt/NAS folder on the pi. Went into fstab and added the line [192.168.1.37:/volume1/homes/Tom/Rpi301 /mnt/NAS nfs defaults 0 0]
Tested that with a sudo mount -a and then checked the directory to see if some test files had been shared. Everything OK so onto to an Amanda install. BTW I ignored all the intermediate set up steps such as setting up mail, samba adjusting locale and time zones etc.
Provided password, set containers to 15 and available space to 15000MB (three times the image size) chose not to backup SD card or use AWS
.
Everything seems to have worked so switch to backup user and do amcheck openhab-dir. Same problem as yesterday the installation does not seem to have granted permissions on the mounted share and it hangs with slot 0 volume “ “.
Ctrl C to exit and run an Amreport on openhab-dir and it says parse error; could not open config file “etc/Amanda/openhab-dir/Amanda.conf; No such file etc. I ran into this issue yesterday with the previous install. So while still backup user I tried to edit the Amanda.conf file and that generated a Security Exception “this will be reported”. OK change back to openhabian user and try to edit the Amanda config file. But it will not even let me CD into the directory (unless I use sudo -s). So to change Amanda config files I have to be root user not the normal openhabian. Is this how it’s supposed to work?
As I already have a working backup system (even though it may not optimised) I aborted the install. However despite what it told me on pi when I went to the NAS it HAD already created the required slots! Huh?
Yesterday, I fixed both of these issues by adding backup to the Sudoers group and granting permissions on the mount folder using “Sudo chmod -R 777 /mnt”. Now bearing in mind what you said about not needing to add backup to sudoer group would there have been alternative way of fixing these issues?
Anyway I know you have an updated installation script but I just though this might be useful info for you.
My comments on the README by sections. If there is no comment I thought it was very good .
Amanda Background Section.
This does a good job of explaining what a dumpcycle and a tapecycle are. The explanation of how data will get a full backup and then incremental backups is easy to understand.
After the words “Amanda will store into each tape subdirectory as (storage size) / (number of tapes)’ may I suggest you add the following (or something along these lines) “ The Amanda installation will by default do a backup every day at 0100 hrs. The first backup will be a full backup into slot 1 directory. On the second day an incremental backup will be placed in slot 2 and this will continue until all slots are full at which time Amanda will start to overwrite the slots starting at slot1. If you have selected SD card image backup the whole image will be backed up every day”. (Edit I see you have said this later on but I think this information needs to be provided sooner and I suggest it be in this section -and I do not see any harm in it being duplicated later)
Installation Section (mounting part)
Personally, I would have been happy to just see a reminder to mount the NAS and or SD card and leave it at that.
I see you have pasted your session onto the page and in this respect can I suggest you split the graphic into two parts and move the introductory text to suit. Eg: procedure1 for NAS with graphic then procedure 2 for SD card with graphic. Sorry if this sounds a bit pedantic but it might make it a bit easier to understand.
I can follow the mount procedure for the NAS quite well as I did it almost the same way as what you show.
I can’t comment on the USB section as I did not try it. Also to be honest to understand what you have done I am going to have to google each of the commands to see what you have done in detail. Now in this respect that’s OK with me - I can do that; but I fear other’s expectations might be a bit higher because of all the help they get when they install openhabian. Have another look at the install instructions for openhabian itself and how detailed they are and I think you will see what I mean.
Installation Section (after console paste)
In the first bullet point in the part that says “This is where you have mounted your USB storage or NAS disk share (see above).’ May I suggest you change this to “This is where you have mounted your USB storage or NAS disk share (which in above example for the NAS is /storage/server and for the usb stick is ……).
In the second bullet point there is a typo “typecycles” should be “tapecycles” . You have not suggested a number or told us how to work it out . I do not think a novice user would be that bothered about changing this so I suggest this prompt be left out of the install and you just set the default to 15. As an alternative may I suggest you add the words “the default is 15 and it is suggested to leave this at 15 unless you have a really good understanding of Amanda”
Further Comments
I suggest a separate section or sub section setting out when the backup user should be used and when the normal user is used… This is mentioned but I think this should be given a bit more visibility by having its own paragraph (probably somewhere after installation). This was one of the areas where I got myself confused.
Overall Impressions
A very nice detailed explanation of Why What and How. Thanks Marcus.
Mmh, I guess you should be able to upload it to your openHABian box, go into shell, make it executable (chmod a+x ) and just run it. Assuming your openHABian box is connected to the inet, you can also download it FROM your box using wget https://github.com/mstormi/openhabian/blob/patch-11/openhabian-setup.sh
If that does not work for some reason (environments settings or other), you would need replace the openhabian-setup.sh on your box with my copy. Attach your card writer to your pi, put in your openHABian image SD, find it (fdisk -l) and mount the proper partition, then copy it there.
Ok. So it’s not created on Amanda package installation. The command actually is /usr/sbin/chpasswd <<< "${backupuser}:${password}"
So it should not make a difference WHEN the user was created.
Yes.
Well, the whole openHABian-setup script isn’t built to be transaction-safe, so if you interrupt it, you get a partial installation. There is no builtin routine to do the cleanup.
Now that’s what I’ve changed in the install script update, so if you did use that one, it should work right away and you would not need to do this.
Well, sudoers group defines which users are allowed to execute sudo commands.
Now as per lets say general good UNIX practice, you shouldn’t use functional users (such as “backup”) for administration tasks but use your personal user instead (which should have been created at the beginning of openHABian install, if I recall correctly). That user should already be part of the sudoers group, i.e. no need to edit it.
Thanks for reviewing. I’ve changed the working copy (link see above) to reflect your input.
I intentionally placed the info like that as it does not apply to Amanda in general but only to our specific openHABian installation instance, so it should not be part of the ‘background’ section.
Me, too. But I believe to not show that example set of commands would result in a number of people give up on this.
On the other hand, to give fool-proof examples for each and every type of storage is desirable but impossible.
So I fought against myself … and lost , so I came up with this compromise.
done.
I removed that from the install routine of the “directory” (NAS or USB mount) and AWS storage variants.
I added a paragraph at the end of the Amanda background section.
First up apologies for spelling your name incorrectly. Sorry.
Personally, I am comfortable understanding what you have written. I do have a few questions about Amanda but I am going to try and sort them out myself by using your install as a lever.
The one question I do have relates to the various users. In the background section you talk about the various users and suggest to use the personal user and sudo (if left as is during the default install the personal user would be openhabian@openHABianPi). I am comfortable with that and I tried to use it to open the /etc/Amanda directory and I get “permission denied”. I got around this using sudo -s which makes me the root user root@opeHABianPi. So now I can get access to the Amanda configuration. However, if (as opehabain and not root) I try to access other config files in the /etc directory (like for instance int or network) then moving into them (using CD) DOES work without having to use sudo-s. So if I understand it I have to use root to change the Amanda configuration files but I can just use the personal user and sudo for other config files for equally important functions. Unless I am missing something this inconsistency is going to confuse new users (like me).
I do have a minor suggestion which relates to the storage preparation section. Now when I went to mount my drive I googled how to do it. There were a lot of examples and most of them did not work. The reason they didn’t work was because the answers provided were not specific to the Debian/Raspberry Pi system or the NFS file system. Now I know the objective of the README is NOT to teach people how to Google but if might help if you added a sentence after CAN DESTROY YOR SYSTEM that said something like. “Google is your friend, so you are encouraged to use it to find out how to mount a disk. However, when searching you should include the operating system you are using in the search terms. So don’t just type “how to mount a NAS disk” type “how to mount a NAS disk on a debian raspberry pi”. If you don’t do this you will get answers related to mounting using Redhat or ubuntu and the commands may not work or may actually harm your system”. I will leave it up to you if you take this suggestion up as I agree it is a matter of balance in how much you have to say – and it’s not me who is answering newbie questions.
BTW I didn’t have the confidence to attempt testing your install. I could probably get it started but I figured I would not know enough to gauge the effectivity of the results.
Well, this is one of the subtleties of UNIX … I don’t feel it to be inconsistent, so if it is confusing you, this is not because of the Amanda install but UNIX in general. I’ve changed the install script, though, to add the openhabian user to the backup group so you at least get (…mostly…) the same rights w.r.t. Amanda config as the “backup” user has.
changed that a little.
Well it’s meant to be a replacement, so you should run it instead of the original one (at least when it comes to the Amanda installation part) and afterwards, compare the system it created to your expectations. Probably easiest to go through your install notes you took last time and see if the irritations/issues are still there.
As the script changes were just merged into the repository, I believe you should be able to obtain them when you select the openHABian ‘update’ menu option (sorry, none at hand to try it myself).
OK I figured that would be easy to test.
Steps taken
reflash clean SD card
Let openhabian install
Update
Ugrade
Mount remote storage
Run sudo openhabian-config
Install Amanda backup using prompts. Only tested with the files and not a SD card image and not AWS either or backing up to an SD card.
AND
Test it with amcheck - no issues
Run amdump - worked fine
So looks like any bugs are no longer there. I even tested trying to run it as a normal user (no not on purpose) and it told me to change to backup user so that works as well.
Very interesting topic guys, reading the to and fro between you has been useful. I have but one ‘newbie’ question.
I first tried to install Amanda months ago but gave up for another non-automated backup solution, my question is can I uninstall amanda and clean everything up for the fresh install without creating a new SD card. I have a stable system and prefer not to rock the boat right now.
Thanks
PS I think I’d done the original install before mounting any NAS redirect so there won’t be any mounts to worry about.
Hard to answer as we don’t know about the Amanda leftovers on your system and there is no ‘uninstall’ routine.
Then again, if you choose to install it again (from the openHABian menu), it should overwrite existing config.
I’m positive that’ll work out.
Worst thing to happen I guess is that you still have the old cron job in existence, so backups might be started twice a day, but you can easily correct that (deduplicate lines in /etc/cron.d/amanda).
I have the same problem. I run into that problem because of the openhabian menu entries.
This entry is not enough. It should be changed to:
51: Mount a backup/restore drive (You could show here fstab? Or a readme/link what to do)
52: Setup Amanda Backup (What to backup: complete SD or USB or only config files and data (for users who know what they are doing))
53: Start Backup (for testing purpose)
54: Restore Backup (It should show me all backup versions and allow me to select one)
As a newbie I would expect a functionality like that.
I know that for entry 54 I may have to make a new install of openhabian first.
With the last comments on this topic I learned a lot more about amanda and the specific problems.
A big thanks goes to TCB1 from me for showing the problem a nomal user is confronted with when starting up with amanda. I did not find the entry point, but now it looks a lot clearer.
I still have a mounting question with a nas. When the nas is not used it normaly falls a sleep. What happens with the nas when it is mounted to a 24/7 running raspberry pi. Will it fall to sleep mode after the amanda backup? Or will it stay running all the time? If yes, how could I avoid this?
No. You ran into it because you expected the thing to be simple and didn’t know about or figured that you wouldn’t need to read the docs first.
Granted, you should get a warning and a pointer when you select this menu option … I’ve just added that to the code.
No you don’t.
… and that’s where the problem is. As an experienced UNIX admin and doc writer, I can tell that it is impossible to write up a README to match each and every level of user knowledge, or to even create an installation routine that anticipates and automatically takes care of each and every possible variant.
But every newbie expects this to work with his specific setup / show the steps needed for his specific level of knowledge right away.
Read the now-updated README. See post 37 for the link.
Markus, thanks a LOT for the time and effort to write the README.
Had Amanda installed and working in about 30 minutes after reading about it for 2 hours
It is using a Synology NAS for the backup.
Thanks everbody for bringing this feature to Openhabian.
I am wondering how i can achieve the most clever way to backup my whole system with it. I am running everything on an Rpi3, but not from an SD card, but an USB harddrive (WD PiDrive).
My root mount point / is on sda7 so i added
HausPi /dev/sda7 user-tar
to my disklist, based on the approve to backup a raw image of an sd card. However when i start the backup it just finishes after a few seconds without doing something.
Or would it be most convenient to just backup / and make exceptions to folder like /tmp /dev etc?
Happy christmas everyone
Edit: Ah sorry nevermind
HausPi /dev/sda7 amraw
seems to be the right way. It is running now and i hope the result will be ok
I haven’t designed/tested this to work, but there’s also a dumptype root-tar meant to backup root partitions, plus you can define your own dumptypes.
If it doesn’t work out of the box, there’s logs in /var/log/amanda that, combined with a quick G**gle search, will help you figure out why.
jumping in here as I just set up amanda on my openhabian … first of all … thanks for the nice work with the setup procedure … worked perfectly with all the warnings in the readme ;).
As mentioned in the readme, I also thought about how I can restore the backup later, and as many of us will run amanda on the pi they want to possibly restore later, it would be nice to have a section in the readme that shortly explains how to do this, if you lost the whole system and you don’t have a raw image at hand.
As far as I understood for amrecover to work, you would need to have the index, but we could still use amrestore. So from my understanding I would do a fresh install of openhabian and amanda, pointing to the same directory I provided in the first place for the backups. Then I should be able to restore my files with amrestore. Right?
Additionally it might be a good idea to also backup amanda itself. This would include /var/lib/amanda/ and /etc/amanda/ if i’m not mistaken. So you can use amrestore for the amanda backups and later on you could use amrecover. Would this work or did I miss something?
Agreed, but that’s where it’s getting complicated. It’s a chicken-and-egg-problem, and to do it all right, you’d need a second, independent backup system, to work with a variety of HW, under a variety of conditions of what’s broken and what’s restoreable … or in short, too much and too difficult to “shortly explain” or to even automate.
That’s why there’s the recommendation to create a backup SD right away and to keep doing so at times. So why not stick to that one. Note you’ll also be replicating the storage index as per that last date of replication.
I myself would need to dig the Amanda docs and try it for myself in order to be able to reliably answer your question, but yes, to save /var/lib/amanda/* and /etc/amanda/* using cp or tar to some off-openhabian location is probably a good idea so you can restore your backup index (without using Amanda) if in need after a crash/reinstall.
hm … having a quite big SD Card (maby I should change this) a raw Backup will take a while over the network, that’s why I was looking for other alternatives.
Anyway … I’ll give it a try when I’ve some spare time and report back on my success … or failure
You don’t have to permanently backup the raw device. Done once, just take it off the disklist file.
Yes a smaller SD card (or to be more precise: smaller partition(s) that are in need of backup) makes sense.
, so I came up with this compromise.
