Amanda howto for openhabian and NAS

Tags: #<Tag:0x00007fd313813f38> #<Tag:0x00007fd313813dd0> #<Tag:0x00007fd313813c90>

(Stefan) #21

Now I have tried it 2 hours and read many tutorials.
Unfortunately without any success.
Now amanda is installed and I get via the shell

amcheck openhab-dir


 / etc / amanda / openhab-dir / disklist ", line 4: duplicate disk record, previous on line 2

reported. I do not know how to continue.
openhabian is supposed to be hassle free, but unfortunately I do not notice anything more!

(Markus Storm) #22

Hmm, did you use the Amanda install menu in openHABian?
The error you see is because there’s a duplicate or bad line in the disklist file. The fourth line, obviously.
As explained in the tutorial, that’s the file to contain all the directories and raw devices to be backed up.
One line per device or dir.
I believe you’ve inserted that error yourself, so you should be able to manually edit that file to get rid of the error.
If you didn’t touch it, show me the contents.

(Stefan) #23

yes I used the Amanda install menu in openHABian.

well, could you please tell me how to edit this file in the shell? I could not open the config with user backup or openhabian

(Markus Storm) #24

put "sudo " in front of your edit or cat command
and show me the output of ‘ls -l /etc/amanda/openhab-dir/disklist’

(Stefan) #25

[22:19:14] root@openHABianPi:/home/openhabian# ls -l /etc/amanda/openhab-dir/disklist’
ls: cannot access /etc/amanda/openhab-dir/disklist’: No such file or directory

and now?

(Markus Storm) #26

as I said, delete the duplicate line

(Stefan) #27

THX I deleted the duplicates:

Now I see the following

[22:23:12] backup@openHABianPi:~$ amcheck openhab-dir
Amanda Tape Server Host Check
WARNING: skipping tape test because amdump or amflush seem to be running
WARNING: if they are not, you must run amcleanup
NOTE: host info dir /var/lib/amanda/openhab-dir/curinfo/openHABianPi does not exist
NOTE: it will be created on the next run.
NOTE: index dir /var/lib/amanda/openhab-dir/index/openHABianPi does not exist
NOTE: it will be created on the next run.
Server check took 0.011 seconds

Amanda Backup Client Hosts Check
Client check: 1 host checked in 2.395 seconds.  0 problems found.

(brought to you by Amanda 3.3.6)


[22:22:39] root@openHABianPi:/home/openhabian# sudoedit /etc/amanda/openhab-dir/amanda.conf
  GNU nano 2.2.6                                        File: /var/tmp/amandaXXkApLfW.conf

org "openHABian openhab-dir"                            # Organization name for reports
mailto ""                                               # Email address to receive reports
netusage 10000 Kbps                                     # Bandwidth limit, 10M
dumpcycle 2 weeks                                       # Backup cycle is 14 days
runspercycle 7                                          # Run 7 times every 14 days
tapecycle 10 tapes                                      # Dump to this number of different tapes during the cycle
runtapes 1
tpchanger "chg-disk:"    # The tape-changer glue script
taper-parallel-write 2
autolabel "openHABian-openhab-dir-%%%" empty
changerfile "/etc/amanda/openhab-dir/storagestate"                      # The tape-changer or SD- or disk slot or S3 state file
tapelist "/etc/amanda/openhab-dir/tapelist"                             # The tapelist file
tapetype DIRECTORY
infofile "/var/lib/amanda/openhab-dir/curinfo"          # Database directory
logdir "/var/log/amanda/openhab-dir"                    # Log directory
indexdir "/var/lib/amanda/openhab-dir/index"            # Index directory
define tapetype SD {
    comment "SD card size"
    length 50000 mbytes                                 # actual Bucket size 5GB (Amazon default for free S3)
define tapetype DIRECTORY {                             # Define our tape behaviour
        length 50000 mbytes                             # Every tape is 100GB in size
define tapetype AWS {
    comment "S3 Bucket"
    length 50000 mbytes                                 # actual Bucket size 5GB (Amazon default for free S3)

amrecover_changer "changer"                             # Changer for amrecover

# don't use any holding disk for the time being
#holdingdisk hd {
#    directory "/holdingdisk/openhab-dir"
#    use 1000 Mb

define dumptype global {                                # The global dump definition
        maxdumps 2                                      # maximum number of backups run in parallel
        holdingdisk no                                  # Dump to temp disk (holdingdisk) before backup to tape
        index yes                                       # Generate index. For restoration usage
define dumptype root-tar {                              # How to dump root's directory
        global                                          # Include global (as above)
        program "GNUTAR"                                # Program name for compress
        estimate server                                 # Estimate the backup size before dump
        comment "root partitions dumped with tar"
        compress none                                   # No compression
        index                                           # Index this dump
        priority low                                    # Priority level
define dumptype user-tar {                              # How to dump user's directory
        root-tar                                        # Include root-tar (as above)
        comment "user partitions dumped with tar"
        priority medium                                 # Priority level
define dumptype comp-user-tar {                         # How to dump & compress user's directory
        user-tar                                        # Include user-tar (as above)
        compress client fast                            # Compress in client side with less CPU (fast)
define application-tool app_amraw {                     # how to dump the SD card's raw device /dev/mmcblk0
        plugin "amraw"                                  # uses 'dd'
define dumptype amraw {
        program "APPLICATION"
        application "app_amraw"
# vim: filetype=conf

(Markus Storm) #28

well, then. Move on to amdump

The duplicates were there because you wrongly called the installation menu twice.

The ls command failed because you wrongly copied the ’ char

(Stefan) #29

Are you sure that I should run amdump now?

When I run amreport openhab-dir I got the following information

[22:23:45] backup@openHABianPi:~$ amreport openhab-dir

Hostname: openHABianPi
Org     : openHABian openhab-dir
Config  : openhab-dir
Date    : October 27, 2017

*** A TAPE ERROR OCCURRED: [directory '' does not exist].
No dumps are left in the holding disk.

The next tape Amanda expects to use is: 1 new tape.
  openHABianPi /dev/mmcblk0 RESULTS MISSING
  openHABianPi /etc/openhab2 RESULTS MISSING
  openHABianPi /var/lib/openhab2 RESULTS MISSING

                          Total       Full      Incr.   Level:#
                        --------   --------   --------  --------
Estimate Time (hrs:min)     0:00
Run Time (hrs:min)          0:00
Dump Time (hrs:min)         0:00       0:00       0:00
Output Size (meg)            0.0        0.0        0.0
Original Size (meg)          0.0        0.0        0.0
Avg Compressed Size (%)      --         --         --
DLEs Dumped                    0          0          0
Avg Dump Rate (k/s)          --         --         --

Tape Time (hrs:min)         0:00       0:00       0:00
Tape Size (meg)              0.0        0.0        0.0
Tape Used (%)                0.0        0.0        0.0
DLEs Taped                     0          0          0
Parts Taped                    0          0          0
Avg Tp Write Rate (k/s)      --         --         --

  planner: Adding new disk openHABianPi:/dev/mmcblk0.
  planner: Adding new disk openHABianPi:/etc/openhab2.
  planner: Adding new disk openHABianPi:/var/lib/openhab2.

                                                         DUMPER STATS   TAPER ST                                                                                                 ATS
HOSTNAME     DISK              L ORIG-kB  OUT-kB  COMP%  MMM:SS   KB/s MMM:SS                                                                                                    KB/s
-------------------------------- ---------------------- -------------- ---------                                                                                                 ----
openHABianPi /dev/mmcblk0        MISSING ---------------------------------------                                                                                                 ----
openHABianPi /etc/openhab2       MISSING ---------------------------------------                                                                                                 ----
openHABianPi /var/lib/openhab2   MISSING ---------------------------------------                                                                                                 ----

Additional information: the destination share folder \\backup\openHABianPi\ is empty.
That means, their is no folder like slots or slots\drive0 or slots\drive1

(Markus Storm) #30

It does not do harm to run it.
But it seems you have (wrongly) forgotten to mount the disk and entered your server’s IP+server-side directory when the installation routine asked you to enter a directory …(genervt) das ist hier kein Windows, und wenn da steht directory dann ist damit auch directory gemeint und man kann da nicht was anderes wie server-ip:<remote-directory> eintippen und erwarten daß das geht. Natürlich muß man erst händisch (dazu gibt es keinen openHABian Menüpunkt) die Platte vom Server mounten und dann den lokalen Mountpoint als directory angeben.

(Markus Storm) #31

Because it was not mounted before you ran the Amanda install, so openHABian was unable to create those dirs/links.
Properly mount the dir and re-run Amanda install from the menu.

(Stefan) #32

I am very sorry and I can understand that you are annoyed.

Thanks for your help, but I’m newbie on the linux domain.

That’s why I had initially asked for a “step by step” tutorial.
I tried to get to know myself. I try to read a lot and certainly do not ask because I do not want to inform myself.
But it is really hard to get involved here.

Again, I am very sorry that I annoyed you.

Could anybody explain me, how to mount a NFS Share after I run the openhabian Amanda install guide, and forgot to mount my NFS share before?
I am not sure what all I forgot und lost on configuration, in that case.

Thank you so much.

(Markus Storm) #33

That’s basic Linux knowledge and a quick Google search for ‘Linux mount’ should get you an answer like this one.
As I said in my previous post, properly mount the disk and then re-run Amanda install from openHABian menu.

(Stefan) #34

I like to learn, but in this way,it makes no sense for me.
Now I know how the Linux share mount works, but neither I know, where to set the mountpoint to point, still I know whether then amanda maybe would have to work.
Unfortunately, this is not a big help for me.

Now for me it is time to deactivate the Cronjob (/etc/cron.d/amanda) and continue to read and have to wait, that maybe someone could helps me.

@mstormi: Thanks for your “help”

(Tom barker) #35

I read this thread with some interest. I have had problems with my install as well.

Given the stated openhab objective of “Simpler setup and configuration possibilities for “regular” i.e. not tech-savvy users” one would expect the Readme would contain at least some guidance as to what needs to be done. As is I don’t think the current README does that. I have some suggestions to fix that

In this post I provide my experiences with installing AMANDA using the existing README. Note that these comments are based on a NAS backup so some or all of these comments may not apply if using the USB attached storage option (which I did not try). While there is a lot of detail I have provided this as I think it would be useful feedback for the maintainers. I have not provided details of the exact linux commands I used as I figured anybody using a NAS would know how to access man pages and Google for more details.

Firstly, I read the Amanda README several times in great detail trying to get a feel for what was going on. In this respect I found the introduction section to be excellent background information. Was a bit confused as to how to actually do it so I just dived right in.

Since I would be using a Synology NAS I mounted it on the Pi and tested it. It later transpired that I should have given the “backup” user permissions on the mount on the Pi and this is not in the README nor is it obvious (at least it wasn’t to me anyway). And BTW you cannot switch to backup user unless you know the backup users password – which is not in the README. I found out how to change that via Google and an Amanda users group!

In respect of mounting the NAS I had read this thread and knew that I had to place the mount at “/mnt/NAS” before running install but this is NOT mentioned in the README and is only obvious if you look at the config files (more on that later).

On installation users will get a step which asks them the mount location. A non tech savvy user may think that answering this step will create the mount which of course it doesn’t.

I tried to open the config files but it seems that I am not logged in as backup user. Change to backup user and I cannot traverse the directory structure because backup is not in the sudoer group. Fixed that but this is getting frustrating. Entered my email details but left everything else alone.

So I then ran the install and it asked me how many tapes or slots I wanted. The concept of tapes or slots is not described in the README . I had no clue so I aborted the install and went to read the docs at Still no clue. I initially entered four and then it wants to know the size of the tapes. Back to Still no clue so entered size of the SD card (32GB). I still don’t know if this is optimal but I have heaps of space on my NAS. In this respect I suggest the Readme should briefly explain how Amanda uses slots to emulate physical tapes and how these are used by the installed configuration. This does not have to be complicated and for a NAS backup I was thinking something like.

  • “ For a NAS backup the installed configuration will set up a number of slots in the storage directory. Each of these slots will emulate a physical tape. The default/recommended number of slots (tapes) is 15 but this can be changed during the installation process. The installation process will ask for the size of the storage directory and each slot will be allocated that size divided by the number of slots. The suggested/recommended size required for a full openhabian backup is XXXXXMB. The installation will run a full [not sure if its a full back up or incremental?] backup (amdump) at midnight and that backup will be stored in the first non-empty slot. The next day’s backup will be stored in the next available slot and so on until each slot is full at which time the first slot will be overwritten. All of these actions can be changed via the Amanda configuration files– see manual pages for further details”

By the way it appears the Amanda config files are only accessible AFTER you have actually installed Amanda. Which means users have no idea what the default configuration is until after the install - by which time they have changed it and may already have entered sub optimal values for tapes and tape size and it doesn’t work. They then have to go back and edit and its starts to get messy. Similarly you cannot see the adopted backup strategy set out by Cron and Disks configuration files until after you have installed Amanda . This means it’s more difficult to figure out what is going on before you install and thus there is more chance of not setting it up properly. As a suggestion could a section be added to the README which sets out the questions the installation process will ask and also gives some model answers based on a standard openhabian backup.

Anyway in my situation I left the config files as is and only changed the email address field. I had already set up EXIM4 email on the pi.

Also when I mounted my NAS It was not clear that I did not have to create a slots directory with the required number of slots in it because the installation process does that for you.

Ok so I had the installation finished and I try to run it and it fails. I had to wait 5 hours before amdump reported a fail and I found that out – (again not mentioned in the readme but is mentioned in several forum posts so I was expecting it).

So after accessing the detail of the default config files (from a post on the forum because you cannot access them until after the installation is finished) I change the file back to default configuration with 15 slots and re-entered my email details

I ran Amcheck and it just hangs with the display of - Slot:0 volume”” with no response to keybd input on Putty (ctrl C to stop it) which is when I figured out I needed to give backup user permissions on the mount folder. Ahh …. Now it seems to work. Amcheck runs ok. Amreport says nothing found so we do an Amdump. Which works. Finally - after four days (a large portion of which was trying to understand Amanda! and what it wanted/needed).

Now I go to sort out cloning an SD card. I used my NAS as the temporary storage location which worked OK. However, I note that the Readme instructions for the amfetch are misleading as they use “openhab” rather than the default “openhab-dir” as the dump to fetch! They are also referencing a different user name and directories than would be found in a normal openhabian setup. This was a bit confusing.

So in summary I got it working but it took quite a while. For the benefit of others I have provided a suggested checklist for using the NAS option (with comments questions): but please note I have simply listed the steps I followed to get it to work. There may be a more optimal approach:

  • Before trying to install Amanda from the config console decide on the backup strategy, tape numbers and sizes .
  • Make sure there is a NAS (NFS type) local mount in the location that is expected by Amanda (which is /mnt/NAS?). Test the mount to make sure any local changes are reflected in the remote location. Also give ‘backup’ user permissions on the mount location on the Pi. Configure the NAS to allow NFS access by the Pi.
  • Install Amanda using the config console
  • Change the backup users password (the default password is ???)
  • Add backup user to the sudo group. You need to do this or you will not be able to edit any config files.
  • If you want email notifications from Amanda set up and enable an email system on the Raspberry pi (Exim4?) and amend the Amanda config file to enter your email address.
  • Run amcheck on the configuration see for troubleshooting
  • Run amreport see for troubleshooting
  • Run amdump on the configuration. Note: this may take a number of hours
  • Check wherever the backup went to and that it has actually backed up the files
  • Run amrestore to check that functionality
  • If backing up the SD card image make a clone of the SD card. [Add some sub steps on how to do that such as mount second card or add second mount for temp NAS location or whatever]

Finally: and on a more positive note I think this is a very good backup solution. Thanks for that. I feel the README just needs a bit of tweaking to provide a little more background and some high-level steps for using the NAS option. Also while I did not appreciate it at the time I have learned a lot about Linux while debugging the install.

(SiHui) #36

As I have read a lot about users having problems with Amanda setup you may have some time to contribute your findings to the docs?

(Markus Storm) #37

Thanks for your constructive input. I’ve reworked the README, find the updated working copy here and tell me what else you feel I should add or change therein.

I’m a little irritated by a couple of things though, such as what you write on that backup user part.
There should not have been a backup user prior to running the Amanda installation menu option (IIRC, it’s created as part of the package install). You then were asked to enter its password during installation, so backup user’s default password is what you’re entering. Also, you can switch to become the ‘backup’ user without a need to enter its password. Also no need to have backup in the sudo group as the backup user is not meant to be used to edit Amanda config files (doesn’t do harm but you don’t need it).
You can use sudo to become root from your ordinary user (“pi” or whatever you usually login as) and edit files or in order to switch over to become ‘backup’. You just have to be ‘backup’ to run amdump etc.
I added switching user IDs as an example to the tutorial.
There’s also no need to use /mnt/NAS as THE mountpoint. You can use anything here as long as you specify the directory (asked for during Amanda installation) to have the name of the mountpoint that you have previously mounted the NAS disk on.

As a general comment, it’s still amazing me every day how naive most users approach an important and complex topic like this one.
As a spare time enthusiast-only maintainer, I cannot write documentation that educates each-and-every user on everything on-the-fly, you have to start somewhere above Zero, we cannot start teaching the Linux basics.
And it seems one cannot write an installation routine to anticipate every possible queer user move or a documentation without a possibility to misunderstand given hints and explanations.
I admit that it wasn’t obvious that you need to mount your storage dir BEFORE you run the Amanda installation routine, but if you did, it should have set the proper permissions and links. I think I now made that clearer in the new version.

Actually, ALL people to report problems (at least all I remember ATM) were Windows people with little or no UNIX experience that tried to apply Windows concepts and failed simply because you cannot apply Windows concepts to a UNIX box:
There’s people trying to use CIFS (i.e. Windows sharing protocol) to share a disk from a UNIX NAS to a UNIX client (openHABIan) instead of properly using NFS for that task.
There’s people asking how to edit a file.
There’s people to interrupt the installation or to run it twice or even more often and wonder why that makes a difference.
There’s people to enter <server-ip>:<remote-dir-on-server> when they were asked to enter a directory.

People’s problems with most of this is a general understanding thing, and while I agree on a need for and welcome any enhancements to the tutorial, this cannot really be fully solved inside openHABian/Amanda code or docs.
And most users pay little attention themselves … I bet you and others haven’t carefully read the text in the windows.
Find below the code copied right from the installation routine. It’s showing what you (the user) were asked at Amanda installation time. I don’t feel this to be ambiguous or unclear, but if you feel so, please tell me how to change it.

dir=$(whiptail --title "Storage directory" --inputbox "What's the directory to store backups into?\nYou can specify any locally accessible directory, no matter if it's located on the internal SD card, an external USB-attached device such as a USB stick or HDD, or a NFS or CIFS share mounted off a NAS or other server in the network." 10 60 3>&1 1>&2 2>&3)
        tapes=$(whiptail --title "Number of virtual storage containers in rotation" --inputbox "How many virtual containers will you setup inside the storage dir ?" 10 60 3>&1 1>&2 2>&3)
size=$(whiptail --title "Storage capacity" --inputbox "What's your backup storage area capacity in megabytes ?" 10 60 3>&1 1>&2 2>&3)


 if (whiptail --title "Backup raw SD card ?" --yes-button "Backup SD" --no-button "Do not backup SD" --yesno "Do you want to create raw disk backups of your SD card ? Only recommended if it's 8GB or less, otherwise this can take too long. You can always add/remove this by editing ${confdir}/disklist." 15 80) then 

A locally accessible directory is a local directory, so nobody should feel encouraged to enter <server-ip>:/<remote-dir> or similar there, right ?
It’s also asking “How many virtual containers will you setup inside the storage dir ?”, what’s wrong with understanding that ?
The backup storage area is the area (and no tape or subdir or anything else), so what’s unclear about that this question is asking to specify the total amount of storage to be used by Amanda ?

It’s asking you to enter the backup user’s password. How do you come to ask its default password afterwards ?
And it’s asking you whether you want to backup your raw SD card and even gives a warning that doing so will take long.

Seriously… I’m not trying to ‘bark back’ but I’d really like to understand how I need to change the docs to make you understand them better .

(Markus Storm) #38

I’ve reworked the README once more. Link is still this.

All, please read this and let me know what you believe is missing or wrong.

(Tom barker) #39


Thanks for listening. I have been out all day but I had a very quick look at the new README and my first impression is what an outstanding job. I am very impressed.

Now I need to apologize about my backup password comments but in my defence I just looked at my log (I keep a note of what I do or did so I can remember how to do it again) and it seems I did set an initial password in response to the installation prompt. But I also have a note that it would not recognize that password and I had to enter a new one. More on that later.

I also see in your new README that if you have your mount set up the Amanda install should recognise that and set permissions on the mount folder. That does not seem to have happened in my case because even though I had the mount working before I did the install it would not work until I gave the backup user the proper permission.

Now contrary to what you may think you lose your bet because I did read the install prompts before I responded to them. I even abandoned the install because I did not know how to answer them and went to to try and work out what I was being asked. Having read the new README I have a better (but still incomplete) understanding but that was not the case a couple of days ago.

I see you are not understanding why I am confused about virtual containers. Try and see this from the point of view of somebody who does not know that much about Amanda (and did not have the benefit of the explanation in the new README). So I get asked “How many virtual containers will you setup inside the storage dir”. I have no clue how many I want but I am able to understand (from the Amanda docs) that this relates to how many tapes I want to emulate. Well looking at this from a logical point of view I know what a tape is and how it can store data but to work out how many I want I need to know how these tapes are being used by the installed configuration of Amanda. So when it asks me how many tapes - I ask myself how can I answer that when I don’t know how the tapes are being used because I do not know how the installation has been configured and even if I did would I understand it. Anyhow I can figure out from the README that we are doing a backup every day but is it a full backup or incremental? And how is the backup being put onto the tapes. For example Is the backup being duplicated across tapes to provide redundancy (so for example identical backup data is saved to tape 1 and tape 2). Or is the backup being spread across tapes. When will the tapes be overwritten. What happens if I specify too many tapes or not enough. If it is a small backup related to the size of the tape will it continue to write until that tape is full or will the system switch to a new tape every day. I had no idea so I tried to decode the information at - How to Set Up Virtual Tapes - and it looks like I need a whole bunch of data which I don’t know - like number of incremental dumps per full dump, size of incremental dumps, total tapecycle etc etc. So I still cant’ answer the question. In the end I just used the default of 15.

Then it asks me “What’s your backup storage area capacity in megabytes”. This is rather obvious question which I can answer because I am dumping to a NAS and there are currently 2.6TB of available space. However, I suspect that what it really wants to know is not how much space I have but how much space I will let Amanda use which is an entirely different thing isn’t it. I am still not sure what to put here. I will have another read of the new information shortly. BTW I realize that I could answer this if I had a dedicated disk or partition but I don’t. One of the reasons I have not done this is I don’t understand how big a disk/partition I would NEED. Again I will read the new information and see if that helps.

I hear what you are saying when you say “ one cannot write an installation routine to anticipate every possible queer user move” but with the greatest of respect I think you will agree there is a fine point in achieving a balance between providing nothing and trying to cover everything anybody could possibly do. In this respect your new README looks like it has achieved a better balance. While I have not read it in great detail my first impression is it looks exactly like enough information to point people in the right direction.

In respect of the new README and given the couple of issues I experienced (see second and third paras) I am going to reinstall openhabian on my Raspberry Pi 3B (not to worry I have a clone on standby -courtesy Amanda) and reinstall Amanda (using the new README as a guide) to see if I can recreate those issues. This might take a couple of days and I will get back to you after I have done that with any further suggestions on the README.

Thanks for your efforts in providing a robust back up solution and your quick turnaround of my suggestions

(Markus Storm) #40

Possibly so. While it was intended to work without write access, I have changed that code to set ownership and group write rights of the storage directory for the backup user.

Ok, yes this is misleading. I meant to ask for the total amount of space but actually used the input as the number of tapes.
Will remove that question on the number of containers for the directory and S3 storage variants.
Reason it was there is that I also plan for a variant to use removable SD cards, and these you have to specify in number and size. Unfortunately this requires Amanda 3.4+ which is not yet available as a Raspian package.
Will change the size question to make clear it is about the total amount of space you want to dedicate to amanda on your storage area.

The updated (but untested) version of the install routine is here, awaiting review.
If you know how to replace script before you execute it to setup your next box, please do and let me know if it works.