flexbackup frequently-asked questions

Suggestions/additions welcome

FAQ Revised: Wednesday 18 June 2003 17:02:26

Table of Contents

1. Configuration
2. Backup
3. Common problems
4. Recovery
5. Implementation

1. Configuration

1.1. I can't get my exclude expressions to work...
Use regular expressions, not wild cards (aka globs). See a perl, sed, or grep reference, or "man 7 regex". Simplistically, you probably mean to say ".*" instead of "*", and ".?" instead of "?". Also, leave OFF the first part of a path that is the filesystem you are archiving; the exclude expressions only match sub-paths of the backup directory. Test to see if your expression matches what you want to exclude by:

cd <topdir>; find . -regex "<expression>"

1.2. I don't want to run backups or remote shells as root
You can do this with 'sudo'.
  1. Set up a passphraseless ssh key for a user to run the backups.
  2. Set $path{command} = 'sudo command'; in the config file. Do this for all commands that will need privilege (find, afio/tar/etc)
  3. Set up an /etc/sudoers file with something like this:

    Cmnd_Alias BACKUP = /usr/bin/afio, /usr/bin/find
    backupuser ALL = NOPASSWD: BACKUP


1.3. On my system, "tar" is not GNU tar but is the vendor's "tar" - GNU tar is installed as "gtar". On my system, "dump" is "ufsdump".
Use the path overrides at the end of the config file. A couple examples are given there, such as: $path{'dump'} = 'ufsdump';. You can use this mechanism to override paths or just change command names.

2. Backup

2.1. Can you explain numeric dump levels?
Level 0 gets everything. For 1 or higher, files that have been changed since the last backup at a lower level are archived. For instance, if a level 2 backup was done on Monday, followed by a level 4 backup on Tuesday, a subsequent level 3 backup on Wednesday would contain all files modified or added since the level 2 (Monday) backup.

See also these sun docs for an example with some pictures.

Traditional Unix 'dump' behavior only allows levels 0-9; flexbackup removes this limit if you are using any of the non-dump archive formats.

For those people coming from other platforms that are not used to dump-style numeric levels, the latest versions of flexbackup have added -full, -differential, -incremental that get translated into appropriate dump-style level numbers. Definitions:

These are simply aliased to numeric levels: full = 0, differential = 1, and incremental = (latest backup + 1)

2.2. Do you support multi-tape archives?
If you split filesystems/directories in a "set" across mulitple tapes, yes, that is supported. As of now, having one directory archive span multiple tapes is not supported. True, some of the archiver programs can deal with end-of media, but it is hard to generically support. We'll be working on a generic solution for future versions. For now, split the directory up into workable portions, use larger media, or backup to disk...

2.3. How do I archive multiple filesytems to tape at once without rewinding in between, and without redefining "all" $filesystems in the configuration file?
Do this:

flexbackup -fs /dir1 -norewind
flexbackup -fs /dir2 -norewind
flexbackup -fs /dir3 -norewind

2.4. I'm backing up to files on disk, and I don't want the filenames to have timestamps in them.
Set $staticfiles to true in the config file. Archives will then have names such as "etc.3.dump.gz" rather than something like "home.0.200301212305.afio-bz2". There is a similar parameter for the log files, by the way.

2.5. What is the maxiumum size of the archives flexbackup can produce?
FIXME

3. Common problems

3.1. I get I/O errors or other errors from my tape drive.
Run this: flexbackup -test-tape-drive. It writes a couple small files to the tape, then reads & diffs them. This will flush out any problems with parameters like blocking, filemarks, and padding; or issues with the tape drive or driver itself.

If it fails, try these one at a time:

  1. If you are using buffering, turn it off ($buffer='false') until you can get things working without it.
  2. Change the block size parameter ($blksize) in the configuration file. The default is 10k, but some tape drives like 32k or 64k much better.
  3. Try setting $pad_blocks to false
  4. Switch the setting of $mt_var_blocksize to false
  5. Try with with $indexes set to false

If you are using an IDE tape drive under Linux, and are still having trouble, you might also try the 'ide-scsi' layer with the 'st' module, and treating the thing like a SCSI tape - I've had more success that way.

You can also just try writing/reading stuff straight from the drive with tar and dd (to factor flexbackup out of the loop), although that's what the -test-tape-drive switch tries to do. If you can get that working, but flexbackup still seems to malfunction, let us know. Try things like: (The example below uses afio to test, tar/cpio/others will be different) 

  # Backup two dirs
  mt -f /dev/tape setblk 32768
  mt -f /dev/tape rewind
  mt -f /dev/tape erase
  find /dir1 -print | afio -o -z -v -b 32k - | dd ibs=32k obs=32k of=/dev/tape
  find /dir2 -print | afio -o -z -v -b 32k - | dd ibs=32k obs=32k of=/dev/tape

  # List
  mt -f /dev/tape rewind
  dd ibs=32k obs=32k if=/dev/tape | afio -t -z -v -b 32k -
  dd ibs=32k obs=32k if=/dev/tape | afio -t -z -v -b 32k -


3.2. Backups seem to work ok, but I can't get extracting/listing to work.
Usually a matter of tape positioning, or else is a tape drive parameter problem as talked about above. For positioning, use 'mt rewind; mt fsf <n>' to get to the right file number on the tape, or use the "-num" switch with "-list|extract|compare|restore" to have flexbackup do it for you. If you are using flexbackup's indexes, remember that file number 0 is a header used for the index itself.

3.3. How do I get remote backups over ssh to work without prompting for a password?
For rsh, use .rhosts or hosts.equiv. But you really should NOT be using rsh.

If you are using ssh and are having a problem with it asking for a password, then you don't have a passphrase-less authorized key set up, your RSAAuthentication parameters are incorrect, or you have some other public/private key problem. Section 7.2.2 of the ssh FAQ might help:

http://www.onsight.com/faq/ssh/ssh-faq.html

In a nutshell the value of your ~/.ssh/identity.pub needs to be present in your ~/.ssh/authorized_keys file for RSA authentication to work. For root accounts you might also need to put

PermitRootLogin yes

in your sshd_config file, as root is treated with more care than normal user accounts. Note: you might choose without-password or forced-commands-only instead of yes for more security. See your ssh(1) and sshd(1) man pages.

If you do not want to use passphrase-less ssh keys with root logins, See the section on use with "sudo". (Or else sit at the terminal and type passwords during the backups if you *really* want to...)

3.4. I keep getting only one backup per tape when I do "-fs all"!
Use the non-rewinding device.

3.5. 'buffer' complains that it can't run successfully or that it doesn't have enough shared memory.
Set $buffer_megs to a lower value and try again. I usually run with it set somewhere between 5-20MB.

On FreeBSD, SysV shared memory can run low using the GENERIC kernel at its default value (especially if you are running something like GNOME). You can modify your kernel file and change the line

options SHMMAXPGS=2048

To something really big. You can also change this dynamically at startup by putting

kern.ipc.shmmax={large integer}

in /etc/sysctl.conf. Other BSD's will have very similar mechanisms.

3.6. When using -newtape or -erase, the tape drive sits there forever making noise and nothing happens on the screen
FIXME

4. Recovery

4.1. How do I format the filelist for extacting only certain files in an archive?
FIXME

4.2. How do I find out which archive(s) contain a certain file?
If you don't know in which archive to find a certain file, look at the log files. As long as you have verbose turned on (default), you can just 'zgrep filename /var/log/flexbackup/*.gz', and that works pretty well.

4.3. The disk totally died. How do I get flexbackup's archives off the tapes on a machine without flexbackup or even perl installed?
Something like this, if using compression with tar/cpio/dump:

dd if=/dev/tapedevice bs=10k | gzip -dc | tar xvf -

replace "tar xvf -" with "restore -i -f -" or whatever. Skip the gzip pipe if you aren't using compression. afio is a bit more complex, but read the manpage...

4.4. I don't have the index files! (disk died, different machine, etc.)
This is OK, you can still extract/restore/etc. The index files are for human convenience only. Just skip the first file on the tape, it's a simple "tag" text file. The rest of the tape is your normal archives.

5. Implementation

5.1. Why keep the table of contents db on disk rather than on the tape?


5.2. This index stuff is a useless mess/I can't get it to work!
It isn't necessary if you don't like it. Set $indexes = 'false'; in the config file, and just use written labels on the tapes...

flexbackup-help at lists dot sourceforge dot net

FAQ compilation thanks to makefaq