Marilyn Schweitzer
High Performance and Parallel Computing Department
Dorota Genser, Gene Oleynik, Margaret
Votava
Online Systems Department
Fermilab Computing Division
The Fermi Tape Tools library is a C subroutine package that provides consistent, platform-independant tapeI/O for a variety of tape drives on a variety of platforms . It provides basic tape input and output, support for obtaining and tracking available tape statistics, and various informationnal and support routines for tools which need to allocate and control tape usage.This users guide will provide an overview of teh ftt library calls, take the reader through the process of writing a simple ftt based tape copy program, and also provide a complete call reference of the routines provided for tape input, output, and information
This document corresponds to version v2_7 of the ftt package.
#include <ftt.h> ftt_descriptor d; d = ftt_open("/dev/rmt0", FTT_RDONLY); /* do some stuff here */ ftt_close(d);There are several items that should be mentioned here.:
If your program needs to deal with multiple tape drives, it
can call ftt_open multiple times, to get a unique
descriptor for each drive. It should then issue an
ftt_close on each descriptor it got from ftt_open
before exiting
length = ftt_read(d, buffer, buffer_length);or;
length = ftt_write(d, buffer, data_length);
fprintf(stderr, "%s\n", ftt_get_error(0));which will print the error string on standard error. If your program wants to attempt to do something to recover from errors, you will need to pass a pointer to an integer into the first argument of ftt_get_error which will give you back an error number. Then you can attempt to take appropriate action based on the error. This looks like:
char *p; int n; ... p = ftt_get_error(&n); if (n == FTT_ENOTAPE) { request_tape_mount(); }The error numbers you get from ftt_get_error will be one of the following:
ftt_errno | Description |
FTT_SUCCESS | no error has occurred |
FTT_EPARTIALSTAT | unable to obtain some or all of the information requested |
FTT_EUNRECOVERED | previous error was unrecoverable, tape can only be unloaded/rewound (to protect data) |
FTT_ENOTAPE | there is no tape in the drive |
FTT_ENOTSUPPORTED | Unsupported device |
FTT_EPERM | you do not have permission to perform the specified operation on the device. |
FTT_EFAULT | you gave a buffer address range whose access yields a memory fault |
FTT_ENOSPC | A write attempted to cross end of media |
FTT_ENOENT | User attempted to set density to a nonexistent one |
FTT_EIO | Physical read or write error.0 |
FTT_EBLKSIZE | Attempt to write/read unsupported block size s |
FTT_ENOEXEC | Attempt to run setuid executable for security bypass failed -- not executable |
FTT_EBLANK | Encountered blank tape (i.e. passed logical EOT) |
FTT_EBUSY | Asynchronous I/O is going on this device, or another process has it open |
FTT_ENODEV | System does not have drive configured |
FTT_ENXIO | Tried to contact a nonexistent process |
FTT_ENFILE | No system file descriptors available |
FTT_EROFS | Tape is write protected |
FTT_EPIPE | Setuid bypass died |
FTT_ERANGE | Record too large for buffer |
FTT_ENOMEM | Unable to allocate memory required to perform call |
FTT_ENOTTAPE | What we thought was a tape device was not. |
FTT_2SMALL | The block size issued is smaller then the device can handle |
FTT_ERWFS | Tape is not write protected (and should be) |
FTT_EWRONGVOL | Wrong volume label on tape for verification |
FTT_EWRONGVOLTYP | Wrong type of label on tape for verification |
FTT_ELEADER | hit tape leader while skipping backwards |
FTT_EFILEMARK | hit file mark while skipping records |
FTT_ELOST | The tape postion is unknown |
FTT_ENOTBOT | The tape is not at BOT |
Your code can compare the error number (n in our
example) to the values in this table to decide what went
wrong in the preceding ftt library call.
A utility routine, ftt_open_logical(), is provided to create an ftt descriptor for a possibly fictitious device. It provides the opportunity to see what the various device names would be for specific modes/densities on a different operating system platform. The average user would not use this routine to open the device, but would use ftt_open() instead.
The pair of close routines, ftt_close() and ftt_close_dev(), are similar in function to the pair of open routines, ftt_open() and ftt_open_dev(). In general, the user need simply call ftt_close() when finished with the device. This routine will internally call ftt_close_dev() if needed. ftt_close_dev() will ensure that the proper number of filemarks have been written at the end of tape (if writing) and that the tape is correctly positioned. ftt_close() will free the ftt file descriptor.
It's important to note that some ftt routines will internally call ftt_close_dev(), which may write filemarks, if they need to close the normal unix device driver and open the raw scsi device. The documentation for the individual subroutines will indicate if a specific routine exhibits this behavior.
Note that several ftt_open_dev() and ftt_close_dev() pairs can be called within a single ftt_open() and ftt_close() pair.
The rdonly argument indicates
TABLE 2. rdonly flag values
Value | Description |
FTT_RDONLY | Device will not be written on |
FTT_RDWR | Device is read and writable. |
Note that FTT_RDONLY does not check that the device is physically write protected, nor does FTT_RDWR check that is not physically write protected; a call to ftt_status is required to obtain that information.
This routine performs no positioning of the device.
Note that ftt_open and ftt_close do not need to be called to switch between reading and writing, to cross filemarks, etc. The ftt software will perform operating system opens and closes as needed to allow those transitions.
ftt_open does not perform any operations on the tape drive or tape devices, it simply does bookkeeping to prepare for such operations.
Example:
#include "ftt.h" ftt_descriptor d; d = ftt_open("/dev/rmt/tps2d4", FTT_RDWR);
Example:
#include "ftt.h" ftt_descriptor d; d = ftt_open_logical("/dev/rmt/tps2d4", "IRIX", "EXB-8500", FTT_RDWR);
If the density and mode have not been set by ftt_set_mode(), ftt_set_mode_dev(), etc., the default density and compression mode of the highest density the device supports and no compression will be used.
The routine may sets appropriate modes and/or flags on the underlying device to allow other ftt operations to proceed properly.
ftt_open_dev performs antecessor operations to set the device density, etc. and actually performs an operating system open on the device. No tape motion is performed.
Example:
#include "ftt.h" ftt_descriptor d; int status; status = ftt_open_dev(d);
ftt_close will invoke ftt_close_dev if needed, but performs no other tape or device operations.
Example:
#include "ftt.h" ftt_descriptor d; int status; status = ftt_close(d);
Note that ftt_close_dev can fail, due to insufficient room to write tape filemarks, etc.
ftt_close_dev performs a close on the operating system device. It will only perform tape motion (writing filemarks and backspacing) if the preceding operation was a write or write filemark. No other operations or tape motions are performed.
Example:
#include "ftt.h" ftt_descriptor d; int status; status = ftt_close_dev(d);
All of these routines return negative numbers on failure, for which more information can be obtained as described in the section on Finding out what went wrong.
If called with the tape positioned before a data block, the ftt_read entry reads a block of length at most length from the tape into buffer buf, and returns the length of the block actually read. If the next block on the tape will not fit into the buffer, or if any other error occurs, -1 will be returned
If called with the tape positioned before an end-of-file mark, 0 is returned, and the position is moved to just after the end-of-file mark.
Example:
#include "ftt.h" ftt_descriptor d; char data[32768] int status; status = ftt_read(d,data,32768);
Note that the result of a write should be checked for equality to the length passed in, in most conditions a short write is considered an error.
In the case of an unrecoverable error, an error number of FTT_UNRECOVERED will be returned for this call, and all subsequent calls made until one of ftt_rewind, ftt_unload, ftt_close_dev, or ftt_close is called.
Example:
#include "ftt.h" ftt_descriptor d; char data[32768] int status; status = ftt_write(d,data,32768);
Example:
#include "ftt.h" ftt_descriptor d; int status; status = ftt_writefm(d);
The ftt_retry call calls the routine passed in as op, with arguments d, buf, and len, and if the call fails, retries it up to tries more times. Before retrying, ftt_retry will skip backwards to the nearest filemark, then skip forward to just after the last successfully written block, and retry the operation. The recovery sequence is correspondingly sluggish on some tape drives.
The op passed in should be either ftt_read or ftt_write.
Note that if retrying ftt_write calls, the reverse skip will attempt to write 2 filemarks, making the recovery even slower.
This call will fail if:
* any of the file or record skips fail
* the number of retires is exceeded without a successful write
The return code will be the result of the last operation, either ftt_skip_fm, ftt_skip_rec, or the return code from the passed in op.
Example:
char buf[2048]
res = ftt_retry(d, 5, ftt_read, buf, 2048);
will make up to a total of 6 read calls, interspersed with
file and record skips.
Returns the readonly flag used in ftt_open when creating this ftt descriptor. If the descriptor is invalid, returns -1.
All of these routines return -1 on failure, for which more information can be obtained as described in the section on Finding out what went wrong.
Example:
#include "ftt.h" int status; ftt_descriptor d; status = ftt_skip_fm(d, 3);to skip forward 3 filemarks, or:
status = ftt_skip_fm(d, -2);to skip backwards 2 filemarks.
Note that skipping backwards will write two filemarks and
skip back over them first if the preceding operation was an
ftt_write or ftt_writefm call.
skips forward just past records, or backwards just before
records. count indicates the number of records to
skip. count should be negative to skip backwards.
Example:
#include "ftt.h" int status; ftt_descriptor d; status = ftt_skip_rec(d, 3);to skip forward 3 records, or:
status = ftt_skip_rec(d, -2);to skip backwards 2 records.
Note that skipping backwards will write two filemarks and skip back over them first if the preceding operation was an ftt_write or ftt_writefm call.
Example:
#include "ftt.h" int status; ftt_descriptor d; status = ftt_skip_to_double_fm(d);
Example:
#include "ftt.h" int status; ftt_descriptor d; ftt_rewind(d);
Example:
#include "ftt.h" int status; ftt_descriptor d; ftt_rewind(d);
Example:
#include "ftt.h" int status; ftt_descriptor d; ftt_unload(d);
Example:
#include "ftt.h" int status; ftt_descriptor d; ftt_erase(d);
All of these routines normally return a character pointer to a string containing the name of whatever system device will be used to perform I/O in this mode. The routines all return 0 if anything goes wrong, see the section Finding out what went wrong for a discussion of errors.
Example:
#include "ftt.h" char *name; ftt_descriptor d; int density; int cmp = TRUE; /* compression on */ int blocksize = 0; /* variable block */ name = ftt_set_mode(d, density, cmp, blocksize);
Example:
#include "ftt.h" char *name; ftt_descriptor d; int density; int cmp; int blocksize; name = ftt_get_mode(d, &density, &cmp, &blocksize);
Example:
#include "ftt.h" char *name; ftt_descriptor d; int density; int cmp = TRUE; /* compression on */ int fixed = FALSE; /* variable block */ name = ftt_avail_mode(d, density, cmp, fixed);
ftt_list_all performs no tape operations. It returns the list of devices we think should be present, which is determined when ftt_open_logical is called.
Example:
#include "ftt.h" char **all_names; ftt_descriptor d; all_names = ftt_list_all(d);
ftt_chall performs no tape operations, it merely changes permissions on the devices in /dev.
The rewind value returned will have a value of FTT_RWOC, FTT_RTOO, or bitwise combinations of these values, to indicate ReWind On Close and ReTension On Open, respectively.
A fixed value of 0 indicates variable blocking, a fixed of 1 indicates fixed block operation, but does not provide the current blocksize.
ftt_get_mode_dev performs no tape operations.
Example:
#include "ftt.h" int status; ftt_descriptor d; char *devname = "/dev/my_favorite_device"; int density; int cmp; int fixed; int rewind status = ftt_get_mode_dev(d, devname, &density, &cmp, &fixed, &rewind);
The force flag (if set to one) forces the exact device name specified to be used, allowing access to devices which are otherwise unsupported.
ftt_set_mode_dev performs no tape operations, it merely sets flags, etc. which will be used by ftt_open_dev.
Example:
#include "ftt.h" int status; ftt_descriptor d; char *devname = "/dev/my_favorite_device"; int force = FALSE; status = ftt_set_mode_dev (d, devname, force);
Example:
#include <stdio.h> #include "ftt.h" int status; ftt_descriptor d; char *devname = "/dev/my_favorite_device"; status = ftt_describe_dev (d, devname, stdout);
ftt_get_basename performs no tape operations.
Example:
#include "ftt.h" char *basename; ftt_descriptor d; basename = ftt_get_basename(d);
TABLE 3. ftt statistics
stat | Description | Notes | Origin |
FTT_VENDOR_ID | vendor id data | string | Drive:SCSI Inquiry |
FTT_PRODUCT_ID | product id | string | Drive:SCSI Inquiry |
FTT_FIRMWARE | firmware rev level | string | Drive:SCSI Inquiry |
FTT_SERIAL_NUM | Device serial number | string | Drive:SCSI2 Inquiry |
FTT_CLEANING_BIT | Device needs cleaning | flag | Drive: SCSI ReqSense |
FTT_READ_COUNT | Decimal number of KBread since rewind/reset | counter | Drive: SCSI ReqSense (EXB specific) or SCSI2 Log Sense |
FTT_WRITE_COUNT | Decimal number of KB written since rewind/reset | counter | (see above) |
FTT_READ_ERRORS | Read errors - drive corrected | counter | (see above) |
FTT_WRITE_ERRORS | Write errors - drive corrected | counter | (see above) |
FTT_READ_COMP | Compression factor for reads | numeric | Drive: SCSI2 Log Sense |
FTT_WRITE_COMP | Compression factor for writes | numeric | Drive: SCSI2 Log Sense |
FTT_FILE_NUMBER | File number from BOT | numeric | Ftt: d->current_file |
FTT_BLOCK_NUMBER | Block number in current file | numeric | Ftt: d->current_block |
FTT_BOT | at beginning of tape | flag | Ftt: bookkeeping (above), or SCSI Request Sense, or SCSI Read Position |
FTT_READY | Device ready status | flag | Drive: SCSI ReqSense or SCSI Test Unit Ready |
FTT_WRITE_PROT | Device write protect status | flag | Drive: SCSI Mode Sense |
FTT_FMK | At filemark | flag | Drive: SCSI ReqSense |
FTT_EOM | At end of media (early warning) | flag | Drive: SCSI ReqSense |
FTT_PEOT | At physical end of tape (really at end) | flag | Drive: SCSI ReqSense (EXB-Specific) |
FTT_MEDIA_TYPE | Media type | numeric | Drive: SCSI ModeSense |
FTT_BLOCK_SIZE | Tape block size | numeric | Drive: SCSI Mode Sense |
FTT_BLOCK_TOTAL | Tape length in KB | numeric | Drive: SCSI Mode Sense |
FTT_TRANS_DENSITY | Translated density ala ftt_get_mode | numeric | Ftt: table and FTT_DENSITY |
FTT_TRANS_COMPRESS | Translated compression ala ftt_get_mode | numeric | (see above) and SCSI2 Mode Sense page 0x10 |
FTT_REMAIN_TAPE | Remaining KB of tape | counter | Drive: SCSI ReqSense |
FTT_USER_READ | KB read since open/ unload | counter | Ftt: d->readkb |
FTT_USER_WRITE | KB written since open/ unload | counter | Ftt: d->writekb |
FTT_CONTROLLER | Controller type | string | Ftt: table |
FTT_DENSITY | Tape density | numeric | Drive:SCSI Mode Sense |
FTT_ILI | Invalid Length Indicator | flag | Drive: SCSI ReqSense |
FTT_SCSI_ASC | Additional Sense Code | numeric | Drive: SCSI ReqSense |
FTT_SCSI_ASCQ | Additional Sense Code Qualifier | numeric | Drive: SCSI ReqSense |
FTT_PF | Power Failed! | flag | Drive: SCSI ReqSense (EXB specific) |
FTT_CLEANDED_BIT | Cleaning performed per FTT_CLEANING_BITt | flag | Drive: SCSI ReqSense |
FTT_TRACK_RETRY | Track retry errors | numeric | Drive:SCSI ReqSense (EXB 8x05 specific) |
FTT_UNDERRUN | stop/start counter from | numeric | (see above) |
FTT_MOTION_HOURS | hours tape drive is in motion | counter | Drive SCSI ReqSense (DLT specific) |
FTT_POWER_HOURS | Number of hours device has been on | counter | Drive:SCSI ReqSense (DLT specific) |
FTT_TUR_STATUS | Status | numeric | Drive:ScsiTestUnit Ready |
FTT_COUNT_ORIGIN | Origin of read/write kb | string | Ftt |
FTT_N_READS | Number of read calls | counter | Ftt: d->nreads |
FTT_N_WRITES | Number of write and writefm calls | counter | Ftt: d->nwrites |
FTT_TNP | Tape Not Present bit | flag | Drive: SCSI ReqSense (EXB specific) |
FTT_SENSE_KEY | Sense Key | numeric | Drive:SCSI ReqSense |
FTT_TRANS_SENSE_KEY | Above translated to strings | string | Ftt: |
FTT_RETRIES | total ftt_retry() repeats | counter | Ftt: |
FTT_FAIL_RETRIES | total ftt_retry() failures | counter | Ftt |
FTT_RESETS | Unexpected BOT discoveries | counter | Ftt |
FTT_HARD_ERRORS | Counter of recovered errors | counter | Ftt |
Example:
#include "ftt.h" ftt_stat_buf stbuff; stbuff = ftt_alloc_stat();
Example:
#include "ftt.h" int status; ftt_stat_buf stbuff; status = ftt_free_stat(stbuff);
Example:
#include "ftt.h" int status; ftt_descriptor d; ftt_stat_buf stbuff; stbuff = ftt_alloc_stat(); status = ftt_get_stats(d,stbuff);
ftt_extract_stats performs no tape operations.
Example:
#include <stdio.h> #include "ftt.h" char *value; /* value of extracted stat */ ftt_stat_buf stbuff; /* statistics buffer */ int status, ftt_status; /* status of ftt calls */ char *estring; /* error string */
stbuff = ftt_alloc_stat(); status = ftt_get_stats(d,stbuff); if (status == -1) { estring = ftt_get_error(&ftt_status); fprintf (stderr, "Error:%s Error String:%s \n", ftt_ascii_error[ftt_status],estring); } value = ftt_extract_stats(stbuff,FTT_MOTION_HOURS); ftt_free_stat(stbuff);
Note that media type and compression rates, while technically numeric, are not treated as such by these routines, since subtracting them is meaningless.
ftt_sub_stats performs no tape operations.
Example:
#include <stdio.h> #include "ftt.h" int status, ftt_status; /* status of ftt calls */ char *estring; /* error string */ ftt_stat_buf b1,b2,res; /* statistics buffers */ b1 = ftt_alloc_stat(); /* get buffers */ b2 = ftt_alloc_stat(); res = ftt_alloc_stat(); status = ftt_get_stats(d,b1); /* get tape statistics */ if (status == -1) { estring = ftt_get_error(&ftt_status); fprintf (stderr, "Error:%s Error String:%s \n", ftt_ascii_error[ftt_status],estring); }
/* do some tape i/o here */
status = ftt_get_stats(d,b2); /* get tape statistics again */ if (status == -1) { estring = ftt_get_error(&ftt_status); fprintf (stderr, "Error:%s Error String:%s \n", ftt_ascii_error[ftt_status],estring); } ftt_sub_stats(b1, b2, res); /* get deltas and store in res */
ftt_free_stat(b1); /* free buffers */ ftt_free_stat(b2); ftt_free_stat (res);
Note that media type and compression rates, while technically numeric, are not treated as such by these routines, since adding them is meaningless.
ftt_add_stats performs no tape operations.
Example: see ftt_sub_stats example.
8.7 ftt_stat_buf *ftt_init_stats (ftt_descriptor d)
returns pair of statistics buffers. The first is initialized with zeros and the second with current statistics for the drive.
This routine performs a ftt_get_stats which in turn calls ftt_close_dev which will write file marks if the preceding operation was a write or write file mark.
Example:
#include "ftt.h" ftt_stat_buf *bp; ftt_descriptor d; bp = ftt_init_stats(d);
Example:
#include "ftt.h" ftt_stat_buf *bp; /* status buffer pair */ ftt_descriptor d; /* ftt file descriptor */ int status, ftt_status; /* status of ftt calls */ char *estring; /* error string */
bp = ftt_init_stats(d); /* do some tape i/o here */ status = ftt_update_stats(d,bp); /* update statistics again */ if (status == -1) { estring = ftt_get_error(&ftt_status); fprintf (stderr, "Error:%s Error String:%s \n", ftt_ascii_error[ftt_status],estring); }
Example:
#include <stdio.h> #include "ftt.h" int status; ftt_descriptor d; ftt_stat_buf stbuff; stbuff = ftt_alloc_stat(); status = ftt_get_stats(d,stbuff); status = ftt_dump_stats (stbuff, stderr); ftt_free_stat(stbuff);
TABLE 4. ftt_status flag bits
Bit setting | Description |
FTT_ABOT | At beginning of tape |
FTT_AEOT | At Physical end of tape |
FTT_AEW | At Early Warning mark near EOT |
FTT_PROT | Write Protected tape |
FTT_ONLINE | A tape is loaded and online |
FTT_BUSY | The tape is rewinding, unloading, etc. and not responding. |
indicating the current tape position. This routine shall block up to timeout seconds if there is a tape in the drive, waiting for the tape to come online, or to become not busy. If we cannot determine the presence of a tape, we will simply block until the timeout expires.
An external setuid executable will be run to obtain this data if necessary (e.g. AIX), which means that file marks will be written if the last operation was a ftt_write or ftt_writefm, (see ftt_close_dev()). Therefore, calling ftt_status after every write should be avoided.
Example:
#include "ftt.h" int status; ftt_descriptor d; status = ftt_status (d, 0);
ftt_get_position performs no tape operations, it merely reports bookkeeping information.
Example:
#include "ftt.h" int status; ftt_descriptor d; int fileno, blockno status = ftt_get_position (d, &fileno, &blockno);
ftt_get_error performs no tape operations. Any mode sense data reported was obtained by the routine reporting the error.
Example:
#include <stdio.h> #include "ftt.h" int error; char *estring; /* error string */
estring = ftt_get_error(&ftt_status); fprintf (stderr, "Error:%s Error String:%s \n", ftt_ascii_error[error],estring);
This routine performs numerous tape operations.
#include "ftt.h" int status; ftt_descriptor d; status = ftt_write_vol_label(d, FTT_ANSI_HEADER,"mydog");
The tape is left positioned after the volume label.
Example:
#include "ftt.h" int status; ftt_descriptor d; status = ftt_verify_vol_label(d, FTT_ANSI_HEADER,"mydog",0,TRUE);
TABLE 5. ftt_guess_label label types
Value | Description |
FTT_ANSI_HEADER | An ANSI X3.27 standard tape |
FTT_FMB_HEADER | An FMB tape |
FTT_TAR_HEADER | A tar tape (POSIX nnn) |
FTT_CPIO_HEADER | A Cpio -c format tape |
FTT_UNKNOWN_HEADER | No known label type |
FTT_BLANK_HEADER | Verifies length is -1 and last error was FTT_BLANK |
FTT_DONTCHECK_HEADER | Only used by ftt_verify_vol_label() |
The vol and vlen arguments are taken to be the address of a character pointer and of an integer, and will be filled in with the address and length of the volume name in the tape header, if any.
ftt_guess_label performs no tape operations.
In most cases, the user could use ftt_verify_vol_label directly, which internally makes a ftt_guess_label call.
Example:
#include "ftt.h" int type; char *databuf; int buflen; char *vollabel; int vollen; type = ftt_guess_label(databuf, buflen, vollabel, vollen);
ftt_format_label performs no tape operations.
Example:
#include "ftt.h" int length; char *databuf; int buflen; length = ftt_format_label(databuf, buflen, "mydog", strlen ("mydog"), FTT_ANSI_HEADER);11
Issuing Commands Asynchronously
By default, all ftt commands are issued synchronously. There are several commands, however, that a user may wish to issue asynchronously, e.g. ftt_rewind(). With the exception of ftt_open() and ftt_close() themselves, all ftt commands can optionally be asynchronous.
When a user decides to issue a command asynchronously, he should issue an ftt_fork() command. The child then does the desired commands and returns the status vi ftt_report(). The parent process can continue doing whatever it wants and then rendezvous with the child via ftt_wait(). It can also periodically check if the child process has completed via ftt_check();
It is important to note that only one asynchronous operation at a time may be active on a given file descriptor.
Example:
#include "ftt.h" char *switch (ftt_fork(d)){ case 0: /* child */ ftt_skip_fm(d,2); ftt_rewind(d); ftt_report(d); default: /* parent */ do_stuff_during_rewind(); ftt_wait(d); break case -1: /* error */ pc = ftt_get_error(&err); fprintf(stderr,pc); exit(0); }
ftt_wait performs no tape operations of its own.
ftt_check performs no tape operations.
12
Expert Commands
As much as ftt would like hide protect the user from implementation specifics, there are times that the expert user would REALLY like to something detailed. Most of the routines were born when trying to recover data from tapes that are overwritten (see the Appendix at the end of this user's guide). As a result, ftt provides a few routines to facilitate some of these functions for the knowledgeable user. It is not recommended that the novice user issue these commands.
Example:
#include "ftt.h" #include "ftt.h" int status; ftt_descriptor d; ftt_clear_unrecovered (d);
Example:
#include "ftt.h" #include "ftt.h" int status; ftt_descriptor d; ftt_all_scsi(d);
Example:
#include "ftt.h" int status; int blockno; ftt_descriptor d; status = ftt_locate(d,blockno);13
Linking an Executable
Linking your code with the ftt library should be accomplished by adding the following link options to your Makefile or other build script:
-L$(FTT_DIR)/lib -lftt 'cat $(FTT_DIR)/lib/libs'
where the last portion in backquotes places the contents of the file libs in the ftt library directory on the link command line. This will include any platform-specific libraries needed on the current platform in the link without you having to know what they are, or tailor your build scripts to different platforms. For example, a complete makefile for a program using ftt would look like:
INC = -I$(FTT_DIR)/include
LIB = -L$(FTT_DIR)/lib -lftt 'cat $(FTT_DIR)/lib/libs'
CFLAGS = $(INC)
LDFLAGS = $(LIB)
EXAMPLES = tapecopy
all: $(EXAMPLES)
tapecopy: tapecopy.c
$(CC) $(CFLAGS) -o tapecopy tapecopy.c $(LIB)
ftt_test can only have one drive open at a time. The drive name is specified in one of the following fashions, in descending order of priority:
* on the ftt_open call itself (e.g., ftt_open /dev/rmt/tps2d4)
* on the invocation line (e.g., ftt_test -f /dev/rmt/tps2d4)
* from the environment variable FTT_TAPE (e.g. setenv FTT_TAPE /dev/rmt/tps2d4).
All commands optionally take a "-usage" switch as well. This is to aid the interactive user in determining what are the valid options and parameters for a given command. If the "-usage" switch is used, the actual command will NOT be executed, but a usage message will simply be printed to stderr.
calls the corresponding ftt_open library routine using basename as the input filename. -readonly will open the file for read only access. Without this flag, the device is opened for read/write access. The ftt file descriptor is stored in a global variable for all other accesses on this device by the test routines.
ftt_open_logical [basename] [-flavor <flavor>] <-driveid <driveid>> [-readonly] [-status <error>] [-usage]
calls the corresponding ftt_open library routine using basename as the input filename. flavor indicates the operating system flavor to use. If not supplied, the test routine will use the current operating system flavor. driveid specifies the drive type. -readonly will open the file for read only access. Without this flag, the device is opened for read/write access. The ftt file descriptor is stored in a global variable for all other accesses on this device by the test routines.
ftt_open_dev [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_close [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_close_dev [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
write n (default = 1) test blocks of size nbytes (default = 32768). If ndbytes is specified, the writes will be of random size between nbytes and nbytes + ndbytes. If alignment is specified, random size records must will be adjusted to this byte alignment. Valid values for nalign are 0 (allows odd byte writes), 1 (allows odd word writes), and 3 (allows only longword multiples). If delay is specified, a random delay of 0 to nsec will occur between writes. The first longword in the test block is the number of bytes in the block, the second is the block number, the third is the filename. All remaining bytes are filled with ascending bytes starting with (file number + block number)% 256.
ftt_verify_tblock [-nblocks <n>] [-delay <nsec>] [-oddbyte] [-filemark] [-status <error>] [-maxbytes <readsize>] [-usage]
read and verify n (default = 1) test blocks If delay is specified, a random delay of 0 to nsec will occur between reads. If the -oddbyte switch is used, an odd number of bytes will be used for the input buffer. -filemark indicates that the record should be a filemark. readsize will indicate the maximum number of bytes to read. This value cannot be greater than 64kbytes.
ftt_dump [-nblocks n] [-filename <filename>] [-status <error>] [-usage]
read n blocks (defaults until end of tape) and then write them to stdout or filename if specified.
ftt_undump [-filename <filename>] [-status <error>] [-usage]
write until end of data from stdin or filename if specified.
ftt_writefm [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_write2fm [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_verify_vol_label [-timeout <nsecs>] [-type <type>] [-label <labelname>] [-readonly] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. Information is written to stderr. This command will wait nsecs for the drive to come online. The default value is 0. Valid types are:
* FTT_ANSI_HEADER
* FTT_FMB_HEADER
* FTT_TAR_HEADER
* FTT_CPIO_HEADER
* FTT_UNKNOWN_HEADER
* FTT_BLANK_HEADER
* FTT_DONTCHECK_HEADER
If not specified, a type of FTT_ANSI_HEADER is assumed.
ftt_write_vol_label [-type <type>] [-label <labelname>] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The valid values for type are the same as in ftt_verify_vol_label.
If not specified, FTT_ANSI_HEADER is assumed.
calls the corresponding ftt library routine using the global variable for the ftt file descriptor and prints the current drive status. The user can specify a timeout of nsecs. If no timeout is specified, the default value is 0.
ftt_test_status [-<flag1>... [-<flag>]] [-timeout <nsecs>] [-status <error>]
calls ftt_status using the global variable for the ftt file descriptor. Each flag is tested to see if the condition is true. Valid values for flag are: FTT_ABOT, FTT_AEOT, FTT_AEW, FTT_PROT, FTT_ONLINE, and FTT_BUSY. The user can specify a timeout of nsecs. If no timeout is specified, the default value is 0.
ftt_skip_to_double_fm [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_skip_fm <nfilemarks> [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_skip_rec <nrecords> [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_rewind [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_retension [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously
ftt_erase [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_unload [-async] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor where async will issue the command asynchronously.
ftt_get_position [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The current file and block number is written to stderr.
ftt_verify_position <fileno> <blockno>
verifies that the position specified by fileno and blockno match those returned by ftt_get_position.
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_check [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The list of device node pathnames is written to stderr.
ftt_chall <uid> <gid> <mode> [-status error] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The user name is specified in uid, the group name in gid, and the mode in mode.
ftt_verify_exist
verifies that all of the filenames listed by ftt_list_all really do exist.
ftt_avail_mode <density> <mode> [-blocksize <b>] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The density and mode must be specified in density and mode respectively. If the blocksize is not specified, variable block will be assumed. The device node is printed to stderr.
ftt_get_mode [-status <error>] [-usage]
calls the ftt_get_mode library routine using the global variable for the ftt file descriptor.
ftt_set_mode <density> <mode> [-blocksize <b>] [-status <error>] [-usage]
calls the ftt_set_mode library routine using the global variable for the ftt file descriptor. If no blocksize is specified, variable block mode is assumed.
ftt_get_mode_dev [device_name] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. Node information is written to stderr.
ftt_set_mode_dev <devname> [-blocksize <b>] [-force] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. If no blocksize is specified, variable block mode is assumed. -force will turn the force flag on. It is off by default.
ftt_verify_modes
verifies all modes of the device. This routines will loop through all modes of the device. It will set the mode, write, verify the mode is what we think it should be, rewind, and then verify the data.
ftt_get_basename [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_describe_dev [device_name] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. Information is written to stderr.
ftt_extract_stats <statistic> [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. Information is written to stderr. Valid values for statistic are from the list described earlier in this document (e.g., FTT_PRODUCT_ID, FTT_VENDOR_ID, FTT_DENSITY, etc.).
ftt_init_stats [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_update_stats [-errors][-part_display] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. The accumulated totals are written to stderr. -part_display will only display the statistics that are counters, while -errors will display the read/write counts, errors, and percent error rates.
ftt_dump_stats [-filename <filename>] [-status <error>] [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. It will dump the stats to filename if specified, otherwise stderr.
ftt_list_supported [-filename <filename>] [-usage]
calls the corresponding ftt library routine. It will dump the stats to filename if specified, otherwise stderr.
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_get_error [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. Information is written to stderr.
ftt_max_error [<maxerror>]
Set/show the current maximum number of errors before ftt_test will abort.
ftt_clear_unrecovered [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_all_scsi [-usage]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor.
ftt_locate <blockno> [-usage] [-status <error>]
calls the corresponding ftt library routine using the global variable for the ftt file descriptor. It will locate the tape to the logical block number specified by blockno.
output current date to stderr.
ftt_echo [string] [-usage]
print string to stderr.
ftt_debug [level] [-test] [-usage]
Set/show the current debug level for the ftt library as well as the test routines. The debug level of the library will be set to level. If -test is specified, it will set the debug level for just the test routines. If level is not specified, the current debug level for both the library and the test routines are displayed.
Debug levels for the ftt library are:
* 0 = no debug prints
* 1 = print "Entering routine..." in each routine
* 2 = miscellaneous arguments "got this far" prints
* 3 = debug prints on each pass through loops, etc. More debug prints than you can shake a stick at.
Debug levels for the test routines are:
* 1 = display commands entered by user.
$FTT_DIR/bin/ftt_test ftt_test> ftt_open /dev/rmt/tps2d4 ftt_test> ftt_rewind ftt_test> ftt_close
#!/bin/ksh #========================== # This will rewind the tape # ========================= # Get the tape device by looking in $1. # If that's not set, try FTT_TAPE. # If that's not set either, then exit. export FTT_TAPE FTT_TAPE=${1:-${FTT_TAPE:-""}} FTT_TAPE=${FTT_TAPE:?"No device specified"} #========================================== $FTT_DIR/bin/ftt_test << EOD ftt_open ftt_rewind ftt_close EOD
cat my.ftt ftt_open ftt_rewind ftt_close $FTT_DIR/bin/ftt_test -f /dev/rmt/tps2d4 <my.ftt
All tests can either rely on the device name being specified in the environment variable FTT_TAPE or passed in as argument 0 on the command line.
Name and location: [1]
$FTT_DIR/ftt_test/scripts/<script name>.dat[2]
Usage:
ftt_test -f <drive_name> < $FTT_DIR/ftt_test/script/<script_name>.dat
or:
ftt_run_test <script_name> <drive_name>
2fm
This test will test to make sure that two and only filemarks are written at the end of tape with the ftt_write2fm call. If two individual calls to ftt_writefm are called, four filemarks are actually written.
align
This test will write a bunch of records, rewind, and then verify. It will do this for block sizes that are of longword multiples, word multiples, and byte multiples.
async
This script will verify the asynchronous capability in ftt. It assumes that a test tape is inserted that has been written with the "write" test. It will then skip to the double file mark and then display a time stamp around an asynchronously issued rewind.
This output will need to be also checked manually to verify that the change in time is small. It will then check ftt_wait. An asynchronous command will also be issued that is expected to fail in order to verify that ftt_wait will return a failure.
close
This test will verify that a close command will leave the tape in a known position so that when it's opened again, the user knows where he is.
delay
This script will test write a bunch of blocks and reading a bunch of blocks with a random delay between i/o operations. This tends to uncover strange behavior with devices and device drivers. It is a lengthy test and may take a few hours to run.
describe[3]
Prints all device files corresponding to the one tape drive.
erase
This script will write a volume label, erase the tape, and then verify that the volume label is blank.
full
This script will write to end of tape to verify appropriate errors reporting. It will take FOREVER to run.
label
This script will write all of the various label types and verify them.
mode
This script will test the various ftt_mode routines. It will be a nested loop which for all densities, all compressions, and all block sizes (variable and two different fixed block sizes) will check if the mode is available. If so, it will rewind the device, set the mode, write some data, verify the mode, rewind, and finally verify the data.
notape
This script will unload the tape and then verify that the correct error is returned when trying to access the drive.
position
This script will verify all the positioning commands using the a tape written by write.ftt. It will verify spacing both records and filemarks both forwards and backwards. It will verify that the correct error code is returned if the operation can't be performed. It will check that ftt_status returns the correct positioning information and that ftt_skip_to_double_fm works.
rate [-n nblocks] [-s blocksize] [device][4]
This script will rewind the device, and then write the specified number of blocks with the specified block size. It will time the time for the data transfer and write this information to stdout. The default block size is 32768 and the default number of blocks to write is 1000.
read_only
This test verifies that all ftt_write functions will return an error if the drive is opened as read only. It should also verify that ftt_status determines that it's a read only drive.
root
This script needs to run as root to work successfully. It does all of the operations that require root privileges. Currently this is on ftt_chall.
stats
This script will verify the statistics information. The output of the script will need to be checked manually. It will do a few writes and display the delta stats and then do a few reads and display the delta stats.
verify
This script verifies a tape written with write.ftt. It verifies that the volume label, the number of blocks, the contents of the blocks themselves, the number of file marks, and that the tape is blank at the end. It also verifies that FTT_ERANGE is returned if the number of bytes to read is less than the number of bytes in the record.
twrite
This script writes a test tape beginning with an ANSI label. It will write a few files with a few thousand blocks. The tape written by the script can be used by other scripts for additional testing. This test should leave the tape with only two filemarks at the end.
random
This is a special script that is actually generated by a c program. The c program will generate this ftt script to write a random number of files of random block size. It then randomly picks a position, skips to it, and verifies a random number of blocks.
ftt_fast [-a] [-u] [device]
This script tests all of the reasonable quick tests to verify that the ftt library is behaving the way that it should. It may take on the order of an hour or two to complete. If -a is specified, the alignment tests are performed as well. Note that this may cause SCSI resets. If the -u switch is specified, the unload tests are performed at the end. Note that the tape will have to be reloaded before any other tests can be run.
ftt_slow [-a] [-u] [device]
This tests is all inclusive test suite for the ftt routines. It will call ftt_fast to perform all of the reasonable quick tests. In addition, it will test of the slow routines (e.g., erase, write to end of tape). It may take several hours to run.
ftt_exercise [device]
This test is provided as a convenience for the user who wants to exercise the device to make sure that it is ok. For example, if a drive is replaced, the user would want to verify it with this script.
ftt_ansi_prelabel [device] [label]
This script writes ansi label and rewinds tape.
ftt_dump [drive]
This will dump a tape - used in recovering overwritten data tapes (see See "ftt_dump/undump" on page 49.).
Usage:
ftt_dump <input drive name> | ftt_test -f <output drive name>
ftt_run_test [script] [drive]
This script is used to execute ftt_test with data file located in $FTT_DIR/ftt_test/scripts directory.
Usage:
ftt_run_test [script] [drive]
is evaluated to:
ftt_test -f [drive] < $FTT_DIR/ftt_test/scripts/[script].dat[5]
ftt_multitest [-s -p] [script] [drive_1] [drive_2]....
Runs the requested script to run sequentially (-s) or parallel (-p) on listed drives.
15
In each of the above opts may be -w to open the drive for writing, (which really only makes a difference for getting statistics on 8200's) and/or -x which turns on ftt debugging. The command performs the minimum number of ftt calls to perform the task, as well as ftt_open, and ftt_close. These are all operations which require superuser privilege on one or more systems. Currently the only platform where ftt_suid does not need to be setuid root at all is on IRIX systems from Silicon Graphics.
In all cases, after performing the requested operation, the values returned by ftt_get_error() are printed to file descriptor 0.
Note that for the ftt suite to work properly on those systems which require ftt_suid to be setuid to root, the ftt_suid executable must be in your command search path when the library is run, and installed with the correct permissions.
ftt_suid -s /dev/rmt0would yield the output
FTT_VENDOR_ID is EXABYTE FTT_PRODUCT_ID is EXB-8200 FTT_FIRMWARE is 2600 FTT_CLEANING_BIT is 0 ... FTT_SENSE_KEY is 0 FTT_TRANS_SENSE_KEY is NO_SENSE - is - 0 Okshowing the statistics, then the error code and error string. In this case, the error counters on the Exabyte 8200 would be interpreted as read errors, since -w was not specified.
On IRIX we cannot currently distinguish between Exabyte 8500's and 8505's, or between DLT4000's and DLT2000's, since the hinv command lists them identically.
For IRIX we also recommend the following system patches be installed:
Note that the software will work without the setuid binary as far as doing tape I/O, but the software will be unable to identify the drive type or do SCSI passthrough, so information about tape density names will be generic, and most statistics will be unavailable.
mv ftt_suid ftt_suid.real
cp /bin/true ftt_suid
in the bin directory of the ftt product, and you will
not get errors about the setuid binary -- of course
functionality will be missing, as described in the preceding
sections.
When data is written to an 850x drive, a special end of data mark (EOD) is written after the last data record. When the tapes were overwritten, they contained two EOD marks: one at the end of the original data, and one at the end of the volume label. We wanted to skip past this first EOD and into the original data. Exabyte drives do NOT allow you to skip past an EOD mark UNLESS the drive is configured to support a directory structure. This requires special firmware from Exabyte. Note that the 850x documentation implies that with directory structure support, you can only skip past one EOD mark. We never tried more than one.
We loaded the special firmware into an 8505 drive and then loaded one of the overwritten tapes in this drive. We needed a few specialized commands in the ftt toolkit, so we added ftt_all_scsi[6], ftt_clear_unrecovered[7], and ftt_locate.
Using the ftt_test program, we were able to use ftt_locate to position the tape to a specific logical block in the original data set: For example:
ftt_test ftt_test> ftt_open -readonly /dev/rmt/tps0d4 ftt_test> ftt_rewind ftt_test> ftt_locate 1000 command failed with FTT_EBLANK, expected FTT_SUCCESS ftt_test> ftt_locate 1200 command failed with FTT_EIO, expected FTT_SUCCESS ftt_test> ftt_locate 1500 ftt_test> ftt_locate 1300 ftt_test> ftt_rewind ftt_test> ftt_locate 1300 ftt_test. ftt_close
ftt_dump /dev/rmt/tps0d4 | ftt_test /dev/rmt/tps0d5
ftt_copy -i /dev/rmt/tps0d4 -o /dev/rmt/tps0d5
Jim Meadows (meadows@fnal.gov) and Margaret Votava (votava@fnal.gov) had a test stand setup for such recoveries. It consisted of an mvme133 in a VME crate running pSOS and a Ciprico Rimfire board as the VME->SCSI adaptor. A PC was connected to the terminal port of the drive with the overwritten data. Jim would be able to overwrite some memory location in the drive to convince the heads to spin forward to some arbitrary point. We then attempted to read. This was procedure was repeated until good data was found. The Fermi written rimfire routines were used to control the drives.
[1] The name and usage was changed for version v2_0 to have the same behavior on UNIX and Windows NT
[2] Scripts rate and describe are not ported to WIndows NT and they should executed directly or using ftt_run_test shell script
[3] This shell script is not ported to Windows NT since the is only one tape drive device
[4] This script wasn't ported to Windows NT
[5] with the exemption of describe and rate scripts
[6] Some device drivers (e.g., IRIX) won't even issue commands on the SCSI bus if it thinks they will fail. For example, if you read until end of tape, the driver remembers this and won't even issue future reads to the drive. We originally thought that we might need to read even though the device driver thought we shouldn't, so we added ftt_all_scsi. This will force all drive operations to use the raw SCSI device driver, guaranteeing that the command would actually be sent to the drive. As it turned out, we didn't need this feature.
[7] If ftt is ever confused about tape position (e.g., a skip filemark command fails), then it sets an internal flag that prevents any future tape movement. We believed that while we were looking for good data in the recovery process, we might need this feature. It's not clear that it was really needed.