Enstore release notes, from RELEASE-2008-09-17 to RELEASE-2008-10-09

Enstore changes:
=============


Misc.:
======


                      Detailed cvs commit logs


========== drive_utilization_plotter_module.py ====================================================================================

points -> impulses 

========== inventory.py ====================================================================================

The string function find() returns -1 when a string is not found, that should have been used instead of index() which always throws a traceback. 

Scrubbed the inventory script good. Most generated files now in a consistant form list the date & time the page was generated and that inventory created it. The CMS_VOLUMES_WITH_ONLY_DELTED_FILES page has been moved to its own script. Migrated files only show up on the MIGRATED_VOLUMES page and not also on the RECYCLEABLE_VOLUMES page. 

========== enstore_show_inventory_cgi.py ====================================================================================

Fixed a minor bug where a non-defined variable was being used. 

Hide the complete file listing files per storage group from the inventory and inventory summary pages. These are pointed to through yet another cgi. 

Rewrote the inventory and inventory summary sgi scripts to not display a hard-coded list of (special) files located in the tape_inventory directory. Instead the contents of the tape_inventory directory are searched, and every file that does not match a volume name is automatically put into the list of (special) files to display. The summary version imports the non-summary version and uses what it needs; thus avoiding a lot of duplicate code. 

========== duplication_util.py ====================================================================================

Fixed a bug when /pnfs/fs/usr/ and /pnfs/ paths to the same file where being failed for swapping. Also, patched the make_original_as_duplicate() and make_migrated_as_duplicate() to return success or failure like their callers expect. 

========== library_manager_client.py ====================================================================================

Encp will now use encp_port from the library's configuration for communicating with a library manager. If encp_port is not defined, then the normal port value from the configuration is used. 

========== make_original_as_duplicate.py ====================================================================================

Cleaned up and scrubbed the code. Should actually work now. 

========== duplicate.py ====================================================================================

Removed --restore from the duplication help, since it only works with migration. 

Purge --show-summary. A better display is generated from duplication_summary. 

========== option.py ====================================================================================

Make the tape_inventory directory the default output directory. Add the --duration and --output-dir swiches to control the output. 

Added --read-to-end-of-tape to the migration. 

========== migrate.py ====================================================================================

One more change to the LIKE issue with underscores and _copy_. Also, purged show-summary from the code. 

Added a new function log_uncopied() that clears the pair of bfids from the migration table. This function is now used with --restore to allow for a file to (without other intervention) remigrate the file. 

Fixed a problem in get_migration_type() where testing for duplication volumes was matching to many things becuase the like operator was matching underscores as any single character instead of a literal underscore. Also, fixed a problem with the testing for migrated volumes where two percent signs (one for python to eat up, the other to be passed to the SQL) were missing for character matching. 

Fixed some issues when gathering media_types to determine if migrating or cloning should be used. The issues was that the Migration directory does not necessarily exist by the time the directory library tag is checked. So, now the code checks parent directories until it reaches the /pnfs/fs/usr/Migration directory. 

Fixed yet another case where /pnfs/sam/zero confuses the code, since /pnfs/fs/usr/dzero does not match to /pnfs/dzero like the pattern that everything else matches to. 

========== recent_file_listing.py ====================================================================================

Fixed a problem with the rename() failing across filesystems. 

Corrected self.out_dir to be self.output_dir. 

Make the tape_inventory directory the default output directory. Add the --duration and --output-dir swiches to control the output. 

========== encp.py ====================================================================================

Encp will now use encp_port from the library's configuration for communicating with a library manager. If encp_port is not defined, then the normal port value from the configuration is used. 

Fixed a bug in create_zero_length_pnfs_files() where an IOError was not caught, but OSErrors are. Obviously, IOErrors are now caught too. 

Fixed the help. The parameters for admin and user2/dcache modes show correct usage. This is the inclusion of --get-bfid, --get-cache and --put-cache. 

========== ratekeeper.py ====================================================================================

catch exception - print the error, debugging... 

remove debug output 

fix typo 

fill storage group column in drive_utilization table 

========== scanfiles.py ====================================================================================

Better handling of multiple copies. 

========== swap_original_and_copy.py ====================================================================================

Cleaned up and scrubbed the code. Should actually work now. 

A few more silly fixes from Tracebacks. 

Scrubbed the swapping code. 

========== plotter_main.py ====================================================================================

fix "-h" and "--help" option output 

========== migration_summary_plotter_module.py ====================================================================================

Instead of blindly displaying 3480, try and show LTO1 and LTO2 seperately. Include the name of the Enstore instance, if available, from the configuration. Changed the calculations behind the goal line to be more accurate. 

Attempt to make the started values more correct. 

Included a better goal adjustment for dealing 9940A that are clobbered into 9940Bs. Also, added a extra sorting select statement to make sure that the days are in chronological order. 

========== pnfs.py ====================================================================================

Corrected a call to the list index() function that had two arguments, instead of an expected one argument. 

When looking for the mounted /pnfs/fs/usr, /pnfs/fnal.gov/usr or /pnfs/ filesystem, return OSError(ENOENT) instead of type error in nothing is found. 

Fixed yet another case where /pnfs/sam/zero confuses the code, since /pnfs/fs/usr/dzero does not match to /pnfs/dzero like the pattern that everything else matches to. 

========== enstore_show_inv_summary_cgi.py ====================================================================================

Rewrote the inventory and inventory summary sgi scripts to not display a hard-coded list of (special) files located in the tape_inventory directory. Instead the contents of the tape_inventory directory are searched, and every file that does not match a volume name is automatically put into the list of (special) files to display. The summary version imports the non-summary version and uses what it needs; thus avoiding a lot of duplicate code. 

========== make_migrated_as_duplicate.py ====================================================================================

Cleaned up and scrubbed the code. Should actually work now.