ENCP release notes, from v3_4 to v3_5
Encp changes:
=============
Fixed encp so that the --no-crc switch actually works.
Fixed a bug that could cause the transfer to hang until the mover timedout.
The transfers would retry, but with the fix the retires are started immediately
instead of after the timeout.
Added the ability for encp to obtain the filesystem quota information for the
user. If the read transfer will put them over their soft limit, a warning will
be printed to standard error. If the read transfer will put the user over
their hard limit the transfer will be aborted. (NOTE: This feature is broken
in encp v3_5, but has been fixed in encp v3_5a.)
Added a new encp switch: --check. Using encp with this switch will cause
encp to check if there are any up-front errors that would cause an actual
transfer (without --check) to fail. If no problems are found, an exit status
of 0 is returned. If there is a problem, then an exit status of 1 is returned.
If "encp --check" is unable to determine if it would or would not succeed; an
exit status of 2 is returned. Reasons for this possibility include, but not
limited to: Enstore is down for maintenance or a library is paused.
Fixed a --data-access-layer defect from v3_4. Upon a successful transfer
encp was leaving most of the data-access-layer fields blank.
Misc.:
======
A number of network failure diagnostics have been added. This is in response
to some intermittent errors that occur on some nodes.
Added --file to "enstore info". This command prints out the file DB
information on the specified file. The command will look like:
enstore info --file <filename>
enstore info --file <bfid>
enstore info --file <pnfsid>
enstore info --file <volume:loc>
If passed a bfid, the result will be exactly like that of the
"enstore info --bfid" command.
A script is now included in the encp product: en_check. The dCache has a
script called dc_check that reports if the specified file is on dCache
disk or not. en_check is the Enstore equivalent, but reports if a file
is on tape or not. It returns 0 if it is on tape or 1 if there is an error
found. The most likely reason for an exit status of 1 to be returned is that
the file is not on tape (yet). There is the possibility that it is unable to
determine if it is or is not on tape at the time it was executed; in
these situations it returns an exit status of 2. If you have a concern
that en_check does not indicate that a file is on tape after a "reasonable
amount of time" do not panic, do not remove any files and contact the
Enstore administrators with your concern.
Detailed cvs commit logs
========== Trace.py ====================================================================================
Fixed an error message. The message is supposed to mention the function that it originates from, default_alarm_func. However, the error string was "default alarm_func".
========== accounting_client.py ====================================================================================
add volume to encp_error
add more field to encp_xfer and encp_error
add file_family and wrapper to encp_xfer
========== accounting_query.py ====================================================================================
fix port and logname
========== aml2.py ====================================================================================
diagnostics for new_library failure
diagnostics for new_library failure
fixed not complete return statement in aci_drivestatus2
========== atomic.py ====================================================================================
Undue a modification that attempted to speed up the opening of a file. If the file already existed it should exit, however there exists a special case that EEXIST may be returned. This change puts it back the way it was with a comment describing why.
Handle EEXIST errors when linking the temporary name the the real name. Instead of trying again (like other errors due) this one causes it to give up immediatly.
========== callback.py ====================================================================================
Added support for ioctl(FIONREAD) to actually work from within python.
Added code to handle printing the size of the socket queues when an unexpected error will occur. Unfortunatly, python doesn't support this functionality at this time, but when it does this will work.
Added getsockopt() calls to timeout_recv() to obtain the amount of data still in the sockets buffer queues when the tcp connection becomes 'broken'.
Added code (for linux) that will get the socket state from /proc when the file descriptor becomes unusable.
Seperate out the "bad bytecount" error into "bad bytecount" when the type is not string and "wrong bytecount" when the number of bytes is not 8.
========== charset.py ====================================================================================
Added function is_string_in_character_set(). This allows someone to pass in the character set to compare against.
========== checkdb.py ====================================================================================
turn off listening to localhost
keep cases of storage group
take care of '\n'
record more time stamp
make output tab delimited
skip (num); add total for each
skipping the last lines which are not valid data
correcting a typo
split listings
take care of two sequencers
skip indexes
========== configuration_client.py ====================================================================================
Moved --show from using VALUE_LABEL to the new DEFAULT_LABEL. Otherwise the help output was wrong.
If an error occured with --show the error was printed twice (bug from previous patch). The error is handled correctly now.
Allow the "enstore config --show" command to print out the specified information in the config file more than one level deep.
Moved the Trace.trace() output from level 13 to 23. These message are about when configuration information comes from the configuration server or the cached local version.
========== e_errors.py ====================================================================================
allow rejecting encp requests for restrict_host_access
Added NO_VOLUME AND NO_BFID as non-retriable errors.
fixing e_errors.ERROR in return status
========== encp.py ====================================================================================
bumping version to v3_5 because of encpCut
Fixed encp so that the --no-crc switch actually works.
This change makes sure that two encps don't clobber each other when writing to tape. Theoretically this collision is only possible with dcache writes, but this change addresses the "what if" it happend form normal writes too.
Fixed a bug with --copies. If --copies was used, there was a problem with the copies failing the newly introduced layer 1 and layer 4 existence. Removed two other debugging print lines that were still in the code.
Added a check in handle_retries() to see if layer 1 and/or layer 4 was set. This check is activated in two places (for writes only): 1) Just before transfer begins. 2) Just before setting the pnfs (and file db) metadata.
When an error comes from EXfer close the data socket. Otherwise we run into a situation where the mover is waiting on the data socket and encp is waiting on the control socket.
Modied encp to use the EXfer module's new quota capability. When a transfer will exeed a users quota the transfer will be aborted. If the transfer will exeed the users soft quota then a warning message will be printed to standard out.
Added support for the encp option --override-deleted. This will allow a user to use --get-bfid and --override-deleted to read any file on a tape.
In the check_crc() function, log the mover CRC and the encp CRC. Previously, only errors were reported. This patch is in responce to a Scientific Linux 3.0.3 with XFS using direct i/o bug. When reading a file larger than 4GB using direct i/o after the 4GB point the kernel/filesystem retrieves the wrong bytes. If it wasn't for a check against the CRC dCache/dccp put into layer 2 we would have stored a corrupted file on tape that would have been missed by CRC checks.
Log the dictionary/ticket passed into is_read() before raising the "Inconsistant file types" exception. This is for debugging purposes.
Reworked how set_pnfs_settings() works for copy transfers. The layers, file size and permsissions should never be changed when a copy is written.
Modified encp to be able to handle using .(access)() filenames on dCache writes. This avoids intensive name lookups to pnfs.
Added a Trace.log() message. In open_control_socket() if a second message is read in this message is logged.
Fixed the --shortcut option for dcache reads. Modified encp to allow --shortcut to work for writes. Added option --override-path to allow the user to specify a path when --shortcut is used. CV: ------------------------------------------
Catch an EncpError that should not happen in the request dictionary was put together correctly.
Fixed a bug that prevented the pnfsid to be set in the filedb for copies.
Added original_bfid to the file clerk subticket. (It should have already been there). Place the copies in the master list of write requests. Before they were stored inside the ticket of the original request. This change was motivated when printing the requests for the original request were being cut off. Fixed a bug that compared the conanical path with a hard coded first 19 characters of the path. 19 works for /pnfs/fnal.gov/usr/ but not in the general case. Now it uses len(conanical_path).
Moved the copy.deepcopy() call to inside the loop when creating the requests for copies. A traceback was occuring with the copy before the loop becuase the file_family was subticket entry was already removed.
Fixed a problem at the beginning to recognize when a file is a pnfs or local file w.r.t the check for all three types of paths. The old way had problems with automounting paths not beginning with /pnfs/... Found a few more KEYERROR vs. NO_VOLUME/NO_FILE bugs. The copy numberi, when --copies is used, is relative to this encp. It is not absolute w.r.t. the number of copies of the file in the entire system. Also, the original bfid and original file family are not mangled on the encp side. They will be mangled on the mover side.
Improved --check to return an exit status of 2 for 'temporary errors'. Also, take advantage of the new TIMEDOUT errors, and NO_VOLUME and NO_BFID. Lastly, place into the accounting db the new fields supported by the accounting client (like file_family, volume, drive, etc.).
Modified --verbose with a level less than zero to also turn off the data access layer for both successes and failures. Also, fixed the "Not all input files are %s files" message to correctly list /pnfs/... or unix.
Fixed two bugs with respect to --data-access-layer. First, was that successful transfers with this option enabled would place the success into the encp_error table in the accounting database. Second, on successes the information printed out to the screen with this option enabled was not printing out the correct information (it was leaving most values blank).
Modified the "Volume clerk (%s) knows nothing about %s.\n" messages. If the volume clerk returned a KEYERROR then the old error is still logged. If a different error is returned then, "Failure communicating with volume clerk..." is logged instead.
Attempt to handle an exception. If the output file did not exist when trying to get the directory, an OSError occured. It is now caught inside of the log_encp_start() function.
========== encp_ticket.py ====================================================================================
changes for multiple copy
========== alarm_server.py ====================================================================================
Moved two functions from enstore_functions2.py to enstore_mail.py. These functions are format_mail() and send_mail(). The modifications to inquisitor.py and alarm_server.py are to reflect the new module these functions are in.
========== enstore_html.py ====================================================================================
display alarms on page latest to earliest
========== enstore_overall_status.py ====================================================================================
Modified to use the new enstore_mail.py module for the send_mail() function. It used to be in the enstore_functions2.py module.
========== enstore_plots.py ====================================================================================
better implementations of xfer sizes per storage group
========== enstore_files.py ====================================================================================
better implementations of xfer sizes per storage group
========== enstore_saag.py ====================================================================================
fix bug to let config server appear in nodes list
========== enstore_show_inv_summary_cgi.py ====================================================================================
change COMPLETE_FILE_LISTING
========== enstore_show_inventory_cgi.py ====================================================================================
change COMPLETE_FILE_LISTING
========== enstore_up_down.py ====================================================================================
change wait time
add info server, accounting server and drivestat server
========== file_clerk.py ====================================================================================
bug fix
fixing e_errors.ERROR in return status
========== file_clerk_client.py ====================================================================================
fixing e_errors.ERROR in return status
========== ftt_driver.py ====================================================================================
raise an exception it tell if there is FTT_UNRECOVERED
========== info_client.py ====================================================================================
add --file
correct a typo
fixing e_errors.ERROR in return status
========== info_server.py ====================================================================================
like history with vol.deleted
sort bfids according to location_cookie
change some INFO to ERROR
cosmetic touch
fix a typo
add --file
fixing e_errors.ERROR in return status
========== inquisitor.py ====================================================================================
Moved two functions from enstore_functions2.py to enstore_mail.py. These functions are format_mail() and send_mail(). The modifications to inquisitor.py and alarm_server.py are to reflect the new module these functions are in.
========== inquisitor_plots.py ====================================================================================
check number of storage groups so cdfen and d0en are not confused
1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py empty log message 1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py
better implementations of xfer sizes per storage group
========== inventory.py ====================================================================================
do not close closed databse any more
auto format
add up recycleable and migrated
bug fixing
bug fixing
fix a typox
add recycleable volumes and migrated volumes
========== library_manager.py ====================================================================================
back off changes in mover_error, this need morer thinking
modified mover_error to delete request from volumes_at_movers
Modifications for multiple file copies
allow rejecting encp requests for restrict_host_access
allow rejecting encp requests for restrict_host_access
allow rejecting encp requests for restrict_host_access
========== mounts_plot.py ====================================================================================
remove wrapper = 'null' check
========== mover.py ====================================================================================
set volume readonly of crc check fails
There was one case when mover did not execute sys.exit in restart. As a result the process stayed alive and kept retrying to restart after exceptions. Added alarm to alert about this case.
fixed a bug
add diagn. for reads
fixed a bug in start(). If dismount failed dispatching worker did not get instantiated,m causing exception and restart in a loop.
diagnostics for stuck stream writes
1. Fixed send_client_done to close sockets 2. fixed unload_volume to correctly exit afted dimount failure 3. Fixed start to go offline in case of precautionary dismount failure.
added multiple copy functionality
if FTT_EIO during write do not set tape NOACCESS as it is already set to readonly
set mover offline and volume noaccess if FTT_EIO occurs
fixed mover to set volume to NOACCESS when FTT_EUNRECOVERED occurs in seek_to_location
fix indentation problem
more code to properly process FTT_EUNRECOVERED in seek_to_location
set tape to noaccess if there is FTT_UNRECOVERED error
return to IDLE when in setup there is a wrong wrapper
error out if encounter FTT_EIO
========== net_driver.py ====================================================================================
added diagnostics
========== option.py ====================================================================================
add --clear-sg
Added support for the encp option --override-deleted. This will allow a user to use --get-bfid and --override-deleted to read any file on a tape.
For some switches that take an unknown number of arguments from intf.args this patch fixes a bug in formating the --help output. The DEFAULT_LABEL string was added to force a string into the help output that would otherwise not be there.
Made a change for switches that process an unknown number of arguments via intf.args list to override the (empty) argment portion of the help string with the VALUE_NAME string.
Fixed the --shortcut option for dcache reads. Modified encp to allow --shortcut to work for writes. Added option --override-path to allow the user to specify a path when --shortcut is used. CV: ------------------------------------------
Renamed the encp switch --copy to --copies.
put FILE in valid option list
add file option
Added --file-threads and --infile for scanfiles.py.
========== plotter.py ====================================================================================
1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py empty log message 1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py
better implementations of xfer sizes per storage group
1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py empty log message 1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py
1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py empty log message 1 accounting_client.py accounting_client.pyc accounting.py accounting.py~ accounting.pyc accounting_query.py accounting_query.pyc accounting_server.py accounting_server.py~ accounting_server.pyc aci.py alarm_client.py alarm_client.pyc alarm.py alarm.pyc alarm_server.py alarm_server.py~ alarm_server.pyc aml2_log.py aml2.py atomic.py atomic.py~ atomic.pyc backup_backup.py backup_client.py backup_client.pyc backup.py backup.pyc bfid_db.py callback.py callback.py~ callback.pyc cern_wrapper.py change_loc_cookie.py change_s_i.py charset.py charset.py~ charset.pyc checkdb.py checkdbs.py check_pnfs_db.py cleanUDP.py cleanUDP.py~ cleanUDP.pyc configuration_client.py configuration_client.py~ configuration_client.py.bak configuration_client.pyc configuration_server.py configuration_server.py~ configuration_server.pyc cpio_odc_wrapper.py create_sg_db.py CVS db_compare.py db_dump.py db.py db_retrieve_backup.py dbs.py dcache_make_queue_plot_page.py delete_at_exit.py delete_at_exit.pyc discipline.py discipline.pyc disk_driver.py disk_driver.pyc dispatching_worker.py dispatching_worker.py~ dispatching_worker.pyc drivestat2.py drivestat2.pyc drivestat_client.py drivestat_client.pyc drivestat_server.py drivestat_server.pyc edb.py edb.pyc e_errors.py e_errors.py~ e_errors.pyc ejournal.py ejournal.pyc encp_admin.py ENCPBIN encp.py encp.py~ encp.py.bak encp.pyc encp_ticket.py encp_ticket.pyc encp_user.py encp_wrapper.py enmail.py enmv.py enmv.py~ enstore_admin.py enstore_alarm_cgi.py enstore_alarm_search_cgi.py enstore_constants.py enstore_constants.pyc enstore_display.py enstore_display.py~ enstore_display.pyc enstore_display.py.test enstore_erc_functions.py enstore_erc_functions.pyc enstore_file_listing_cgi.py enstore_files2.py~ enstore_files2.pyc enstore_files3.py~ enstore_files3.pyc enstore_files.py enstore_files.py~ enstore_files.pyc enstore_functions2.py enstore_functions2.py~ enstore_functions2.pyc enstore_functions.py enstore_functions.pyc enstore_html.py enstore_html.pyc enstore_log_file_search_cgi.py enstore_mail.py enstore_mail.py~ enstore_mail.pyc enstore_make_log_calendar.py enstore_make_plot_page.py enstore_overall_status.py enstore_overall_status.py~ enstore_overall_status.pyc enstore_plots.py enstore_plots.pyc enstore.py enstore.pyc enstore_restart.py enstore_restart.pyc enstore_saag_network.py enstore_saag_network.py~ enstore_saag_network.pyc enstore_saag.py enstore_saag.py~ enstore_saag.pyc enstore_show_inventory_cgi.py enstore_show_inv_summary_cgi.py enstore_start.py enstore_start.pyc enstore_status.py enstore_status.pyc enstore_stop.py enstore_stop.pyc enstore_up_down.py enstore_up_down.py~ enstore_up_down.pyc enstore_user_cgi.py enstore_user.py enstore_utils_cgi.py ensync_admin.py ensync.py ensync_user.py ENTV_BIN entv.py entv.py~ entv.pyc entv.tar esgdb.py esgdb.pyc espion.py estart.py estop.py event_relay_client.py event_relay_client.pyc event_relay_messages.py event_relay_messages.pyc event_relay.py fake_quota.py fake_quota.pyc fdbdump.py file_clerk_client.py file_clerk_client.pyc file_clerk.py file_clerk.py~ file_clerk.py-1.217 file_clerk.pyc fileinfo.py ftt_driver.py ftt.py generic_alarm.py generic_client.py generic_client.py~ generic_client.pyc generic_driver.py generic_driver.pyc generic_server.py generic_server.py~ generic_server.pyc get_all_bytes_counter.py get_cron_title.py get.py get.pyc get_total_bytes_counter.py histogram.py hostaddr.py hostaddr.pyc host_config.py host_config.py~ host_config.pyc idlemovers.py info_client.py info_client.py~ info_client.pyc info_server.py info_server.py~ info_server.pyc inquisitor_client.py inquisitor_client.pyc inquisitor_plots.py inquisitor_plots.py~ inquisitor_plots.pyc inquisitor.py inquisitor.py~ inquisitor.pyc interface.py inventory.py inventory.pyc journal.py label_tape.py library_manager_client.py library_manager_client.pyc library_manager.py library_manager.pyc lintit lm_list.py lm_list.pyc log_client.py log_client.pyc log_finish_event.py log_server.py log_server.pyc log_start_event.py log_trans_fail.py m2.py Makefile Makefile~ make_ingest_rates_html_page.py make_ingest_rates_html_page.py~ makeplot.py makeplot.py~ makeplot.pyc manage_queue.py manage_queue.pyc match_syslog.py media_changer_client.py media_changer_client.pyc media_changer.py media_changer.py~ media_changer.pyc media_changer_test.py migrate.py migration_scope.py module_trace.py monitor_client.py monitor_client.pyc monitored_server.py monitored_server.pyc monitor_server.py mounts_plot.py movcmd_mc.py mover_client.py mover_client.pyc mover_constants.py mover_constants.pyc mover-nanny.py mover.py mover.pyc mpq.py mpq.pyc multiple_interface.py multiple_interface.pyc net_driver.py net_driver.pyc null_driver.py null_driver.pyc null_wrapper.py on-call.py option.py option.py~ option.pyc plotter.py plotter.py~ pnfs_backup_plot.py pnfs.py pnfs.py~ pnfs.pyc pnfs.py.saved priority_selector.py priority_selector.pyc quickquota.py quota.py quota.pyc ratekeeper_client.py ratekeeper_client.pyc ratekeeper.py ratekeeper.pyc ratemeter.py rate_test.py readonly_volumes.py report_volume_quotas.py restore.py rm_volmap.py run1.py safe_dict.py scanfiles.py scanfiles.py~ scanfiles.pyc scanfiles.py.SAVED send_alive.py set_lm_noread.py setpath.py setpath.pyc sg_db.py show_file_cgi.py show_volume_cgi.py stethoscope.py strace_out string_driver.py string_driver.pyc time_fifo.py timeofday.py timeofday.pyc Trace.py Trace.py~ Trace.pyc udp_client.py udp_client.py~ udp_client.pyc udp_common.py udp_common.pyc udp_server.py udp_server.pyc vdbdump.py volume_assert.py volume_assert.py~ volume_assert.pyc volume_clerk_client.py volume_clerk_client.pyc volume_clerk.py volume_clerk.py-1.349 volume_clerk.pyc volume_family.py volume_family.pyc www_server.py www_server.pyc YANKED yank.py
added handling of directory containing encp rate plots per storage group
========== quota.py ====================================================================================
reverse quota and sg_count in outer join so that a quota without allocation will show up correctly
========== scanfiles.py ====================================================================================
deal with missing pnfsid
bug fix
try pnfs_name0 first in reverse scan
change help strings for --bfid and --vol
bug fix
When the library name for a volume is not found in the current configuration, don't just give the error. First, check if "shelf" appears in the library name. Renaming a library as "shelf-" is the convention applied for old media.
conform to errors_and_warnings()
cosmetic change
bug fix
cosmatic change
more on deleted files ... add check_vol()
add --bfid
stkensrv1 has a problem with returning .(get)(database) information. This fix allows it to not traceback.
bug fix
Added a new check to make sure that the volume's library really does exist. Also, added support to accept location cookies for disk movers.
Only include the pool information if there is any. This will prevent empty lists from being outputed.
Include pool information (if available) when a layer 1 or layer 4 error occurs.
First, when stat(1)ing the file if an error is received try a few more times. If the PNFS server is really busy sometimes timeout errors are false interpreted as "does not exist". Second, if there is no layer 1 and no layer 4, we check the stat(1) size and layer 2 size. If both of these sizes are present and have a value of zero, then the informational message of "zero length dCache file not on tape" is outputed.
Make mylint.py happy.
Moved scanfiles to understand e_errors.NO_FILE and e_errors.NO_VOLUME instead of just e_errors.KEYERROR.
Only create a new thread when a new database is found.
Modified the original_pnfs_path warning to be split between renamed and moved. Also, they are not considered warnings, but informational.
Cleanup from the previous commit.
Added a check to compare the parent id of a directory with the id of the parent directory.
Handle situations were we do not have read permissions for the layers, but apparently do for the 'real' file metadata.
========== volume_clerk.py ====================================================================================
bug fix
link history with .deleted
fixing e_errors.ERROR in return status
========== pnfs.py ====================================================================================
bug fix
deal with .(access)(onfsid) files
reverse bug fix
bug fix
allow pnfsid to be set in layer4
Modified encp to be able to handle using .(access)() filenames on dCache writes. This avoids intensive name lookups to pnfs.
Fixed the --shortcut option for dcache reads. Modified encp to allow --shortcut to work for writes. Added option --override-path to allow the user to specify a path when --shortcut is used. CV: ------------------------------------------
add get_normal_pnfs_path
========== udp_client.py ====================================================================================
Fixed a comment. It stated that we need to catch an error when sending 'to' localhost. It should have been 'from' localhost.
Attempt to return the name of the server when a request times out. Errors should now look like (example shone for file clerk): ticket['status'] = (e_errors.TIMEDOUT, enstore_constants.FILE_CLERK) instead of just ticket['status'] = (e_errors.TIMEDOUT, None)
When other threads delete the entry from the tsd ticket the resulting KeyError would print off an error to stderr. This is an understood situation and is now handled differently. Also, Control-C is re-raised.
========== accounting.py ====================================================================================
remove db.get()
back out from previous fix
to get around a stupid PyGreSQL-3.6.2 bug
accDB() now accepts port number
add volume to encp_error
add more field to encp_xfer and encp_error
add file_family and wrapper to encp_xfer
========== accounting_server.py ====================================================================================
turn on/off database debugging at the same time
turn on/off database debugging at the same time
accDB() now accepts port number
bug fix
add volume to encp_error
add more field to encp_xfer and encp_error
add file_family and wrapper to encp_xfer
========== generic_client.py ====================================================================================
For the situation where the config server is not running, a better error message is not provided.
Attempt to return the name of the server when a request times out. Errors should now look like (example shone for file clerk): ticket['status'] = (e_errors.TIMEDOUT, enstore_constants.FILE_CLERK) instead of just ticket['status'] = (e_errors.TIMEDOUT, None)
========== enstore_functions2.py ====================================================================================
Moved two functions from enstore_functions2.py to enstore_mail.py. These functions are format_mail() and send_mail(). The modifications to inquisitor.py and alarm_server.py are to reflect the new module these functions are in.
========== migrate.py ====================================================================================
bug fix for uninitialized dst
bug fix for skipping swapping of copied files
bug fix on scan_queue
remove debugging code
add --priority option
keep location cookie for deleted files
fix is_copied checking order to perserve src and tmp
fix temporary path for deleted files
New functions: --vol-with-deleted # migration with deleted files --use-file-family # use designated file family
add ability to read deleted files
fix missing ')'
allow reading deleted file
connect to database using user enstore
========== volume_clerk_client.py ====================================================================================
add --clear-sg
correct a typo