ENCP release notes, from v3_7 to v3_8 Encp changes: ============= If encp is run as root, the temporary ownership of the output file will match the owner of the directory. It is changed to the owner of the source file when the transfer successfully completes. Known encp defects: =================== The --version string is "v3_7d CVS $Revision$". It should be "v3_8 CVS $Revision$ ". This version (along with v3_7c and v3_7d), have a bug where encp always returns success, even if the transfer ws a failure. Encp versions v3_7c, v3_7d and v3_8 have been pulled from KITS. Misc.: ====== Detailed cvs commit logs ========== dispatching_worker.py ==================================================================================== added rawUDP module. Just set use_raw=1 in dispatchong_worker.__init__ _do_print added to enable printing from inside of the program ========== configuration_client.py ==================================================================================== removed some traces ========== file_utils.py ==================================================================================== More fixes to get encp to work with untrusted pnfs clients as user root. Added match_euid_egid() to file_utils.py. This is now used by encp.py, delete_at_exit.py and migrate.py to allow root to access pnfs on a non-trusted pnfs file system. ========== pnfs.py ==================================================================================== Modified to support the encp regression testing using pnfs_agent in addition to a directly mounted pnfs mount point. Added PATH_MAX to the module. Its value is 199. ========== generic_client.py ==================================================================================== Send the check_ticket() BAD STATUS message to standard error. ========== option.py ==================================================================================== Modified to support the encp regression testing using pnfs_agent in addition to a directly mounted pnfs mount point. added queue-length LM option ========== library_manager.py ==================================================================================== added queue-length LM option and use_raw ========== delete_at_exit.py ==================================================================================== Before removing a left over file, match the euid and egid of the directory not the file. If the directory is owned by a different uid/gid the unlink() will fail. Added match_euid_egid() to file_utils.py. This is now used by encp.py, delete_at_exit.py and migrate.py to allow root to access pnfs on a non-trusted pnfs file system. ========== encp.py ==================================================================================== Modified to support the encp regression testing using pnfs_agent in addition to a directly mounted pnfs mount point. Handle OSError tracebacks from file_utils.match_euid_egid() in read_hsm_file() and write_hsm_file(). Fixed bug introduced with 1.909 where submit_one_request_send() returns a different number of objects. More fixes to get encp to work with untrusted pnfs clients as user root. Added a debug log comment to better understand the "Noticed the local file changed" errors. Address some issues with testing if we have the permisions and setting the owner when writing from dcache instead of like normal. Fixed the ownership of file creation when running as root. Now, the file's temporary ownership os that of the owner of the directory that the file will go into. Then as part as set_outfile_permissions() if the real UID is root's then the ownership is set to that of the input file. Fixed bug introduced in encp.py 1.912. Added match_euid_egid() to file_utils.py. This is now used by encp.py, delete_at_exit.py and migrate.py to allow root to access pnfs on a non-trusted pnfs file system. ========== enstore_start.py ==================================================================================== do not print interfaces list some changes to accommodate MAC OS ========== pnfs_agent.py ==================================================================================== Modified to support the encp regression testing using pnfs_agent in addition to a directly mounted pnfs mount point. fixed typo replaced Trace.log for INFO with Trace.trace threaded implementation added uid and gid to mkdir added uid and gid to mkdir fixed mkdir added tag setting methods added tag setting methods chanegd return code in mkdir added mkdir Use pnfs.Pnfs() correctly inside of creat(). ========== udp_server.py ==================================================================================== conditional import of rawUDP added rawUDP module. Just set use_raw=1 in dispatchong_worker.__init__ ========== migrate.py ==================================================================================== If the temporary migration file is deleted, don't try attempt checks to layer 4 while verifying the swap was actually done during a previous execution of the migration. Before removing a left over file, match the euid and egid of the directory not the file. If the directory is owned by a different uid/gid the unlink() will fail. Since 1.134, there has been a bug where the layers are not clobbered before removing the migration file after the file is scanned. Now the layers are clobbered. There was a way that final_scan_file() could encounter an undefined variable; pnfs_path was changed to likely_path. Also, spelling fix; sinnce to since. Workaround the situation were Queue.get() returns None that is not in the queue. This modification changes the sentinel value from None to "SENTINEL". Don't use a failed stat()s variable to check if a file needs to be created. Re-stat() the file after it is created to double check. Report error and exit in situation where root can not make a directory because the node is not a trusted node to PNFS. Also, cleaned up some old and commented out code lines. Removed "--verbose 4" for debug output. Added match_euid_egid() to file_utils.py. This is now used by encp.py, delete_at_exit.py and migrate.py to allow root to access pnfs on a non-trusted pnfs file system. Changed Trace.init("MIGRATION") to Trace.init(MIGRATION_NAME) so that cloning is reported correctly. ========== udp_client.py ==================================================================================== do not send done-cleanup it is done internally in dispatching worker Modified recv_deferred() to handle taking a list (or still a single transacion id) and returning the first matching response. ========== library_manager_client.py ==================================================================================== added queue-length option ========== mover.py ==================================================================================== If write failed due to interruption on the client side in a single fm mode mover would error out with wrong thape position when writing a tape mark dismounting tape. It also was setting tape to NOACCESS. This patch fixes the problem. If FTTError exception do not set mover offline after write or read, but still send an error and dismount tape. removed diagnstics used for debugging This patch addresses a problem seen at PIC. During read of tapes imported from Castor there were cases when some files have been successfuly read, but then ftt.tell() generated FTTError exception. After that one of mover threads died, and mover had to get restarted manually. Now this exception is processed correctly. ========== encp_rate_multi_plotter_module.py ==================================================================================== If the database returns a garbage record, ignore it. ========== pnfs_agent_client.py ==================================================================================== added uid, gid to p_mkdirs Modified to support the encp regression testing using pnfs_agent in addition to a directly mounted pnfs mount point. fixed mkdir set gid uid inside of mkdir added uid and gid to mkdir fix typo added tag setting methods added tag setting methods ========== enstore_functions2.py ==================================================================================== In fullpath2() provide an optional arguement for the caller to avoid having the the path split into the directory and basename. To perform the split a stat() call must be made, which the caller may not want.