Nathan, this is a very well-considered and well-composed contribution!
Kudos!
I have a couple of thoughts, neither of which /require/ action on your part.
* If you can stand to bump the version requirement to 1.3, you can
optionally drop all the pool stuff from the script (and let
Python manage the memory usage itself).
* It's a touch embarrassing that this kind of functionality isn't
in 'svnadmin' proper.
There is a matter that *does* require your attention, though. What were
you envisioning as the license on this contribution? Obviously, we'd
prefer you use the same license that Subversion does, and failing that
most of us would probably advise away from the GPL and toward something
that's Apache/BSD-ish, but the call is yours to make.
Thanks for the contribution.
Nathan Kidd wrote:
> Hi,
>
> The latest comments by cmpilato on issue #2507 (
> http://subversion.tigris.org/issues/show_bug.cgi?id=2507 ) indicate that
> a proper code fix for the "zombie lock" problem is not easy/imminent.
> Therefore I have written remove-zombie-locks.py (attached) as a
> workaround. I believe this script will be very useful for anyone that
> deals with locks regularly, and hope it can be included in contrib/.
>
> remove-zombie-locks.py has two modes:
> 1) it can be run stand-alone to clean out all existing zombie locks
> (that is, locks on files deleted in HEAD) from the whole repository
>
> 2) it can be run as a post-commit hook to ensure that files deleted in a
> revision have any corresponding locks removed
>
> Notes:
>
> - I have tested the script on GNU/Linux with SVN 1.2 Python bindings,
> and on Win32 with SVN 1.3 Python bindings.
>
> - The pool and/or API usage may not be ideal; I have little experience
> with them. Criticism welcome.
>
> - Whole-repository lock iteration has quite poor performance, I assume
> because of the many stats required to walk the db/locks/* structure.
> This script's "all" mode will run several times faster on a GNU/Linux
> file system than Win32, because of NTFS's horrific many-file stat
> performance. My db/locks/ dir has 3500 immediate subdirectories, 2500
> actual locks. Iterating them takes 30s on GNU/Linux, 1:30s on Win32.
> The effect on the hook-script mode is negligible when iterating the
> normally-few file deletions of a typical commit.
>
> Regards,
>
> -Nathan
> ________________________
> Nathan Kidd
> Builds Engineer
> Hummingbird Ltd., A division of Open Text Corporation
>
>
> ------------------------------------------------------------------------
>
> #!/usr/bin/env python
>
> #
> # Copyright (C) 2007 Hummingbird Ltd.
> #
> # This program is free software; you can redistribute it and/or
> # modify it under the terms of the GNU General Public License
> # as published by the Free Software Foundation; either version 2
> # of the License, or (at your option) any later version.
> #
> # This program is distributed in the hope that it will be useful,
> # but WITHOUT ANY WARRANTY; without even the implied warranty of
> # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> # GNU General Public License for more details.
> #
>
> """\
>
> remove-zombie-locks.py - remove zombie locks on deleted files
>
> This script is a workaround for Subversion issue #2507
> http://subversion.tigris.org/issues/show_bug.cgi?id=2507
>
> Usage: remove-zombie-locks.py REPOS-PATH <REVISION|all>
>
> When REVISION (an interger) is specified this script scans a commited
> revision for deleted files and checks if a lock exists for any of these
> files. If any locks exist they are forcibly removed.
>
> When "all" is specified this script scans the whole repository for
> locks on files that don't exist in the HEAD revision, removing any found.
>
> Usage Examples:
>
> As a post-commit Hook script to prevent zombie locks:
> remove-zombie-locks.py /var/svn/myrepo 6174
>
> To clean a repository with existing zombie locks:
> remove-zombie-locks.py /var/svn/myrepo all
>
> For additional information read the commented notes in this script.
> """
> #
> # ** FAQ **
> #
> # What is the problem, exactly?
> #
> # When you commit a file deletion with the --keep-locks option then though the
> # file is deleted in the repository, the lock is not. This is a bug in Subversion.
> # If locks are left on deleted files then any future attempt to add a file of
> # the same name or delete/move/rename any parent directory will fail.
> # This is very difficult for the end-user to fix since there's no easy way to
> # find out the names of these zombie locks, and it is not simple to remove them
> # even if you do know the names.
> #
> #
> # Is this script 100% safe?
> #
> # There is a theoretical and very small chance that before this script runs
> # another commit adds a file with the same path as one just deleted in this revision,
> # and then a lock is aquired for it, resulting in this script unlocking the
> # "wrong" file. In practice it seems highly improbable and would require very strange
> # performance characteristics on your svn server. However, to minimize the window for
> # error it is recommended to run this script first in your post-commit hook.
> #
> #
> # How Do I Start Using This Script?
> #
> # 1. One time only, run this script in 'all' mode to start your repo out clean
> # 2. Call this script from your post-commit hook to keep your repo clean
> #
>
> import os
> import sys
>
> import svn.core
> import svn.repos
> import svn.fs
>
> assert (svn.core.SVN_VER_MAJOR, svn.core.SVN_VER_MINOR) >= (1, 2), \
> "Subversion 1.2 or later required but only have " \
> + str(svn.core.SVN_VER_MAJOR) + "." + str(svn.core.SVN_VER_MINOR)
>
> def usage_and_exit():
> print >> sys.stderr, __doc__
> sys.exit(1)
>
> class RepositoryZombieLockRemover:
> """Remove all locks on non-existant files in repository@HEAD"""
> def __init__(self, repos_path, repos_subpath=""):
> self.repos_path = repos_path
> self.repos_subpath = repos_subpath # it's possible to only clean part of the repo
>
> # init svn
> svn.core.apr_initialize()
> self.pool = svn.core.svn_pool_create(None)
> self.repos_ptr = svn.repos.open(self.repos_path, self.pool)
> self.fs_ptr = svn.repos.fs(self.repos_ptr)
> self.rev_root = svn.fs.revision_root(self.fs_ptr, \
> svn.fs.youngest_rev(self.fs_ptr, self.pool), self.pool)
>
> def __del__(self):
> svn.core.svn_pool_destroy(self.pool)
> svn.core.apr_terminate()
>
> def unlock_nonexistant_files(self, lock, callback_pool):
> """check if the file still exists in HEAD, removing the lock if not"""
> if not svn.fs.svn_fs_is_file(self.rev_root, lock.path, callback_pool):
> print lock.path
> svn.repos.svn_repos_fs_unlock(self.repos_ptr, lock.path, lock.token, True, \
> callback_pool)
>
> def run(self):
> """iterate over every locked file in repo_path/repo_subpath,
> calling unlock_nonexistant_files for each"""
>
> print "Removing all zombie locks from repository at %s\n" \
> "This may take several minutes..." % self.repos_path
> svn.fs.svn_fs_get_locks(self.fs_ptr, self.repos_subpath, \
> self.unlock_nonexistant_files, self.pool)
> print "Done."
>
>
> class RevisionZombieLockRemover:
> """Remove all locks on files deleted in a revision"""
> def __init__(self, repos_path, repos_rev):
> self.repos_path = repos_path # path to repository on disk
>
> # init svn
> svn.core.apr_initialize()
> self.pool = svn.core.svn_pool_create(None)
> self.repos_ptr = svn.repos.open(self.repos_path, self.pool)
> self.fs_ptr = svn.repos.fs(self.repos_ptr)
> self.rev_root = svn.fs.revision_root(self.fs_ptr, repos_rev, self.pool)
>
> def __del__(self):
> svn.core.svn_pool_destroy(self.pool)
> svn.core.apr_terminate()
>
> def get_deleted_paths(self):
> """return list of deleted paths in a revision"""
> deleted_paths = []
> for path, change in svn.fs.paths_changed(self.rev_root, self.pool).iteritems():
> if (change.change_kind == svn.fs.path_change_delete):
> deleted_paths.append(path)
> return deleted_paths
>
> def run(self):
> """remove any existing locks on files that are deleted in this revision"""
> deleted_paths = self.get_deleted_paths()
> for path in deleted_paths:
> lock = svn.fs.svn_fs_get_lock(self.fs_ptr, path, self.pool)
> if lock:
> # print path
> svn.repos.svn_repos_fs_unlock(self.repos_ptr, path, lock.token, True, self.pool)
>
>
> def main(repos_path, repos_rev):
> repos_path = os.path.abspath(repos_path)
> if repos_rev.lower() == "all":
> remover = RepositoryZombieLockRemover(repos_path, "")
> else:
> repos_rev = int(repos_rev)
> remover = RevisionZombieLockRemover(repos_path, repos_rev)
>
> remover.run()
>
> sys.exit(0)
>
>
> if __name__ == "__main__":
> if len(sys.argv) < 3:
> usage_and_exit()
> else:
> main(sys.argv[1], sys.argv[2])
>
>
>
>
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
--
C. Michael Pilato <cmpilato@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Received on Tue Jan 9 21:29:33 2007