"David Glasser" <email@example.com> writes:
> On 8/17/07, firstname.lastname@example.org <email@example.com> wrote:
> > -def shard(path, max_files_per_shard):
> > - """Move all the files in PATH into subdirectories of PATH named such that
> > - subdirectory '0' contains at most MAX_FILES_PER_SHARD files, those named
> > - [0, MAX_FILES_PER_SHARD). Abort if PATH is found to contain any entries
> > - with non-numeric names."""
> > +def shard(path, max_files_per_shard, start, end):
> > + """Move the files for revisions START to END inclusive in PATH into
> > + subdirectories of PATH named such that subdirectory '0' contains at most
> > + MAX_FILES_PER_SHARD files, those named [0, MAX_FILES_PER_SHARD). Abort if
> > + PATH is found to contain any entries with non-numeric names."""
> > - # Move all entries into shards named N.shard.
> > - for name in os.listdir(path):
> > - try:
> > - rev = int(name)
> > - except ValueError, OverflowError:
> > - print >> sys.stderr, "error: file '%s' does not represent a revision." \
> > - % os.path.join(path, name)
> > - sys.exit(1)
> I think this breaks the guarantee in the doc string that it will abort
> if a non-numeric entry is found. (I think that's reasonable? But the
> doc string should be changed at least.)
Yeah, just update the doc string.
> Is this a common Pythonism? "if skipped == 0: os.rmdir(tmp)" certainly
> makes me think less. Also, the log message implies that parallel
> reshardings are reasonable, but it doesn't seem to me that this will
> work (it definitely will be strange if the parallel reshardings would
> affect the same shard, but even if they are building distinct shards,
> this loop doesn't make sure that the shard it is installing is in the
> range that it processed, so it might install an incomplete shard from
> another process).
It works if your parallel jobs are using the correct ranges, as
mine do. We should add a check to ensure this.
Eric Gillespie <*> firstname.lastname@example.org
To unsubscribe, e-mail: email@example.com
For additional commands, e-mail: firstname.lastname@example.org
Received on Sat Aug 18 00:59:04 2007