[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: fsfs-reshard.py - offline FSFS resharding

From: Daniel Rall <dlr_at_collab.net>
Date: 2007-04-18 20:31:38 CEST

On Wed, 11 Apr 2007, C. Michael Pilato wrote:

> Malcolm Rowe wrote:
> > All,
> >
> > I haven't finished testing this yet, but here's a snapshot for review of
> > a tool I was working on over the weekend for offline resharding of an
> > FSFS repository. It can convert between the 1.4 and the (proposed) 1.5
> > format, and is safe to interrupt and re-run.
> >
> > I'd welcome any comments (on the approach, implemention, or my
> > fantastically unPythonic coding style :-)), but this is essentially what
> > I'm planning to provide (in tools/) for people with large repositories
> > who can't spare the time/space to dump/load.
>
> I haven't had a chance to really review the code, but I noted several
> functions that look like error handlers using print() instead of
> sys.stderr.write(). Keep error output on stderr, please.

Alternately:

  print >> sys.stderr, "text"

This obviates the need to include a trailing "\n" in the text.

...
> > I part-converted my copy of the ASF repository to test the
> > restartability, producing 400 shards of 1000 files in about 5-6 minutes;
> > going back the other way has taken about 5-6 hours so far. That's
> > partly down to my filesystem (ext2), but it does mean that restarting a
> > sharding operation is less than ideal, because I currently have that
> > achieved by pre-converting to a linear structure.
>
> Ouch.

I'm much less concerned about slow performance when reverting a
sharded FS to a linear format (unless this points to some sort of bug
past native FS deficiencies). Sharding is a good thing, right?

  • application/pgp-signature attachment: stored
Received on Wed Apr 18 20:32:04 2007

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.