Ashod Nakashian wrote on Wed, Apr 04, 2012 at 11:41:43 -0700:
>
>
>
>
> ----- Original Message -----
> > From: Julian Foad <julianfoad_at_btopenworld.com>
> > To: Ashod Nakashian <ashodnakashian_at_yahoo.com>
> > Cc: Daniel Shahaf <danielsh_at_elego.de>; Markus Schaber <m.schaber_at_3s-software.com>; "mtherieau_at_gmail.com" <mtherieau_at_gmail.com>; Subversion Development <dev_at_subversion.apache.org>
> > Sent: Wednesday, April 4, 2012 9:45 PM
> > Subject: Re: Compressed Pristines (Summary)
> >
> > Ashod Nakashian wrote:
> >
> >> Daniel Shahaf wrote:
> >>>>> Would you like to assess the feasibility of compressing the
> > pristine
> >>>> >store by re-mounting the "pristines" subdirectory as
> > a compressed
> >>>> >subtree in the operating system's file system?
> >>>>
> >>>> No :-)
> >>>>
> >>>> There are two ways to answer this interesting proposition of
> >>>> compressed file-systems. The obvious one is that it isn't
> > something
> >>>> SVN can or should control. The file-system and certainly system
> >>>> drivers are up to the user and any requirement or suggestion of
> >>>> tempering with them is decidedly unwarranted and unexpected from
> >>>> a VCS.
> >>>
> >>> The suggestion wasn't that you teach svn how to change the OS's
> > fs
> >>> settinsg, it was that you investigate how solutions at the OS level
> >>> compare to the other approaches already suggested (custom format -based
> >>> and sqlite-based pristine store).
> >
> > Yes, Daniel, that's exactly what I meant. Sorry, Ash, for not being clear
> > enough.
> >
> > Of course that's inherently a non-portable solution that may not be
> > available on all platforms and certainly will differ. But that's
> > somewhat analogous to support for password storage: in Windows land
> > there's a standard way to do it, in Unixy lands you have to choose your
> > favourite third-party subsystem and externally configure Gnome-keyring or
> > KDE-wallet or whatever you chose. Of course there are other
> > disadvantages too, I just think it would be interesting to compare the
> > disadvantages with the advantages.
> >
> >> That's an easy question. The answer is that at *best* they'll do as
> > good
> >> as in-place compression. However, in practice they'll do much worse.
> >
> > Ash, what's your measure of "best" here -- just the compression
> > ratio? What about other kinds of goodness such as the automatic recovering of
> > free space, and the ability to present a plain-text view of the file content
> > through the standard filesystem API which eliminates the need for us to
> > implement a plain-text cache in Subversion?
> >
> >> The
> >> reason is that the OS level compression works on not only the single file
> > level,
> >> but actually at the block level. This is to make modifications reasonably
> > fast
> >> (read compressed data, uncompress, modify, write recompressed data). If the
> >
> >> complete file is compressed then even changing a single byte (neglecting
> > that no
> >> storage works on the byte-level anyway) will yield performance that will at
> >
> >> least linearly degrade by the filesize.
> >
> > Changing a single byte is an irrelevant scenario, because pristine files are
> > always created in full and then never modified.
>
> But the OS compression doesn't know that! It's design assumes average
> file modification operations, and that's rarely replacing the full
> file (which is the norm for us). I was only explaining and justifying
> the design decision of OS level compression.
>
But one _could_ write a filesystem that implements compression and/or
deduplication aimed at .svn/pristine/'s workflow. Should effort be
spent there rather than on implementing a smarter svn_wc__pristine_*()?
Daniel
(devil's advocate mode)
> >
> >>> In short: if 'mount -o compress=yes' provides 90% space savings
> > then
> >> we
> >>> would have little reason to implement space-saving solutions in svn
> > itself.
> >>> But it's the user's, not svn's, responsibility to run that.
> >
> > By the way, Daniel, I'm not ruling out the possibility that we may want to
> > provide some glue logic so that Subversion can help the user to set up the
> > feature. Without such assistance, only expert users would ever benefit.
> >
> > It's clearly an interesting topic!
> >
> > - Julian
> >
Received on 2012-04-04 20:46:53 CEST