>________________________________
> From: Daniel Shahaf <danielsh_at_elego.de>
>To: Ashod Nakashian <ashodnakashian_at_yahoo.com>
>Cc: Julian Foad <julianfoad_at_btopenworld.com>; Markus Schaber <m.schaber_at_3s-software.com>; "mtherieau_at_gmail.com" <mtherieau_at_gmail.com>; Subversion Development <dev_at_subversion.apache.org>
>Sent: Wednesday, April 4, 2012 10:46 PM
>Subject: Re: Compressed Pristines (Summary)
>
>Ashod Nakashian wrote on Wed, Apr 04, 2012 at 11:41:43 -0700:
>>
>>
>>
>>
>> ----- Original Message -----
>> > From: Julian Foad <julianfoad_at_btopenworld.com>
>> > To: Ashod Nakashian <ashodnakashian_at_yahoo.com>
>> > Cc: Daniel Shahaf <danielsh_at_elego.de>; Markus Schaber <m.schaber_at_3s-software.com>; "mtherieau_at_gmail.com" <mtherieau_at_gmail.com>; Subversion Development <dev_at_subversion.apache.org>
>> > Sent: Wednesday, April 4, 2012 9:45 PM
>> > Subject: Re: Compressed Pristines (Summary)
>> >
>> > Ashod Nakashian wrote:
>> >
>> >> Daniel Shahaf wrote:
>> >>>>> Would you like to assess the feasibility of compressing the
>> > pristine
>> >>>> >store by re-mounting the "pristines" subdirectory as
>> > a compressed
>> >>>> >subtree in the operating system's file system?
>> >>>>
>> >>>> No :-)
>> >>>>
>> >>>> There are two ways to answer this interesting proposition of
>> >>>> compressed file-systems. The obvious one is that it isn't
>> > something
>> >>>> SVN can or should control. The file-system and certainly system
>> >>>> drivers are up to the user and any requirement or suggestion of
>> >>>> tempering with them is decidedly unwarranted and unexpected from
>> >>>> a VCS.
>> >>>
>> >>> The suggestion wasn't that you teach svn how to change the OS's
>> > fs
>> >>> settinsg, it was that you investigate how solutions at the OS level
>> >>> compare to the other approaches already suggested (custom format -based
>> >>> and sqlite-based pristine store).
>> >
>> > Yes, Daniel, that's exactly what I meant. Sorry, Ash, for not being clear
>> > enough.
>> >
>> > Of course that's inherently a non-portable solution that may not be
>> > available on all platforms and certainly will differ. But that's
>> > somewhat analogous to support for password storage: in Windows land
>> > there's a standard way to do it, in Unixy lands you have to choose your
>> > favourite third-party subsystem and externally configure Gnome-keyring or
>> > KDE-wallet or whatever you chose. Of course there are other
>> > disadvantages too, I just think it would be interesting to compare the
>> > disadvantages with the advantages.
>> >
>> >> That's an easy question. The answer is that at *best* they'll do as
>> > good
>> >> as in-place compression. However, in practice they'll do much worse.
>> >
>> > Ash, what's your measure of "best" here -- just the compression
>> > ratio? What about other kinds of goodness such as the automatic recovering of
>> > free space, and the ability to present a plain-text view of the file content
>> > through the standard filesystem API which eliminates the need for us to
>> > implement a plain-text cache in Subversion?
>> >
>> >> The
>> >> reason is that the OS level compression works on not only the single file
>> > level,
>> >> but actually at the block level. This is to make modifications reasonably
>> > fast
>> >> (read compressed data, uncompress, modify, write recompressed data). If the
>> >
>> >> complete file is compressed then even changing a single byte (neglecting
>> > that no
>> >> storage works on the byte-level anyway) will yield performance that will at
>> >
>> >> least linearly degrade by the filesize.
>> >
>> > Changing a single byte is an irrelevant scenario, because pristine files are
>> > always created in full and then never modified.
>>
>> But the OS compression doesn't know that! It's design assumes average
>> file modification operations, and that's rarely replacing the full
>> file (which is the norm for us). I was only explaining and justifying
>> the design decision of OS level compression.
>>
>
>But one _could_ write a filesystem that implements compression and/or
>deduplication aimed at .svn/pristine/'s workflow. Should effort be
>spent there rather than on implementing a smarter svn_wc__pristine_*()?
Perhaps we should, but is that a simpler/easier/more-maintainable/first-try approach that we can defend with a straight face? I'm having a hard time doing that in my current mode.
>
>Daniel
>(devil's advocate mode)
>
>> >
>> >>> In short: if 'mount -o compress=yes' provides 90% space savings
>> > then
>> >> we
>> >>> would have little reason to implement space-saving solutions in svn
>> > itself.
>> >>> But it's the user's, not svn's, responsibility to run that.
>> >
>> > By the way, Daniel, I'm not ruling out the possibility that we may want to
>> > provide some glue logic so that Subversion can help the user to set up the
>> > feature. Without such assistance, only expert users would ever benefit.
>> >
>> > It's clearly an interesting topic!
>> >
>> > - Julian
>> >
>
>
>
Received on 2012-04-04 21:26:53 CEST