On Fri, Jul 14, 2017 at 1:18 PM, Evgeny Kotkov
<evgeny.kotkov_at_visualsvn.com> wrote:
> Paul Hammant <paul_at_hammant.org> writes:
>
>> 1. compression-exempt-suffixes = mp3,mp4,jpeg
>>
>> 2. deltification-exempt-suffixes = mp3,mp4,jpeg
>>
>> Regardless of the setting of 'compression-level', #1 above two mean certain
>> things can skip the compression attempt. It must give up at a certain point
>> right?
>
> Hi everyone,
>
> To improve the situation with slow commits of large binary and, possibly,
> incompressible files I committed a patch (http://svn.apache.org/r1801940)
> that adds initial support for LZ4 compression in the backend. While still
> providing a decent compression ratio, LZ4 offers much faster compression
> even than zlib with level=1, and can skip incompressible data chunks.
> (Presumably, LZ4 is used for on-the-fly compression in different file
> systems for these reasons.)
>
> I have seen significant (up to 3 times) speed improvement for svn import
> and commit with large binary files. Sometimes, using LZ4 compression can
> even outperform a configuration with disabled compression, if the file is
> at least somehow compressible, as it reduces the time required to write
> the data to disk.
>
> Currently, LZ4 compression is enabled if the fsfs.conf file specifies
> compression-level=1, and all other levels still use zlib for compression.
> Right now, the support for LZ4 is only implemented for the file:// protocol,
> but support/negotiation for other protocols can be added later.
Nice!
But how about deltification? Has anyone tried / benchmarked the effect
of turning off deltification (with or without compression), to see
what the effect would be on the commit time? Like I suggested in this
thread yesterday (i.e. set max-deltification-walk to 0 in fsfs.conf --
or perhaps play with both max-deltification-walk and
max-linear-deltification) ...
--
Johan
Received on 2017-07-14 13:38:39 CEST