[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Proposal: new fsfs.conf properties

From: Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com>
Date: Fri, 14 Jul 2017 14:18:27 +0300

Paul Hammant <paul_at_hammant.org> writes:

> 1. compression-exempt-suffixes = mp3,mp4,jpeg
>
> 2. deltification-exempt-suffixes = mp3,mp4,jpeg
>
> Regardless of the setting of 'compression-level', #1 above two mean certain
> things can skip the compression attempt. It must give up at a certain point
> right?

Hi everyone,

To improve the situation with slow commits of large binary and, possibly,
incompressible files I committed a patch (http://svn.apache.org/r1801940)
that adds initial support for LZ4 compression in the backend. While still
providing a decent compression ratio, LZ4 offers much faster compression
even than zlib with level=1, and can skip incompressible data chunks.
(Presumably, LZ4 is used for on-the-fly compression in different file
systems for these reasons.)

I have seen significant (up to 3 times) speed improvement for svn import
and commit with large binary files. Sometimes, using LZ4 compression can
even outperform a configuration with disabled compression, if the file is
at least somehow compressible, as it reduces the time required to write
the data to disk.

Currently, LZ4 compression is enabled if the fsfs.conf file specifies
compression-level=1, and all other levels still use zlib for compression.
Right now, the support for LZ4 is only implemented for the file:// protocol,
but support/negotiation for other protocols can be added later.

Regards,
Evgeny Kotkov
Received on 2017-07-14 13:18:58 CEST

This is an archived mail posted to the Subversion Dev mailing list.