[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [RFC] Using LZ4 compression by default

From: Jacek Materna <jacek_at_assembla.com>
Date: Fri, 4 Aug 2017 09:35:31 -0500

Based on what we are seeing - disk storage is at least 10x less concerning
across a huge swatch of enterprise SVN users than UX (performance and speed
of developer workflows). It's becoming less and less so each quarter.
Unless the disk storage increase impacts the server side performance in
aggregate I don't understand why the priority (biased defaults) would not
be to increase the speed of every operation.

?

On Fri, Aug 4, 2017 at 9:30 AM, Nathan Hartman <hartman.nathan_at_gmail.com>
wrote:

> On Aug 2, 2017, at 2:59 PM, Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com>
> wrote:
> > With the recently added support for LZ4 compression (r1801940 et al),
> > we now have an option of using it by default for the on-disk data and
> > over the wire.
> > [...]
> > The amount of the required additional space is proportional
> > to the difference between the compression ratio of LZ4 and zlib-5,
> > which can be roughly estimated as around 30-35% for compressible
> > binary and text files, although that may vary depending on the
> > actual data.
> >
> > To illustrate how these changes will affect the speed of some of the
> > operations, the 'svn import' of a 2 GB file over HTTP on LAN in my
> > environment takes 18 seconds instead of 63 seconds.
>
> Regarding on disk storage, for small repos a 30% size increase is probably
> not material, but it may be significant for large repos. Is it feasible to
> get the best of both worlds by using LZ4 for fast commits and then
> recompress using zlib in svnadmin pack?

-- 
Jacek Materna
Chief Technology Officer
Assembla
+1 210 410 7661
Received on 2017-08-04 16:36:24 CEST

This is an archived mail posted to the Subversion Dev mailing list.