[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [RFC] Using LZ4 compression by default

From: Branko Čibej <brane_at_apache.org>
Date: Fri, 4 Aug 2017 13:46:58 +0200

On 02.08.2017 20:59, Evgeny Kotkov wrote:


> - Using LZ4 over the wire requires both endpoints to advertise that they
> know how to deal with the new svndiff2 format that allows LZ4 compression.

"Allows" or "requires"? I expect that anything in svndiff2 that's
compressed uses LZ4?


> I propose the following approach. Please note that for the wire format
> part, it only considers the http:// protocol, but we can optionally adjust
> svn:// later:
> (A) For the HTTP wire format, we start using LZ4 compression by default,
> but only over local networks.

How do you detect that the network is "local"?

> The reasoning behind this is that we probably wouldn't want to start
> always using LZ4 compression, as that would result in a regression over
> WAN, where the better compression ratio is usually preferable to the
> compression performance. Another point is that even for local networks
> we cannot disable compression altogether, because for slow 10 or even
> 100 Mbps LANs, where the throughput is limited by the slow network,
> using fast compression can be better than no compression. This is
> where LZ4 comes to the rescue by offering reasonable compression
> ratio and fast compression speed.
> This approach is currently implemented with the http-compression=auto
> client-side configuration option (r1803899), which is the new default.
> While the HTTP client is generally in charge of the used compression
> algorithm, there's also a way to override its preference on the server.
> If the mod_dav_svn's SVNCompressionLevel directive is set to 1, a
> server would then override the client's preference and still send
> svndiff2 / LZ4 data if the client can accept it.

I'm not too happy with the idea of another client-side knob for this.
For example, I usually use my SVN client over the WAN but sometimes
bring the laptop to the office ... so I'd "have" to either keep changing
my client configuration (not viable) or configure for the worst case
(which defeats the purpose of having LZ4 as an option). A server-side
setting doesn't help, either, since the server has no idea where the
client is accessing it from.

To make matters more interesting, when I'm working remotely, I can
access the office SVN server either remotely through an HTTPS proxy, or
"locally" by using our VPN ... then my IP address is on the same subnet
as the server's, but it's still working on a WAN.

-- Brane
Received on 2017-08-04 13:47:04 CEST

This is an archived mail posted to the Subversion Dev mailing list.