[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [RFC] Using LZ4 compression by default

From: Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com>
Date: Fri, 4 Aug 2017 16:11:50 +0300

Branko Čibej <brane_at_apache.org> writes:

>> - Using LZ4 over the wire requires both endpoints to advertise that they
>> know how to deal with the new svndiff2 format that allows LZ4 compression.
>
> "Allows" or "requires"? I expect that anything in svndiff2 that's
> compressed uses LZ4?

In this sense, "requires": svndiff2 chunks that don't bloat from the
compression are always compressed with LZ4.

What I was trying to say here when I wrote that, I think, is that among
all svndiff formats, svndiff2 is the only one that "allows" using LZ4
compression.

> How do you detect that the network is "local"?

By comparing the latency against a predefined threshold (currently, 5 ms).

> I'm not too happy with the idea of another client-side knob for this.
> For example, I usually use my SVN client over the WAN but sometimes
> bring the laptop to the office ... so I'd "have" to either keep changing
> my client configuration (not viable) or configure for the worst case
> (which defeats the purpose of having LZ4 as an option). A server-side
> setting doesn't help, either, since the server has no idea where the
> client is accessing it from.

The "http-compression" client-side knob is not new, I think that we had it
for some time now.

The difference is that previously allowed values were "yes(default)/no",
and the compression has always been enabled for the majority of the users.
Now, it is "auto(default)/yes/no", where the new 'auto' value behaves as
'yes', but switches to the faster LZ4 compression for local networks.
The worst-case behavior doesn't change, and it behaves just as it would
behave previously by using the slower compression with a better ratio.

Unless I am missing something, the new 'auto' default should work reasonably
well, or at least, result in an improvement in the case you have described:

 - When working over WAN, the client would use slower compression with
   a better ratio, as it has been doing previously.

 - Once the laptop is brought to the office, the client sees the decreased
   latency and switches to the faster LZ4 compression.

> To make matters more interesting, when I'm working remotely, I can
> access the office SVN server either remotely through an HTTPS proxy, or
> "locally" by using our VPN ... then my IP address is on the same subnet
> as the server's, but it's still working on a WAN.

There is a known limitation of serf_connection_get_latency() that currently,
it doesn't determine the latency for proxied connections. In this case,
this wouldn't matter, as the proxy is used for remote work, where we
would want to keep the slower zlib algorithm. However, if the access
to the local server happens through a proxy, we won't be currently able
to use the faster compression algorithm. (Perhaps, we could improve on
this in the future.)

Thanks,
Evgeny Kotkov
Received on 2017-08-04 15:12:21 CEST

This is an archived mail posted to the Subversion Dev mailing list.