Re: compression

Edward Ned Harvey wrote on Tue, 29 Jun 2010 at 07:15 -0000:
> Svnserve is using standard zlib (not a parallel implementation) so

You can disable all (most?) compression by not advertising the
svndiff1 capability in svnserve/serve.c.

> Some people are seeing 20min commit times on changes they could have
> copied uncompressed in 1min.

How do you know how long the commits would have taken with compression
disabled?

> Based on what I see in the source code, I think I can simply change the
> compression level to 0 or 1 (instead of the default 5) or even just disable
> compression by tweaking a few "if" statements and so forth ... And
> recompile.

IMO, don't disable it entirely; because that way you don't have to
guess, at each stream, whether or not it needs decompression.

> As far as I can tell, there is no harm in doing this. When data is read
> back out ... If the size matches, then it was stored uncompressed, and
> hence, no uncompression needed. If the size is less than the original size,
> then it must have been stored compressed, and hence uncompression is needed.

A compressed file may or may not be shorter than the original file.

You may not know the size/length in advance.

I don't like the idea of getting a stream and not *knowing* whether or
not its compressed.

> Parallel implementation of compression (utilize multiple cores/threads)
>

In what context? svn, svnserve, both? Have you determined that CPU is
the bottleneck?

> Configurable / disable-able compression level (config file edit, no
> recompile necessary)
>

+0 (won't use it myself, but if large installations need it then let's
add it).

> Optional compression algorithm. zlib vs bzip2 vs 7-zip (probably not
> possible to do LZO, but I'd like it, if possible.)
>

Note that the definition of svndiff1 ("svndiff version 1" ) hard-wires
zlib (see notes/svndiff).
Received on 2010-06-29 07:44:25 CEST