On Thu, Jul 13, 2017 at 12:34 PM, Paul Hammant <paul_at_hammant.org> wrote:
>>
>>
>> Mmm ... interesting proposition. Also huge -1 because it'd be a really
>> awesome abstraction leak. :)
>
>
>
> Is there any way to implement the (reasonable in my opinion) feature
> request, without transgressing on programming standards? I think turning
> off all deltification is a bit brute force, when I only really want it
> turned off files I'd sync that would be over 50Mb (.mp4 files and alike).
>
> Is that the alternate request?
>
> 1. compression-exempt-size-limit = 50MB
> 2. deltification-exempt-size-limit = 100KB
>
> Or am I really wanting Svn's backend compression and deltification to be out
> of band ?
>
> 1. compression-strategy = defer-to-idle-time-even-if-days-later
> 2. deltification-strategy = defer-to-idle-time-even-if-days-later
>
> - Paul
It occurred to me that there are two more fsfs.conf settings that you
can try to reduce the cpu cost on the server:
- max-deltification-walk: try setting that to 1 or even to 0 ("Very
small values may be useful in repositories that are dominated by
large, changing binaries. Should be a power of two minus 1. A value of
0 will effectively disable deltification"). ==> i.e. you can use this
to test what happens if you disable deltification entirely.
- max-linear-deltification (default=16): try setting that to a low
value, perhaps 1.
Also: for compression maybe you should try "compression-level=1" too,
to see if it gives you a good tradeoff between "not too much wasted
time for large binaries, and still useful if someone commits text
files". If that tradeoff is acceptable, maybe that's good enough for
now, without the need for the new features you've been proposing.
From fsfs.conf:
[[[
### During commit, the server may need to walk the whole change history of
### of a given node to find a suitable deltification base. This linear
### process can impact commit times, svnadmin load and similar operations.
### This setting limits the depth of the deltification history. If the
### threshold has been reached, the node will be stored as fulltext and a
### new deltification history begins.
### Note, this is unrelated to svn log.
### Very large values rarely provide significant additional savings but
### can impact performance greatly - in particular if directory
### deltification has been activated. Very small values may be useful in
### repositories that are dominated by large, changing binaries.
### Should be a power of two minus 1. A value of 0 will effectively
### disable deltification.
### For 1.8, the default value is 1023; earlier versions have no limit.
# max-deltification-walk = 1023
###
### The skip-delta scheme used by FSFS tends to repeatably store redundant
### delta information where a simple delta against the latest version is
### often smaller. By default, 1.8+ will therefore use skip deltas only
### after the linear chain of deltas has grown beyond the threshold
### specified by this setting.
### Values up to 64 can result in some reduction in repository size for
### the cost of quickly increasing I/O and CPU costs. Similarly, smaller
### numbers can reduce those costs at the cost of more disk space. For
### rarely read repositories or those containing larger binaries, this may
### present a better trade-off.
### Should be a power of two. A value of 1 or smaller will cause the
### exclusive use of skip-deltas (as in pre-1.8).
### For 1.8, the default value is 16; earlier versions use 1.
# max-linear-deltification = 16
]]]
--
Johan
Received on 2017-07-13 14:28:09 CEST