[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Why does NTFS compression reduces size of .pack-files by ~50 %?

From: Thorsten Schöning <tschoening_at_am-soft.de>
Date: Sat, 21 Sep 2019 16:15:23 +0200

Hi all,

I have some repo generating revisions automatically based on some
events of some software, to track some changes to some status and
config files automatically. This repos currently contains ~ 145'000
revisions and because the overhead of individual files is pretty high,
I'm packing them once a week.

That repo currently has a size of ~750 MiB on disk and I have a backup
on my Windows using NTFS. Just out of interest I enabled file system
compression for that repo and found that the storage needs could be
reduced to ~350 MiB of data.

That was pretty surprising for me, because some months ago I enabled
zlib compression level 9 and fully dumped/loaded that repo to benefit
of the compression. NTFS compression is known to prefer performance in
favour of overall savinbgs, so I would have expected to not get much
difference.

Looking at some of the pack files, there's a lot of ASCII and blocks
of binary 0s in them. Maybe NTFS compression is benefitting of those?
In general, is the data in the pack files for revs compressed or not?

From my understanding the data of the individual revs is reused and if
that is compressed, the pack files are as well. Looking at fsfs.conf,
compression for packed revprops need to be enabled additionally, but
that really only applies to the revprops, correct? Those don't count
much in my scenario.

Is there any way to further optimize/compress the pack files using SVN?

My repos are hosted under Linux using ext4, if it's normal what I see
and can't be optimized further using SVN itself, I 'm considering
switching to BTRFs with it's file system compression. Going to see the
same benefits like for NTFS pretty likely.

Thanks for your advices!

Mit freundlichen Grüßen,

Thorsten Schöning

-- 
Thorsten Schöning       E-Mail: Thorsten.Schoening_at_AM-SoFT.de
AM-SoFT IT-Systeme      http://www.AM-SoFT.de/
Telefon...........05151-  9468- 55
Fax...............05151-  9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Received on 2019-09-21 16:15:39 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.