On 2/6/07, Joaquim Oliveira <joaquim.oliveira@atlantico.com.br> wrote:
> Our team tried svn for 4 months, but we're switching back to CVS due to
> performance problems.
I feel your pain. :) Our project actually started out using WinCVS,
but then switched to Subversion. The main reason we would never switch
back is that we version large binaries (3D art and related files), and
require a lock system for preventing work loss from files that can't
be merged. We only started seeing a lot of slowdown when certain
folders started having 512 or more sub-folders only one level deep --
this seems to be an implementation limit in NTFS (and CIFS), as all
interaction with such folders slows down all versions of Windows, both
in and out of Subversion use.
We also version built files, so we were constantly annoyed with
unnecessary "unversioned file of the same name already exists" errors
on update and commit. We had to script around that.
> SVN access was made using svn:// protocol. CVS was faster in all
> situations. Most of then, it is twice faster. For example, an update in
> working copy root folder was about 20% faster in CVS.
I find this strange, because if I remember correctly, .cvs working
copy cache folders are strewn all over the working copy, just like
.svn folders. That is the biggest bottleneck with update and commit
actions for us. I find that SCM systems with central Working Copy
cache stores, including Perforce and SVK, all work *much faster* on
NTFS.
> We noticed that SVN creates more administrative files and directories
> than CVS. The checkout size is:
> - CVS: 24849 files, 4841 folders. Disk usage: 164 MB
> - SVN: 27450 files, 22319 folders (!). Disk usage: 261 MB
Is disk usage a big issue for you, or are you just noting this as
a potential bottleneck? If you have the disk space to spare, I highly
recommend SVK. With SVK 2.0.0 and a Subversion 1.4.2 FSFS repository
on each client, we are getting updates 50x to 1000x faster (depending
on number and breadth of changes), and commits 10x faster. The SVK
FSFS repository is taking about 50GB, due to a deep history of over
26K revisions, which is about 4x as much as a standard .svn WC cache
would take. The time and network bandwidth savings are much more
important to us, than ~$15 worth of extra disk space used per
workstation.
The main drawbacks to SVK are lack of a "svk lock" command, and lack
of GUI clients. We are working around both of those limits with
internal tools and scripts for now. We are also working on an "edit"
command and cache system, similar to a Perforce "checkout", to make
the commit times closer to instantaneous.
> I searched the mail list archives, but couldn't find a solution for
> this. I found something about "the NTFS file system does not perform
> well when you have a large number of small files"
The way NTFS stores meta-data, like timestamps and file size, is
horribly inefficient for large folder structures, and these meta-data
statistics are all analyzed repeatedly during update and commit
cycles. The way .svn and .cvs WC cache files are strewn throughout the
folders doesn't help matters. Matching something like SVK, with a
central WC cache, and a file system with efficient meta-data storage,
like ReiserFS, would be the best of both worlds. With NTFS, your best
hope is a central WC cache -- at least it wont make the recursive
folder-tree walks any slower.
> Is there any way to improve SVN performance? What are the most common
> bottlenecks?
Like I said, keep 1-level deep folder counts to below 512 on NTFS. IFS
might have the same implementation issue, so it's not clear if using a
different file system would help on Win32. Defrag often. Keeping the
server and all clients to SVN 1.4.2+ seems to help a little, in terms
of delta transfer time, and bandwidth used. The SVN 1.4 WC cache is
slightly faster, but it still has the problem of .svn folder
pollution.
If you can, use SVK. If Subversion has the best available
repository model, then SVK has the best available Working Copy cache
model. Using SVK with a CVS repository is a lot more difficult under
Win32 right now, but possible. If you use any central WC cache system,
like SVK, keeping the WC cache/repository on one disk (slower but
bigger), and your work area on another disk (faster writes, like a 10K
rpm disk), improves speed. Disk to disk tranfers are almost always
faster than disk to self copies.
:) Jared
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Feb 15 02:02:39 2007