Anyone able to help with the issue below? Are these normal numbers, has anyone observed similar performance? Any suggestions on how I could improve the performance of svn blame?
I have done some more refined and smaller tests by reducing the number of revisions on these large files, by collapsing a large part of history in cvs before migrating. I came to the following numbers (over https):
- ~8 minutes for file1.xml (2 Mb large, 799 revisions)
- ~12 minutes for file2.xml (1,5 Mb large, 471 revisions)
This is already a lot better, but still quite unusable in a normal development scenario. Besides, collapsing our history is seen as a very undesirable workaround for this issue.
Some more observations:
1) I think it's mainly client-side cpu bound. Most of the time my client cpu is 100% used by svn client (sometimes it gets a rest for a couple of seconds, during which time the server (httpd) is fully loading a cpu on the server).
2) I guess the size of the files plays a big role in this issue. A file with 500 revisions doesn't seem abnormal to me, so file2.xml in the test above should be fine, except that it's large ...
3) I've been trying to figure out why file2.xml was slower in this test than file1.xml, even though it's smaller and has less revisions. I think it's because there are a couple of changes in its lifetime that changed the entire file (all lines), because of changes in indentation (spaces->tabs and vice versa). So I guess that causes huge diffs at that point.
4) I've put my repo on a local disk (to the server) instead of on the NFS share. That improved log performance (more than 50% faster), but did nothing to improve blame. I guess that's to be expected because of point 1).
Thanks for any suggestions or input about this problem.
Regards,
Johan
> Hi all,
>
> We're preparing/analyzing/testing a migration from CVS to SVN for some
> time now. All went well until I started focusing on a particular set of
> files in our repo: two large xml files that are changed very often. I
> already reported problems with svn log on these files (see
> http://subversion.tigris.org/ds/viewMessage.do?dsMessageId=1980568&dsFo
> rumId=10650). But that's still manageable/workaroundable (e.g. --limit
> 100 and the like).
>
> However, running svn blame on one of these files is taking a very long
> time:
> - ~5 hours for file1.xml (2 Mb large, 5500 changes) (locally on server
> with file:// protocol, since https remotely generates an error after 50
> minutes)
> - ~35 minutes for file2.xml (1,5 Mb large, 2300 changes) (remotely over
> https)
>
> Note: to narrow down on the issue, I tried this on a SVN repo only
> containing these two files (performed cvs2svn to a new repo, only
> migrating the directory with these files).
>
> Obviously this is not usable (if only for the timeout/error when trying
> this over https). For comparison, cvs annotate for file1.xml took only
> 17 seconds.
>
> Some info on the setup:
> SVN server 1.5.4 on Solaris 10 (package from sunfreeware.com)
> FSFS backend via NFS on netapp
> SVN client SlikSVN 1.5.5 on Windows XP (when testing remotely)
>
> After discussing with other developers, this is quite a big issue. Some
> developers that have to work on those files perform an "annotate" quite
> often, to investigate particular parts of the file (who/what/why/when
> it changed)
>
> Is there anything I can do about this?
>
> Should I raise this on the dev list to discuss possible improvements
> (e.g. caching line-based history info)? I have seen an open bug report
> discussing this
> (http://subversion.tigris.org/issues/show_bug.cgi?id=3074), but it
> doesn't seem to have a high priority or likeliness to be addressed.
>
> Some more details of some tests:
> --------
> $ time svn blame https://svnserver/svn/trunk/file1.xml
> svn: REPORT of '/svn/!svn/bc/96061/trunk/file1.xml': Could not read
> chunk delimiter: Secure connection truncated (https://svnserver)
>
> real 49m39.066s
> user 0m0.031s
> sys 0m0.140s
> --------
>
> When running it locally on the server with file://:
> --------
> $ time svn blame file:///path/to/repos/trunk/file1.xml
>
> [snip]
> real 320m12.402s
> user 305m40.880s
> sys 11m54.947s
> --------
>
> Any help is greatly appreciated.
>
> Regards,
> Johan
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessage
> Id=2069871
>
> To unsubscribe from this discussion, e-mail: [users-
> unsubscribe_at_subversion.tigris.org].
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2079688
To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-05-06 13:15:10 CEST