On 9/7/06, Johnathan Gifford <jgifford@wernervas.com> wrote:
> Yes Troy,
>
> Please let us know. We are currently running 1.3.0 with over 15,000
> revisions and have found that in our single repository that all projects
> checkout significantly much quicker than one particular project. We
> think this is due to the size and breadth of that one project's
> directory structure. While almost all of our projects take less than
> three minutes to do a full checkout, this one project consistently takes
> more than three. Even a switch from branch to branch results in long
> switch times of around one minute (and then another two for Eclipse to
> build the project workspace). What's different about this project, lots
> of directories deep (up to 10 layers in some sections) with lots of
> files at the senior levels of the directory structure.
>
> How do your full working copying check out times compare to an export
> time? It might be the underly .svn folder choking on the number of
> files and directories.
>
> Johnathan Gifford
>
> >>> On Wed, Sep 6, 2006 at 10:45 PM, in message
> <930F5D613A943B41A5392993AC82940305BD8C47@EXCHANGE.trader>, "Lakshman
> Srilakshmanan" <lakshman.srilakshmanan@tradingpost.com.au> wrote:
> > Hi Troy,
> >
> > Do you have one project that occupies ~2GB of space, or multiple
> > projects contained in one repository consuming ~2GB of space. I
> suspect
> > the later, because we have multiple projects contained in multiple
> > repositories occupying ~1.7GB and no performance impact. The largest
> > project is ~120MB.
> >
> > Maybe you should consider re- organising your repository into
> smaller
> > ones. :)
> >
> > BTW : Were you suggesting that it takes ~6min to checkout the entire
> > ~2GB ?
> >
> > Thanks
> > Lakshman
> >
> >
> >> ----- Original Message-----
> >> From: Troy Curtis Jr [mailto:troycurtisjr@gmail.com]
> >> Sent: Thursday, 7 September 2006 11:50 AM
> >> To: users@subversion.tigris.org
> >> Subject: More FSFS performance comments
> >>
> >> Hey everyone, it's me again complaining about FSFS performance
> issues.
> >>
> >> I was really excited to hear about the improvements in the disk
> usage
> >> and performance of FSFS in the 1.4 release. So I got approval and
> >> converted our 1.3.2 repo of ~2 GB / 60000+ revs over to 1.4. I was
> >> really hoping (but not expecting) for one of the 50% repo size
> >> reductions that are possible , but it only shaved off 300 MB or so.
> >> Oh well that was not to big of a deal as disk space is cheap.
> >>
> >> I was really banking on the performance improvements (recap, FSFS:
> ~6
> >> min checkout times, ~35 min hotcopy times and BDB: ~3 min checkout
> >> times, ~7 min hotcopy times). Alas, my checkout times increased to
> >> just over ~8 minutes! Granted this was only a single test case (all
> I
> >> had time for today) and there may have been some gzipping happening
> on
> >> the server.
> >>
> >> So in the end, I must use BDB. I just hope that I do not have to
> many
> >> admin issues!
> >>
> >> Oh, I do have a question. I did a dump on my 2GB repo with the
> >> -- deltas option and the resulting dump file was (uncompressed)
> just
> >> under 400MB. Why is the size blowing up so much in the repo since
> it
> >> supposedly stores stuff as deltas also? I was thinking it might
> have
> >> something to do with having 10's of 1000s of small revs files
> taking
> >> up more disk space than is necessary, but the BDB repo is only
> >> slightly smaller (<100 MB). What about the properties,
> specifically
> >> the svn:keywords property that cvs2svn sets? Could that account
> for
> >> the size? Maybe it is worth another 15 hour conversion to test
> it...
> >>
> >> Thanks for everything and I really do like subversion a lot.
> >>
> >> Troy Curtis
> >>
> >> --
> >> "Beware of spyware. If you can, use the Firefox browser." - USA
> Today
> >> Download now at http://getfirefox.com
> >> Registered Linux User #354814 ( http://counter.li.org/)
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e- mail: users- unsubscribe@subversion.tigris.org
> >> For additional commands, e- mail: users- help@subversion.tigris.org
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e- mail: users- unsubscribe@subversion.tigris.org
> > For additional commands, e- mail: users- help@subversion.tigris.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
>
I hadn't thought about the directory complexity playing such a key
role but that is probably much more likely than the properties (I
still have not tested the repo size without the keywords property).
Our directory structure is pretty complex, with lots of levels and
quite a few toplevel ones. Hum, don't guess there is much that I can
do about that.
My repo has just one project, but there are several release branches
on it. The code size is approximately 300MB (includes several kernel
images and a couple of "vendor" binary files), and as I mentioned it
does have a fairly complex directory structure.
As to Kevin's question, I did time a commit where I inserted a one
line comment in every .c and .h file in my directory and commited it
to BDB (I had heard that BDB's disadvantage was longer commits), but I
can't seem to remember the value got. But you are absolutely right
about the fact that full checkouts would be pretty rare for
developers. I did reach the same conclusion, but then the hotcopy
thing hit me.
Unfortunatly it is NOT a read only copy. I will be creating a task
branch and then copying the repo to a removable hard disk to take to a
disconnected location. Commits will be made to that branch at the
remote location and then those changes will be dumped and loaded into
the main repo when the disk returns. Then people can go merge their
changes from that task branch into trunk. Thirty minutes is really
unacceptable for that process, especially when you can do it in 7!
I might see how long an export takes, it might be interesting, but I
have resigned myself to using BDB. As I said before, I just hope the
admin issues don't cause to much headache!
--
"Beware of spyware. If you can, use the Firefox browser." - USA Today
Download now at http://getfirefox.com
Registered Linux User #354814 ( http://counter.li.org/)
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Sep 8 06:38:57 2006