Balázs,
I have had problems with large directories in the past as well. The largest
I've tried contained some 5000 files.
I didn't try to tweak the DB_CONFIG values, though.
IIRC, Ben also admitted that SVN doesn't really scale well with the number
of files in a directory.
-- Jan Evert
> -----Original Message-----
> From: Balázs Szabó [SMTP:dlux@dlux.hu]
> Sent: Sunday, December 14, 2003 11:54 PM
> To: users@subversion.tigris.org
> Subject: Subversion and large directories
>
> Hi,
>
> I have just started to use subversion for archiving my mails. Yes, it
> sounds strange, but it costs much less space than always tar.gz my 500M
> mail-archive.
>
> My mailbox-collection is in Maildir format, and the largest directory
> contains 22700 emails. The problem is that subversion is slowed down when
> imports this large directory. (In the beginning it imported 10-15
> mails/sec, now it takes 20 seconds to import one mail).
>
> I have tweaked DB_CONFIG with the following values:
>
> set_lk_max_locks 50000
> set_lk_max_lockers 50000
> set_lk_max_objects 50000
>
> set_lg_bsize 2097152
> set_lg_max 8388608
>
> set_flags DB_TXN_NOSYNC
> set_cachesize 0 234881024 3
>
> I have 256M memory in my machine, so I cannot give more cache to this.
>
> My vmstat showed that it reads a lot of data from the disk (5-10M/s) while
> importing the directory.
>
> Is it possible to speed up the import? I don't want to change my folder
> structure if not necessary...
>
> Regards,
>
>
> Balázs Szabó (dLux)
> dlux@dlux.hu
=== E U R O N E X T - D I S C L A I M E R =============================
This e-mail and its attachments are only intended for the individual(s) or
entity(entities) named above to whom they are addressed and may contain
personal and/or confidential information. Please notify us immediately if
you are not the intended recipient. Any dissemination, duplication,
publication to third parties or other use of the contents of this e-mail or
its attachments is forbidden. Although this information has been compiled
with great care, neither Euronext N.V. nor its subsidiaries shall accept any
responsibility for any errors, omissions or other inaccuracies in this
information or for the consequences thereof, nor shall it be bound in any
way by the contents of this e-mail or its attachments. In the event of
incomplete or incorrect transmission please return the e-mail to the sender.
Cet e-mail et ses annexes sont uniquement destinés a la (aux) personne(s),
ou a l' (aux) entité(s) a laquelle (auxquelle(s)) il est adressé, visée (s)
en tete du présent message. Il peut contenir des informations personnelles
ou confidentielles. Merci de nous notifier immédiatement si cet e-mail vous
a été adressé par erreur. Toute diffusion, copie, publication a des tiers ou
toute autre utilisation de son contenu est interdite. Bien que cette
information ait été rassemblée avec une grande attention, ni Euronext N.V.
ni aucune de ses filiales, ne peut etre tenue responsable des erreurs,
omissions ou inexactitudes contenues dans cette information, ni ne peut etre
liée d'aucune maniere par le contenu de cet e-mail ou ses annexes. En cas de
transmission incorrecte ou incomplete, nous vous prions de retourner cet
e-mail a son émetteur.
Deze e-mail en zijn bijlagen zijn uitsluitend bestemd voor de
geadresseerde(n) als op dit e-mailblad vermeld. Het is mogelijk dat deze
e-mail persoonlijke en/of vertrouwelijke informatie bevat. Wanneer u niet de
geadresseerde bent, verzoeken wij u dringend ons daarvan te berichten. Elke
verspreiding, vermenigvuldiging, gebruik of openbaarmaking aan derden van de
inhoud van deze e-mail en zijn bijlagen, is verboden. Hoewel deze informatie
met de meeste zorg is samengesteld is Euronext N.V., en de tot Euronext N.V.
behorende werkmaatschappijen, op geen enkele wijze aansprakelijk voor
eventuele fouten, omissies of andere onjuistheden in deze informatie of de
gevolgen daarvan noch op enigerlei wijze gebonden aan de inhoud van de
e-mail of zijn bijlagen. Gelieve, in geval van onjuiste of onvolledige
ontvangst, deze e-mail terug te sturen naar de afzender.
================================
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Dec 18 10:40:44 2003