On 10/27/11 13:47, Mark Phippard wrote:
> On Thu, Oct 27, 2011 at 4:47 AM, Attila Nagy <bra_at_fsn.hu
> <mailto:bra_at_fsn.hu>> wrote:
>
> On 10/26/11 15:37, Philip Martin wrote:
>> Attila Nagy<bra_at_fsn.hu> <mailto:bra_at_fsn.hu> writes:
>>
>>> I'm trying to update a working copy of some tens of GBs with svn 1.7.1
>> Did you upgrade with 1.7.0 or 1.7.1?
> I've upgraded the WC with 1.7.0, then switched to 1.7.1, which I'm
> currently using.
> The upgrade took nearly one week (I can't afford a clean checkout,
> because I have to preserve the files' inode numbers), at the start
> it was running very fast, and at the end of the week it could
> print a new directory in every 8-10 seconds, while the svn
> processes CPU usage was 100%.
> The disks weren't touched, it seems that even with a database
> completely in the OS's buffer cache the SQL operations are pretty
> slow with a lot of files.
> Could it be because some indexes are missing (or not optimized for
> this amount of files), or it is what we could get from SQLite?
>
>
> I can see where having all that RAM could help greatly when the
> database is being read, but I do not see how it would help when we are
> writing to the database. I would imagine the database does stuff to
> flush itself to disk whenever a transaction is committed. I thought
> this was why we see 1.7 slower than other versions via NFS, even on
> small working copies where the database would fit in anyones RAM.
>
> I would imagine svn upgrade is almost entirely writes and I recall it
> does quite a few transactions. So couldn't the linear slow down just
> be based on the growth in the amount of bytes that are written to disk
> each time?
>
I've checked the disk IO stat and I couldn't see any bottlenecks there.
In fact while upgrade-ing, I barely saw any disk IO.
Checking the process revealed a lot of random reads from the wc.db,
which the OS could serve out of memory.
BTW, I don't have any transaction-related files next to my wc.db (I
remember about wal, or journal named files, but I'm not a regular user
of SQLite), and definitely couldn't see too much fsyncs.
Received on 2011-10-27 14:11:27 CEST