On 07.02.2011 21:51, Stefan Sperling wrote:
>> A lot of wc databases out there will be
>> so small that the user will hardly notice the memory increase.
> All we'd be doing is allowing sqlite to flush data to disk if needed.
> Even with a temporary table backed by a file, most operations happen in
> memory. Either in buffers managed by sqlite or the operating system's
> buffer cache (until sqlite does an fsync). So for small databases it
> shouldn't make a difference.
Indeed, some data points for the patched recursive proplist case,
in-memory vs. file-backed temporary storage:
* subversion trunk working copy (2k nodes): file-backed is 5% slower
* subversion tags tree (200k nodes): file-backed is 10% slower, svn
proces uses up to 80M of memory.
These numbers are not really significant given the 10x to infinityx
speedup compared to the current code. But I'd suggest that once /all/
the queries are optimized in this way, temporary storage should be made
file-backed by default as that makes horrible memory consumption less
likely.
-- Brane
Received on 2011-02-08 04:40:08 CET