RE: SQLite and callbacks

> -----Original Message-----
> From: Branko Čibej [mailto:brane_at_xbc.nu] On Behalf Of Branko Cibej
> Sent: dinsdag 8 februari 2011 4:39
> To: dev_at_subversion.apache.org
> Subject: Re: SQLite and callbacks
>
> On 07.02.2011 21:51, Stefan Sperling wrote:
> >> A lot of wc databases out there will be
> >> so small that the user will hardly notice the memory increase.
> > All we'd be doing is allowing sqlite to flush data to disk if needed.
> > Even with a temporary table backed by a file, most operations happen
> in
> > memory. Either in buffers managed by sqlite or the operating system's
> > buffer cache (until sqlite does an fsync). So for small databases it
> > shouldn't make a difference.

On NTFS just creating a new 0 byte tempfile requires an fsync (and probably a few in a row), so using the in memory buffers instead of a tempfile improved our SQLite performance significantly (and not only on Windows). Assuming using tempfiles was cheap was one of the major slowdowns of WC-1.0 on Windows.

Please don't suggest 'just making it file backed' as an easy feature if you only measured it on a non journaling filesystem.

With our current query scheme on 'common operations', switching to file based temporary storage will require rewriting almost every sql operation and how we use it to have release acceptable performance on Windows. A simple 'OR' or using a subquery may introduces a temptable.

In this thread we are looking at property storage... which was probably always slower than it is today.

Yes, we can improve that, but please don't suggest introducing a 30% slowdown on the more common code paths like those used in 'svn status' or 'svn update' to improve reading many properties, without measuring the consequences.

        Bert
Received on 2011-02-08 10:51:27 CET