There are new results for local and network disk (checkout large code base):
Local disk:
svn.r1387070 - 256.8 sec
svn.patch - 152.2 sec
Network disk:
svn.r1387070 - 5717 sec
svn.patch - 2790 sec
All systems use Windows 7.
On Wed, Sep 19, 2012 at 10:18 PM, Vasily Tunegov
<vasily.tunegov_at_gmail.com>wrote:
> Hi, Bert
>
> I understand what WAL mode is not suitable for svn. But my patch uses
> PERSIST mode which give very good results, e.g. for NTFS on Windows as I
> shown in tests.
>
> One problem, as says Philip Martin, is reduced performance on NFS disks.
> But I don't understand why. The number of file operations in PERSIST mode
> greatly reduced. Unfortunaly I don't have NFS disk, if it means Network
> File System developed by Sun Microsystems. Windows is using SMB protocol to
> access files via network. I will test SBM and put result here later.
>
> Best regards,
> Vasily Tunegov.
>
> On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben <bert_at_vmoo.com> wrote:
>
>> systems like NTFS on Windows
>> MIME-Version: 1.0
>> Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837
>>
>> --20cf3074b2a0fd54e504ca0ed837
>> Content-Type: text/plain; charset="utf-8"
>> Content-Transfer-Encoding: 7bit
>>
>> For your own install you can enable the WAL on the file and other users
>> of the db will use it. I don't think we can enable it as default (like
>> you said)
>>
>> Bert Huijben (Cell phone)
>> From: Vasily Tunegov
>> Sent: 19-9-2012 7:30
>> To: Philip Martin
>> Cc: dev_at_subversion.apache.org
>> Subject: Re: Subject: [PATCH] WC rep cache optimization for some file
>> systems like NTFS on Windows
>> Hi, Philip
>>
>> I'm using SQLite 3.7.14.
>> I tried to test checkout for our project code base, hosted on svn 1.7.6
>> server (93 984 Files, 3 204 Folders, 1.97GB). There are results:
>>
>> svn co snv://server/project/branches/branche src -q
>>
>> Standard disk:
>> svn.r1387070 - 256.8 sec
>> svn.patch - 152.2 sec
>>
>> My research shows that PERSIST journal mode greatly reducing the number of
>> I/O operations, such as creating, opening and closing SQLite journal files
>> on the disk. The best option here is WAL journal mode. But it works only
>> if
>> SQLite database is used from one computer (it use memory mapped files
>> API).
>> I don't know is this restriction is important to svn client.
>>
>> Best regards,
>> Vasily Tunegov.
>>
>> On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
>> <philip.martin_at_wandisco.com>wrote:
>>
>> > Vasily Tunegov <vasily.tunegov_at_gmail.com> writes:
>> >
>> > > Please review my patch (see attached file patch.txt). It improve svn
>> > client
>> > > performance on file systems like NTFS on Windows. Maybe on some other
>> > file
>> > > systems, e.g. NFS, but I didn't test it. This patch switch rep cache
>> db
>> > > journal mode from DELETE (by default) to PERSISTENT. For more info
>> about
>> > > journal modes in SQLite see
>> > > http://www.sqlite.org/pragma.html#pragma_journal_mode.
>> > >
>> > > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7
>> x64
>> > > Enterprise) for two disk drives: standard and ssd. For tests I used
>> > > Subversion Benchmark Tool, see
>> > >
>> >
>> https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
>> > .
>> > > For tested application I used svn 1.8.0 from trunk, revision 1387070.
>> > >
>> > > Subversion Benchmark Tool tests, total results:
>> > >
>> > > Standard disk:
>> > > svn.r1387070 - 3:05.895
>> > > svn.patch - 2:06.548
>> > >
>> > > SSD disk:
>> > > svn.r1387070 - 2:11.504
>> > > svn.patch - 1:38.397
>> >
>> > Which version of SQLite?
>> >
>> > On my Linux systems this gives a small improvement in checkout
>> > performance for working copies on local disks, unfortunately it also
>> > reduces checkout performance for working copies on NFS disks.
>> >
>> > I'm using SQLite 3.7.12.1 and 3.7.13.
>> >
>> > --
>> > Certified & Supported Apache Subversion Downloads:
>> > http://www.wandisco.com/subversion/download
>> >
>>
>> --20cf3074b2a0fd54e504ca0ed837
>> Content-Type: text/html; charset="utf-8"
>> Content-Transfer-Encoding: quoted-printable
>>
>> <html><head><meta content=3D"text/html; charset=3Dutf-8"
>> http-equiv=3D"Cont=
>> ent-Type"></head><body><div><div style=3D"font-family:
>> Calibri,sans-serif; =
>> font-size: 11pt;">For your own install you can enable the WAL on the file
>> a=
>> nd other users of the db will use it. I don't think we can enable it as
>> def=
>> ault (like you said)<br><br>Bert Huijben (Cell
>> phone)<br></div></div><hr><s=
>> pan style=3D"font-family: Tahoma,sans-serif; font-size: 10pt;
>> font-weight: =
>> bold;">From: </span><span style=3D"font-family: Tahoma,sans-serif;
>> font-siz=
>> e: 10pt;">Vasily Tunegov</span><br><span style=3D"font-family:
>> Tahoma,sans-=
>> serif; font-size: 10pt; font-weight: bold;">Sent: </span><span
>> style=3D"fon=
>> t-family: Tahoma,sans-serif; font-size: 10pt;">19-9-2012
>> 7:30</span><br><sp=
>> an style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
>> b=
>> old;">To: </span><span style=3D"font-family: Tahoma,sans-serif;
>> font-size: =
>> 10pt;">Philip Martin</span><br><span style=3D"font-family:
>> Tahoma,sans-seri=
>> f; font-size: 10pt; font-weight: bold;">Cc: </span><span
>> style=3D"font-fami=
>> ly: Tahoma,sans-serif; font-size: 10pt;">dev_at_subversion.apache.org
>> </span><b=
>> r><span style=3D"font-family: Tahoma,sans-serif; font-size: 10pt;
>> font-weig=
>> ht: bold;">Subject: </span><span style=3D"font-family: Tahoma,sans-serif;
>> f=
>> ont-size: 10pt;">Re: Subject: [PATCH] WC rep cache optimization for some
>> fi=
>> le systems like NTFS on Windows</span><br><br></body></html><div
>> style=3D"c=
>>
>> olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-c=
>> olor:rgb(255,255,255)">Hi,=C2=A0Philip</div><div
>> style=3D"color:rgb(34,34,3=
>>
>> 4);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255=
>> ,255)">
>> <br></div><span
>> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;f=
>> ont-size:13px;background-color:rgb(255,255,255)">I'm using SQLite
>> 3.7.1=
>> 4.=C2=A0</span><div
>> style=3D"color:rgb(34,34,34);font-family:arial,sans-ser=
>> if;font-size:13px;background-color:rgb(255,255,255)">
>> I tried to test checkout for our project code base, hosted on svn 1.7.6
>> ser=
>> ver (93 984 Files, 3 204 Folders, 1.97GB). There are results:</div><div
>> sty=
>>
>> le=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;backg=
>> round-color:rgb(255,255,255)">
>> <br></div><div
>> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo=
>> nt-size:13px;background-color:rgb(255,255,255)">svn co
>> snv://server/project=
>> /branches/branche src -q</div><div
>> style=3D"color:rgb(34,34,34);font-family=
>> :arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">
>> <br></div><div
>> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo=
>> nt-size:13px;background-color:rgb(255,255,255)">Standard
>> disk:<br>svn.r1387=
>> 070 - 256.8 sec<br>svn.patch - 152.2 sec<br><br>My research shows that
>> PERS=
>> IST journal mode greatly reducing the number of I/O operations, such as
>> cre=
>> ating, opening and closing SQLite journal files on the disk. The best
>> optio=
>> n here is WAL journal mode. But it works only if SQLite database is used
>> fr=
>> om one computer (it use memory mapped files API). I don't know is
>> this =
>> restriction is important to svn client.</div>
>> <div
>> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13=
>> px;background-color:rgb(255,255,255)"><br></div><div
>> style=3D"color:rgb(34,=
>>
>> 34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255=
>> ,255,255)">
>> Best regards,</div><div
>> style=3D"color:rgb(34,34,34);font-family:arial,sans=
>> -serif;font-size:13px;background-color:rgb(255,255,255)">Vasily
>> Tunegov.</d=
>> iv><br><div class=3D"gmail_quote">On Wed, Sep 19, 2012 at 2:30 PM, Philip
>> M=
>> artin <span dir=3D"ltr"><<a href=3D"mailto:philip.martin_at_wandisco.com"
>> t=
>> arget=3D"_blank">philip.martin_at_wandisco.com</a>></span> wrote:<br>
>> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
>> .8ex;border-left:1p=
>> x #ccc solid;padding-left:1ex"><div class=3D"im">Vasily Tunegov <<a
>> href=
>> =3D"mailto:vasily.tunegov_at_gmail.com">vasily.tunegov_at_gmail.com</a>>
>> write=
>> s:<br>
>>
>> <br>
>> > Please review my patch (see attached file patch.txt). It improve svn
>> c=
>> lient<br>
>> > performance on file systems like NTFS on Windows. Maybe on some
>> other =
>> file<br>
>> > systems, e.g. NFS, but I didn't test it. This patch switch rep
>> cac=
>> he db<br>
>> > journal mode from DELETE (by default) to PERSISTENT. For more info
>> abo=
>> ut<br>
>> > journal modes in SQLite see<br>
>> > <a href=3D"http://www.sqlite.org/pragma.html#pragma_journal_mode"
>> targ=
>> et=3D"_blank">http://www.sqlite.org/pragma.html#pragma_journal_mode
>> </a>.<br=
>> >
>> ><br>
>> > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows
>> 7 =
>> x64<br>
>> > Enterprise) for two disk drives: standard and ssd. For tests I
>> used<br=
>> >
>> > Subversion Benchmark Tool, see<br>
>> > <a href=3D"
>> https://ctf.open.collab.net/sf/frs/do/listReleases/projects=
>> .csvn/frs.subversion_benchmark_tool" target=3D"_blank">
>> https://ctf.open.col=
>> lab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
>> <=
>> /a>.<br>
>>
>> > For tested application I used svn 1.8.0 from trunk, revision
>> 1387070.<=
>> br>
>> ><br>
>> > Subversion Benchmark Tool tests, total results:<br>
>> ><br>
>> > Standard disk:<br>
>> > svn.r1387070 - 3:05.895<br>
>> > svn.patch - 2:06.548<br>
>> ><br>
>> > SSD disk:<br>
>> > svn.r1387070 - 2:11.504<br>
>> > svn.patch - 1:38.397<br>
>> <br>
>> </div>Which version of SQLite?<br>
>> <br>
>> On my Linux systems this gives a small improvement in checkout<br>
>> performance for working copies on local disks, unfortunately it also<br>
>> reduces checkout performance for working copies on NFS disks.<br>
>> <br>
>> I'm using SQLite 3.7.12.1 and 3.7.13.<br>
>> <span class=3D"HOEnZb"><font color=3D"#888888"><br>
>> --<br>
>> Certified & Supported Apache Subversion Downloads:<br>
>> <a href=3D"http://www.wandisco.com/subversion/download"
>> target=3D"_blank">h=
>> ttp://www.wandisco.com/subversion/download</a><br>
>> </font></span></blockquote></div><br>
>>
>> --20cf3074b2a0fd54e504ca0ed837--
>>
>
>
Received on 2012-09-20 07:01:03 CEST