Hi, Bert
I understand what WAL mode is not suitable for svn. But my patch uses
PERSIST mode which give very good results, e.g. for NTFS on Windows as I
shown in tests.
One problem, as says Philip Martin, is reduced performance on NFS disks.
But I don't understand why. The number of file operations in PERSIST mode
greatly reduced. Unfortunaly I don't have NFS disk, if it means Network
File System developed by Sun Microsystems. Windows is using SMB protocol to
access files via network. I will test SBM and put result here later.
Best regards,
Vasily Tunegov.
On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben <bert_at_vmoo.com> wrote:
> systems like NTFS on Windows
> MIME-Version: 1.0
> Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837
>
> --20cf3074b2a0fd54e504ca0ed837
> Content-Type: text/plain; charset="utf-8"
> Content-Transfer-Encoding: 7bit
>
> For your own install you can enable the WAL on the file and other users
> of the db will use it. I don't think we can enable it as default (like
> you said)
>
> Bert Huijben (Cell phone)
> From: Vasily Tunegov
> Sent: 19-9-2012 7:30
> To: Philip Martin
> Cc: dev_at_subversion.apache.org
> Subject: Re: Subject: [PATCH] WC rep cache optimization for some file
> systems like NTFS on Windows
> Hi, Philip
>
> I'm using SQLite 3.7.14.
> I tried to test checkout for our project code base, hosted on svn 1.7.6
> server (93 984 Files, 3 204 Folders, 1.97GB). There are results:
>
> svn co snv://server/project/branches/branche src -q
>
> Standard disk:
> svn.r1387070 - 256.8 sec
> svn.patch - 152.2 sec
>
> My research shows that PERSIST journal mode greatly reducing the number of
> I/O operations, such as creating, opening and closing SQLite journal files
> on the disk. The best option here is WAL journal mode. But it works only if
> SQLite database is used from one computer (it use memory mapped files API).
> I don't know is this restriction is important to svn client.
>
> Best regards,
> Vasily Tunegov.
>
> On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin
> <philip.martin_at_wandisco.com>wrote:
>
> > Vasily Tunegov <vasily.tunegov_at_gmail.com> writes:
> >
> > > Please review my patch (see attached file patch.txt). It improve svn
> > client
> > > performance on file systems like NTFS on Windows. Maybe on some other
> > file
> > > systems, e.g. NFS, but I didn't test it. This patch switch rep cache db
> > > journal mode from DELETE (by default) to PERSISTENT. For more info
> about
> > > journal modes in SQLite see
> > > http://www.sqlite.org/pragma.html#pragma_journal_mode.
> > >
> > > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7
> x64
> > > Enterprise) for two disk drives: standard and ssd. For tests I used
> > > Subversion Benchmark Tool, see
> > >
> >
> https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
> > .
> > > For tested application I used svn 1.8.0 from trunk, revision 1387070.
> > >
> > > Subversion Benchmark Tool tests, total results:
> > >
> > > Standard disk:
> > > svn.r1387070 - 3:05.895
> > > svn.patch - 2:06.548
> > >
> > > SSD disk:
> > > svn.r1387070 - 2:11.504
> > > svn.patch - 1:38.397
> >
> > Which version of SQLite?
> >
> > On my Linux systems this gives a small improvement in checkout
> > performance for working copies on local disks, unfortunately it also
> > reduces checkout performance for working copies on NFS disks.
> >
> > I'm using SQLite 3.7.12.1 and 3.7.13.
> >
> > --
> > Certified & Supported Apache Subversion Downloads:
> > http://www.wandisco.com/subversion/download
> >
>
> --20cf3074b2a0fd54e504ca0ed837
> Content-Type: text/html; charset="utf-8"
> Content-Transfer-Encoding: quoted-printable
>
> <html><head><meta content=3D"text/html; charset=3Dutf-8"
> http-equiv=3D"Cont=
> ent-Type"></head><body><div><div style=3D"font-family: Calibri,sans-serif;
> =
> font-size: 11pt;">For your own install you can enable the WAL on the file
> a=
> nd other users of the db will use it. I don't think we can enable it as
> def=
> ault (like you said)<br><br>Bert Huijben (Cell
> phone)<br></div></div><hr><s=
> pan style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
> =
> bold;">From: </span><span style=3D"font-family: Tahoma,sans-serif;
> font-siz=
> e: 10pt;">Vasily Tunegov</span><br><span style=3D"font-family:
> Tahoma,sans-=
> serif; font-size: 10pt; font-weight: bold;">Sent: </span><span
> style=3D"fon=
> t-family: Tahoma,sans-serif; font-size: 10pt;">19-9-2012
> 7:30</span><br><sp=
> an style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight:
> b=
> old;">To: </span><span style=3D"font-family: Tahoma,sans-serif; font-size:
> =
> 10pt;">Philip Martin</span><br><span style=3D"font-family:
> Tahoma,sans-seri=
> f; font-size: 10pt; font-weight: bold;">Cc: </span><span
> style=3D"font-fami=
> ly: Tahoma,sans-serif; font-size: 10pt;">dev_at_subversion.apache.org
> </span><b=
> r><span style=3D"font-family: Tahoma,sans-serif; font-size: 10pt;
> font-weig=
> ht: bold;">Subject: </span><span style=3D"font-family: Tahoma,sans-serif;
> f=
> ont-size: 10pt;">Re: Subject: [PATCH] WC rep cache optimization for some
> fi=
> le systems like NTFS on Windows</span><br><br></body></html><div
> style=3D"c=
>
> olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-c=
> olor:rgb(255,255,255)">Hi,=C2=A0Philip</div><div
> style=3D"color:rgb(34,34,3=
>
> 4);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255=
> ,255)">
> <br></div><span
> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;f=
> ont-size:13px;background-color:rgb(255,255,255)">I'm using SQLite
> 3.7.1=
> 4.=C2=A0</span><div
> style=3D"color:rgb(34,34,34);font-family:arial,sans-ser=
> if;font-size:13px;background-color:rgb(255,255,255)">
> I tried to test checkout for our project code base, hosted on svn 1.7.6
> ser=
> ver (93 984 Files, 3 204 Folders, 1.97GB). There are results:</div><div
> sty=
>
> le=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;backg=
> round-color:rgb(255,255,255)">
> <br></div><div
> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo=
> nt-size:13px;background-color:rgb(255,255,255)">svn co
> snv://server/project=
> /branches/branche src -q</div><div
> style=3D"color:rgb(34,34,34);font-family=
> :arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">
> <br></div><div
> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo=
> nt-size:13px;background-color:rgb(255,255,255)">Standard
> disk:<br>svn.r1387=
> 070 - 256.8 sec<br>svn.patch - 152.2 sec<br><br>My research shows that
> PERS=
> IST journal mode greatly reducing the number of I/O operations, such as
> cre=
> ating, opening and closing SQLite journal files on the disk. The best
> optio=
> n here is WAL journal mode. But it works only if SQLite database is used
> fr=
> om one computer (it use memory mapped files API). I don't know is this
> =
> restriction is important to svn client.</div>
> <div
> style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13=
> px;background-color:rgb(255,255,255)"><br></div><div
> style=3D"color:rgb(34,=
>
> 34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255=
> ,255,255)">
> Best regards,</div><div
> style=3D"color:rgb(34,34,34);font-family:arial,sans=
> -serif;font-size:13px;background-color:rgb(255,255,255)">Vasily
> Tunegov.</d=
> iv><br><div class=3D"gmail_quote">On Wed, Sep 19, 2012 at 2:30 PM, Philip
> M=
> artin <span dir=3D"ltr"><<a href=3D"mailto:philip.martin_at_wandisco.com"
> t=
> arget=3D"_blank">philip.martin_at_wandisco.com</a>></span> wrote:<br>
> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
> .8ex;border-left:1p=
> x #ccc solid;padding-left:1ex"><div class=3D"im">Vasily Tunegov <<a
> href=
> =3D"mailto:vasily.tunegov_at_gmail.com">vasily.tunegov_at_gmail.com</a>>
> write=
> s:<br>
>
> <br>
> > Please review my patch (see attached file patch.txt). It improve svn
> c=
> lient<br>
> > performance on file systems like NTFS on Windows. Maybe on some other
> =
> file<br>
> > systems, e.g. NFS, but I didn't test it. This patch switch rep
> cac=
> he db<br>
> > journal mode from DELETE (by default) to PERSISTENT. For more info
> abo=
> ut<br>
> > journal modes in SQLite see<br>
> > <a href=3D"http://www.sqlite.org/pragma.html#pragma_journal_mode"
> targ=
> et=3D"_blank">http://www.sqlite.org/pragma.html#pragma_journal_mode
> </a>.<br=
> >
> ><br>
> > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7
> =
> x64<br>
> > Enterprise) for two disk drives: standard and ssd. For tests I
> used<br=
> >
> > Subversion Benchmark Tool, see<br>
> > <a href=3D"
> https://ctf.open.collab.net/sf/frs/do/listReleases/projects=
> .csvn/frs.subversion_benchmark_tool" target=3D"_blank">
> https://ctf.open.col=
> lab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool
> <=
> /a>.<br>
>
> > For tested application I used svn 1.8.0 from trunk, revision
> 1387070.<=
> br>
> ><br>
> > Subversion Benchmark Tool tests, total results:<br>
> ><br>
> > Standard disk:<br>
> > svn.r1387070 - 3:05.895<br>
> > svn.patch - 2:06.548<br>
> ><br>
> > SSD disk:<br>
> > svn.r1387070 - 2:11.504<br>
> > svn.patch - 1:38.397<br>
> <br>
> </div>Which version of SQLite?<br>
> <br>
> On my Linux systems this gives a small improvement in checkout<br>
> performance for working copies on local disks, unfortunately it also<br>
> reduces checkout performance for working copies on NFS disks.<br>
> <br>
> I'm using SQLite 3.7.12.1 and 3.7.13.<br>
> <span class=3D"HOEnZb"><font color=3D"#888888"><br>
> --<br>
> Certified & Supported Apache Subversion Downloads:<br>
> <a href=3D"http://www.wandisco.com/subversion/download"
> target=3D"_blank">h=
> ttp://www.wandisco.com/subversion/download</a><br>
> </font></span></blockquote></div><br>
>
> --20cf3074b2a0fd54e504ca0ed837--
>
Received on 2012-09-19 20:18:43 CEST