[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [subversion 1.6.5]Poblem when tryin to commit a large amount of source code

From: Johan Corveleyn <jcorvel_at_gmail.com>
Date: Tue, 24 Aug 2010 19:19:56 +0200

On Tue, Aug 24, 2010 at 4:01 PM, fida aljounaidi
<fida.aljounaidi_at_gmail.com> wrote:
> On 24 August 2010 13:35, Johan Corveleyn <jcorvel_at_gmail.com> wrote:
>> On Tue, Aug 24, 2010 at 11:29 AM, fida aljounaidi
>> <fida.aljounaidi_at_gmail.com> wrote:
>> > On 20 August 2010 18:37, Ryan Schmidt <subversion-2010b_at_ryandesign.com>
>> > wrote:
>> >>
>> >> On Aug 20, 2010, at 05:05, fida aljounaidi wrote:
>> >>
>> >> > [Fri Aug 20 17:24:08 2010] [error] [client 10.68.5.82] Commit blocked
>> >> > by
>> >> > pre-commit hook (exit code 99) with output:\n[Error output could not
>> >> > be
>> >> > translated from the native locale to UTF-8.] [409, #165001]
>> >>
>> >> What's in your pre-commit hook script?
>> >>
>> > Hi
>> >
>> > There are some Branch and tags grammar controls written in Perl
>> > language.
>> >
>> > Is there any special things to do with perl scripts?
>> >
>> > Another thing to notice, when i import directory by directory, import
>> > succeeded. it fails only when i try to commit the whole sources. (3GB).
>>
>> Sounds familiar. Probably your pre-commit hook is taking too long when
>> you commit the entire source tree. I had that problem as well, and
>> took special care to optimize my pre-commit hook, so it can handle
>> very big commits. SVN itself does not have a problem with very large
>> commits, but pre-commit hooks (and post-commit hooks) can cause
>> trouble if they take too long.
>>
>> For instance, if your pre-commit hook does an "svnlook changed" or
>> "svnlook dirs-changed", and then does some check for every
>> path/directory that's being changed, that probably scales linearly
>> with the amount of files/directories that are part of the commit. If
>> every check takes 50 ms, and there are 1000 changes, your hook will
>> take 50 seconds.
>>
>> If the pre-commit hook takes too long, the client gives up, and
>> terminates the connection (by default after 30 seconds, I think). You
>> can increase the client-side timeout [1] to avoid this, or make sure
>> your pre-commit hook is fast enough.
>>
>> Alternatively, if this kind of commit is very rare, and you're sure
>> that it is valid for the pre-commit checks, you can temporarily
>> disable the pre-commit hook (or part of it, or only if $USER=<you>),
>> to make it go through. Or commit in pieces, like you did.
>>
>> It's an interesting exercise for the reader to add some debug/timing
>> logging to your pre-commit hook, to see how long it takes. See also
>> [2]
>>
>> [1]
>> http://svnbook.red-bean.com/en/1.5/svn.advanced.confarea.html#svn.advanced.confarea.opts.servers
>> (look for http-timeout)
>>
>> [2] http://svn.haxx.se/users/archive-2010-05/0044.shtml
>>
>
> Thanks a lot of those tips.
> In fact, the pre-commit hook script use 'svnlook changed' for every file. if
> there are all new files,
> But i've tried to change http-timeout and increase it until 7200.
> The was commit interrupted after 93 minutes. So 5640 seconds.

Hm, I think it's best you first try to find out whether the problem is
really due to the pre-commit hook. Maybe try to add some timing
logging, like in the link [2] from above (it requires perl, but
nothing more; and the overhead of that logging is really small, so I
leave it on all the time, so I can take a look if users complain about
problems).

Another possibility is that the server times out itself. You may have
some success by increasing the TimeOut value in httpd.conf (or
whatever .conf file is applicable).

Oh, and please don't top-post on this list (you should put your reply
below the text you're replying to, not above it). I've re-arranged
your mail to that effect.

Cheers,

-- 
Johan
Received on 2010-08-24 19:20:59 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.