Note that HTTP specifies auth is per request, not per connection.
Holding up a pipeline based on the faulty concept of conn auth is an
incorrect design.
I could very much agree with a design that incorporates a dynamic
scaling of the pipeline fill. i.e. start with a few and quickly ramp
to 100 or whatever we believe the max request per child is set to.
This still assumes auth is the same across requests, but that is an
optimization rather than designing an API that assumes a conn level
auth.
Cheers,
-g
On Aug 30, 2007, at 11:21 AM, "Ivan Zhakov" <chemodax@gmail.com> wrote:
> On 8/28/07, Lieven Govaerts <lieven.govaerts@gmail.com> wrote:
>>
>> Ivan,
>>
>> thanks for the review. Inline some comments.
>>
>> On 8/28/07, Ivan Zhakov <chemodax@gmail.com> wrote:
>>>
>>> On 7/25/07, Lieven Govaerts <lgo@mobsol.be> wrote:
>>>> Attached two patches implement a new callback 'cansend' in serf
>>>> which is
>>>> used in ra_serf to hold sending a bunch of requests before the NTLM
>>>> authentication handshake is finished.
>>>>
>>>> We need this callback in this scenario: consider an apache setup
>>>> with
>>>> NTLM authentication MaxRequestsPerChild set to 100. Now use svn to
>>>> checkout a directory with more than 50 files. Serf will make 100
>>>> requests (PROPFIND+GET) for all those files and sends them on one
>>>> of the
>>>> connections. What happens is that the before the NTLM handshake is
>>>> finished, the connection will already max out on the number of
>>>> requests.
>> ..
>>>>
>>> I spend some time reviewing your patches and trying to understand
>>> how to
>>> fix this problem. I didn't come to final decision, but I've some
>>> thoughts.
>>>
>>> Sending first request, complete authentication and then send other
>>> request isn't bullet proof solution. Just imagine if first request
>>> isn't require authentication at all, but following ones want it?
>>
>> The authentication step is only required once per connection, whether
>> that is from the first request or later, it shouldn't make a
>> difference. If a 401 is only received after 10 requests, than the
>> next
>> request added to the pipeline will contain the NTLM auth phrase. No
>> other requests are sent until a response is received from the server.
>> If there were 50 requests still on the pipeline than those will all
>> get a 401 response and retried after the authentication has finished.
>> I'm not sure if the patch currently works like this, I'll test it.
> Ok, I agree with you that serf needs some way to stop sending new
> requests while authenticating.
> Now I have some questions concerning implementation: how serf will now
> when to recover sending requests? How we can sure that authenticating
> request will not be blocked by this callback?
>
>>
>>> Also sending 100 pipelined requests isn't good idea for me. I
>>> think we have to
>>> implement limit for maximum number of concurrent pipelined
>>> requests as other
>>> clients do. Mozilla Firefox has limit to 4 pipelined requests for
>>> example.
>>
>> Why do you don't think it ain't a good idea? What is the benefit of
>> introducing this limit? It's not like pipelining requests are adding
>> extra load to the server, they're just sent more efficiently.
>>
>> I found this blog post concerning the decision to add pipeling
>> support
>> to firefox:
>> http://weblogs.mozillazine.org/asa/archives/2003/11/http11_pipelining.html
>> It says: '3) We limit the number of requests in a pipeline to
>> minimize
>> the effects of head-of-line blocking. Mozilla uses a default value of
>> 4, but any value up to the hard-coded limit of 8 is possible.'
>>
>> Now I think the svn client and Firefox have a different goal here:
>> Firefox parallelizes requests as much as possible (within certain
>> limits), because there's a value in downloading each individual image
>> as fast as possible. In svn we only care for the final result.
>>
> Because we send 100 request and get 401 for all of them, only after
> that we will start authenticating, right? It's better to don't send
> more than 16 requests at once.
>
>>> Actually MaxRequestsPerChild limits number of connections per child
>>> [1]. Number of requests per connection controlled by
>>> MaxKeepAliveRequests [2].
>>>
>>> [1] http://httpd.apache.org/docs/2.0/mod/mpm_common.html#maxrequestsperchild
>>> [2] http://httpd.apache.org/docs/2.0/mod/core.html#maxkeepaliverequests
>>
>> serf currently already tries to estimate the MaxKeepAliveRequests
>> based on the number of received responses before a connection is
>> reset
>> by the server, to limit the number of requests on the pipeline.
> Thanks, I didn't know that.
>
> --
> Ivan Zhakov
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Aug 31 01:10:47 2007