AW: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c

Hi,

Just another crazy idea:

The main problem with the parallelization in ra_serf seems to be the number of http requests (which potentially causes a high number of authentications and tcp connections).

Maybe we could add some partitioned send-all request:

When the client decides to use 4 connections, it could send 4 requests, with some parameter like send-all(1/4), send-all(2/4), ..., send-all(4/4).

Then the server can send one quarter of the complete response on each connection.

An advanced server could even share the common state of those 4 requests through some shared memory / caching scheme, to avoid doing the same work multiple times.

Years ago, I implemented a similar scheme between caching GIS web frontend servers, and the rendering backend server, in the protocol for fetching and rendering the map tiles. It gave a nearly linear speedup with the number of connections, up to the point where the CPUs were saturated.

Best regards

Markus Schaber

CODESYSŪ a trademark of 3S-Smart Software Solutions GmbH

Inspiring Automation Solutions
________________________________________
3S-Smart Software Solutions GmbH
Dipl.-Inf. Markus Schaber | Product Development Core Technology
Memminger Str. 151 | 87439 Kempten | Germany
Tel. +49-831-54031-979 | Fax +49-831-54031-50

E-Mail: m.schaber_at_codesys.com | Web: codesys.com
CODESYS internet forum: forum.codesys.com

Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915
Von: justin.erenkrantz_at_gmail.com [mailto:justin.erenkrantz_at_gmail.com] Im Auftrag von Justin Erenkrantz
Gesendet: Samstag, 1. Dezember 2012 15:15
An: Lieven Govaerts
Cc: Johan Corveleyn; dev_at_subversion.apache.org
Betreff: Re: svn commit: r1415864 - /subversion/trunk/subversion/libsvn_ra_serf/update.c

On Sat, Dec 1, 2012 at 9:01 AM, Lieven Govaerts <svnlgo_at_mobsol.be> wrote:
There are some scenario's where either the server admin or the user
can decide if parallel requests make sense or not.

I'm specifically thinking of the use Kerberos per request
authentication. These responses can't be cached on the client side,
and require the authorization header to be sent for each request.
Assuming 2 step handshake of which serf can bypass the first, this
means an overhead per request of 1-10KB, with a 3 step handshake each
request has to be sent twice further increasing the overhead.
IMHO in this scenario the server admin should be able to veto the use
of parallel requests.

And the same is true for https connections, where it's also the server
admin who can decide if the necessary caches have been put in place to
enable the benefits of parallel requests.

Totally agreed.  I'd favor a three-value httpd directive option on the server-side that is advertised in the capabilities exchange:

- default (client defaults to parallel if ra_serf, serial if older ra_neon client; or if client overrides ra_serf via their local servers options)
- serial (server suggests to client that it should be serial; but permit parallel when client wants it)
- force-serial (same capability advertisement, but always trigger send-all responses regardless of what client asks for)

I'm 95% sure we have code in ra_serf that handles the case where the server sends us inline responses anyway as older (prior to 1.2, IIRC) always sent inline responses no matter what we send...so, it should be fairly straightforward decision tree with minimal code changes.

My $.02...which is still not enough for me to write the patch.  =)  -- justin
Received on 2012-12-03 10:43:54 CET