Re: Merging parallel-put to /trunk

On Fri, Feb 5, 2016 at 3:27 AM, Evgeny Kotkov <evgeny.kotkov_at_visualsvn.com>
wrote:
>...

> (1) Why do we start with adding a quite complex FS feature, given that we
> don't know what kind of problems are associated with implementing this
> in ra_serf?
>

Please do not deny a new feature, on the *supposition* that problems *may*
exist in another component.

Let's move forward, and not hold back on a mere thought of other problems.
... You know of one? Then okay. But otherwise, hold your tongue.

FWIW, serf can manage multiple connections, all runnings PUTs to the
server. No real issue there. We haven't tried it because the server has
never supported it. (and we couldn't under ra_neon)

Our commit code flow may need work to support it, but that does NOT detract
from creating the parallel-PUT capability on the server-side. We need an
update on the client to issue parallel PUTs, so this only means it would be
a bit more work to do so. That has no bearing on introducing the capability
on the server.

> (Can we actually do it? What can be parallelized while keeping the
> necessary order of operations on the transaction? How do we plug that
> into the commit editor? As well as that currently HTTP/2 is not
> officially supported by neither httpd nor serf.)
>

Oh, give me a break. "not officially supported" ... This is a public list.
I'll be nice.

Both of those components have H/2 support now.

 (2) Is making parallel PUTs the proper way to speed up commits?
>

Absolutely. Typically, the PUT is a delta against a prior revision, it may
be compressed, and it may be encrypted. ALL of the work to handle those
aspects can be handled by a different core in the server.

I seem to recall an inspection of our server time, that showed MD5 handling
was a significant component. By separating PUTs, we can get multiple server
cores working for us.

> As far as I know, squashing everything into a single POST would make
> the
> commit up to 10-20 times faster, depending on the amount of changes.
>

Pfft.

>...

> So, are we sure that we need to implement it this way?
>

Yes. We've been wanting parallel PUTs for years. The backend has denied us.
If Stefan can pull off this work, the beer is on me. Let's say 3 nights,
however much Stefan can manage.

Cheers,
-g
Received on 2016-02-06 14:09:04 CET