Re: HTTP protocol v2: rethunk.

There's a lock around appending to the proto-rev file, yes. And it
does write a bunch of little files for noderevs, directory listings,
and props, and gloms them on at finalization time, yes. But there are
no locks around editing the little files themselves; so for example,
if concurrent processes make two files in a directory at the same
time, there can be a race condition and only one will end up in the
listing. This situation isn't possible if you only access a
transaction from a single process (say, via the commit editor).

--dave

On Wed, Nov 5, 2008 at 8:58 PM, Greg Stein <gstein_at_gmail.com> wrote:
> We talked about this, and the group thought that FSFS locked around
> appending to the transaction. Or, that it wrote a bunch of little
> files and then glommed them together at finalization time.
>
> Regardless, we felt the goal was to be able to fill the network pipe
> via parallel PUT operations (and conversely, that a giant glom cannot
> fill the pipe). Thus, a bunch of individual operations in parallel
> makes more sense.
>
> Cheers,
> -g
>
> On Wed, Nov 5, 2008 at 7:37 PM, David Glasser <glasser_at_davidglasser.net> wrote:
>> That's not actually true; FSFS is as broken as it's always been for
>> simultaneous edits to the same transaction (no locking on noderev,
>> directory listing, or prop files). Works fine if you restrict
>> yourself to the repos commit editor, though.
>>
>> --dave
>>
>> On Wed, Nov 5, 2008 at 5:27 PM, Ben Collins-Sussman
>> <sussman_at_red-bean.com> wrote:
>>> No, we still intend to send one request to start a commit, then a
>>> bunch of PUTs in parallel, then one request to finish the commit.
>>>
>>> We certainly *could* put the whole editor drive into a single request,
>>> but that ruins the opportunity for parallelism. At the moment,
>>> libsvn_fs_bigtable is the only implementation that might break if
>>> multiple apache jobs try to simultaneously modify the same commit txn;
>>> BDB and FSFS are totally safe for this approach.
>>>
>>>
>>> On Wed, Nov 5, 2008 at 4:15 PM, David Glasser <glasser_at_davidglasser.net> wrote:
>>>> Or to put it another way: will your new implementation use the
>>>> libsvn_repos commit editor?
>>>>
>>>> --dave
>>>>
>>>> On Wed, Nov 5, 2008 at 2:15 PM, David Glasser <glasser_at_davidglasser.net> wrote:
>>>>> What do your new ideas say about commits? Will it be possible to have
>>>>> a single-HTTP-request commit, or will we still be stuck with the
>>>>> "commits are a random series of events in a random undocumented order
>>>>> that probably go to different backends"?
>>>>>
>>>>> --dave
>>>>>
>>>>> On Wed, Nov 5, 2008 at 8:45 AM, Ben Collins-Sussman
>>>>> <sussman_at_red-bean.com> wrote:
>>>>>> Last month I started a couple of threads about my intent to completely
>>>>>> rewrite our HTTP network protocol.
>>>>>>
>>>>>> I had originally pushed very hard for writing an entirely new
>>>>>> standalone apache module (mod_svn), which could then provide support
>>>>>> for WebDAV and older clients by deferring requests to mod_dav_svn
>>>>>> running "behind" it. My idea was to model the svnserve protocol -- do
>>>>>> exactly one network turnaround for each RA function. My hope is that
>>>>>> this would give us speed somewhat close to svnserve.
>>>>>>
>>>>>> On the plane out to Apachecon New Orleans, I had an opportunity to
>>>>>> really read ra_serf's code and understand what it's doing for each RA
>>>>>> function. I ended up persuading myself that if we can eliminate all
>>>>>> of the DeltaV "discovery" formalities (i.e. cut out all of the extra
>>>>>> PROPFINDs that precede so many different types of requests), we end up
>>>>>> with an HTTP protocol that's about 95% the same as what I would have
>>>>>> written from scratch anyway. Very sane and readable.
>>>>>>
>>>>>> This is the point where cmpilato, gstein, jerenkrantz, and many others
>>>>>> all get to scream "I told you so". :-)
>>>>>>
>>>>>> In any case, I had a late-night face-to-face design meeting with
>>>>>> {gstein, jerenkrantz, striker, pquerna, fitz} and as a group we worked
>>>>>> out all of the details of exactly how we want to change our existing
>>>>>> protocol. I'll be committing some massive updates to the
>>>>>> notes/http-protocol-v2.txt file very soon for everyone to look at. In
>>>>>> a nutshell, though: we're going to abandon DeltaV formalities and just
>>>>>> let the client *assume* the format of different types of URLs; the
>>>>>> client will be able to construct them all by itself.
>>>>>>
>>>>>> Meanwhile, in the same spirit of abandoning DeltaV, we want to do
>>>>>> something we've been discussing doing for (literally) years:
>>>>>> annnouncing a completely open syntax for fetching (rev,path) objects
>>>>>> over HTTP. In the past, people have sort of figured out that
>>>>>> "!svn/bc/REV/path" is the sooper-sekrit syntax, but we've warned
>>>>>> people not to depend on it. No more of that. We'd like to officially
>>>>>> support what source-browsers and other tools have already been doing
>>>>>> for years:
>>>>>>
>>>>>> path?r=REV
>>>>>>
>>>>>> I already have a smallpatch for mod_dav_svn to support this, and look
>>>>>> forward to others' review of it. It's really quite a tiny change, so
>>>>>> I'd like to see it go out in 1.6 if nobody objects. I'll then have 6
>>>>>> months to do all the 'big' protocol changes for 1.7.
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
>>>>>> For additional commands, e-mail: dev-help_at_subversion.tigris.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>>>>
>>>
>>
>>
>>
>> --
>> David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
>> For additional commands, e-mail: dev-help_at_subversion.tigris.org
>>
>>
>

-- 
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org