Re: svn commit: r1716575 - in /subversion/trunk/subversion/libsvn_ra_serf: commit.c ra_serf.h util.c

On 26 November 2015 at 12:03, Bert Huijben <bert_at_qqmail.nl> wrote:
>> -----Original Message-----
>> From: Ivan Zhakov [mailto:ivan_at_visualsvn.com]
>> Sent: donderdag 26 november 2015 09:54
>> To: Bert Huijben <bert_at_qqmail.nl>
>> Cc: dev_at_subversion.apache.org
>> Subject: Re: svn commit: r1716575 - in
>> /subversion/trunk/subversion/libsvn_ra_serf: commit.c ra_serf.h util.c
>>
>> On 26 November 2015 at 11:41, Bert Huijben <bert_at_qqmail.nl> wrote:
>> >> -----Original Message-----
>> >> From: Ivan Zhakov [mailto:ivan_at_visualsvn.com]
>> >> Sent: donderdag 26 november 2015 09:19
>> >> To: dev_at_subversion.apache.org; Bert Huijben <bert_at_qqmail.nl>
>> >> Subject: Re: svn commit: r1716575 - in
>> >> /subversion/trunk/subversion/libsvn_ra_serf: commit.c ra_serf.h util.c
>> >>
>> >> On 26 November 2015 at 11:05, <rhuijben_at_apache.org> wrote:
>> >> >
>> >> > Author: rhuijben
>> >> > Date: Thu Nov 26 08:05:36 2015
>> >> > New Revision: 1716575
>> >> >
>> >> > URL: http://svn.apache.org/viewvc?rev=1716575&view=rev
>> >> > Log:
>> >> > In ra_serf: when a to-be committed file has text and property changes
>> to
>> >> be
>> >> > applied, pipeline both requests.
>> >> >
>> >> This could fail over HTTP/2 if both pipelined requests will be handled
>> >> by different worker threads: FSFS doesn't allow concurrent access to
>> >> transactions.
>> >
>> > I'm surprised to learn that. I would have guessed concurrent access was
>> > always part of the design of the fs layer, by the way we use it in mod_dav.
>> >
>> > I hope we can somehow lift that restriction, as improving commit
>> performance
>> > over mod_dav is high on a few wish lists.
>> >
>> First of all I'm not sure that concurrent *writing* could speed up
>> commit: there is no fsync() calls when working with transactions. As
>> far I remember svn:// also waits for every TXN operation to complete
>> before sending next one, so svn:// and http:// performance should be
>> the same when working over high-latency network.
>>
>> Another problem could be than TXN operations are have dependencies on
>> each other and order is very important.
>
> That assumes that the limit is on the server side.
>
> I usually commit halfway over the earth... my limits are 100% in the pipeline and
> mostly in the latency between client and server. (100Mbit+ connections on both
> sides). Sending concurrent requests would be a huge performance boost.

Yes, it will be the boost in high-latency connection, but it could
degrade performance in low-latency network environment due high
concurrency.

>
>
> But if we require everything in a strict order we should just immediately stop
> using multiple requests. If serialized access is the only thing our backends
> support we should just send one huge commit request. That saves a huge number
> of synchronize events between client and server.
>
> Currently we send a request... wait for the result... (server waiting for us) send a request... wait for the result.
>
Just to clarify: concurrency limitation is only applies for write
operations to TXN. Removing this limitation will be very challenging
task IMO. It this could reduce server-side performance since we would
need more syncronization, cache invalidation etc.

One huge commit request has another problems:
1. Our commit editor API performs operations one by one and caller
expect to get operation result early, instead of final txn commit
stage.

2. In HTTP/1.1 there is no way to signal client that it should stop
sending data due error: server expected to read all request body (and
discard) to send response. But this was improved in HTTP/2 though.

> And more of that...
>
> Removing a tempfile here or there isn't going to help if we can't remove the waiting.
> We can't change the wire (or light) speed between client and server.
>
Yes, we cannot change the light speed as far I know :)

Our tests shows that commit operation is relatively even when done to
the local server and there are some room for improvements.

> We could design the request to return the same intermediate results in the request
> body as we do now, with only a fraction of the synchronization points.
>
> But this is the other way around as the intensions we had with HTTPv2 around 1.7.
> There we tried to do things the HTTP way by optimal pipelining.
>
I was really surprised that HTTP/2 dropped option to send strict
ordered HTTP request. I think it's serious protocol regression.

-- 
Ivan Zhakov