On Tue, 2003-02-11 at 11:11, Karl Fogel wrote:
> I don't know the answer yet; would love some feedback. Are the
> problems identified here the same ones you were thinking of?
Yes. By moving the "implicit base" requirement inward, you require the
addition of callbacks to allow the connecting pipes to produce and
consume deltas. That's doable, but complicated; I think the information
provided to the editor should match the information we expect to see
going over the net; none of this clever judo stuff which moves
complexity from the places you'd expect to find it to the places you
wouldn't.
From a less biased perspective, here is my detailed analysis. First,
the problems:
P1: Neon doesn't have a push interface for request bodies, so right now
ra_dav buffers text deltas on commits. There's a legitimate question of
whether solving this problem is allowing the tail to wag the dog.
P2: We send self-compressed deltas on checkout and import. This saves
bandwidth but takes time, and there might be more efficient ways to save
time and space using commodity compression code (in the ra layers).
More measurement is required to see if we should stop self-compressing,
but I'll assume for the sake of argument that we want to send full texts
for imports and checkouts.
P3: We use deltas when it makes no sense: over ra_local and over
ra_svn/ra_dav connections to loopback. The editor is in the best
position to notice these cases, since editor implementations live inside
ra_lib and driver implementations do not.
P4: We use deltas when it makes very little sense: over fast LANs. We
shouldn't try to guess when bandwidth is cheaper than CPU time, but in
an ideal world the user might be able to make that decision in
.subversion/servers or on the command line.
Here are the solutions people have proposed, and how they solve the
above problems:
S1: Leave apply_textdelta alone.
S1P1: Either don't solve this problem, or redesign Neon requests to
have a push interface. This has other benefits for Neon than making
Subversion happy; after this redesign, it could more easily support
non-blocking operation. (And, for those who have read through issue
510, there's no reason the redesigned interface couldn't still
support passing an fd for the request body, so sendfile() would
still be a possibility.)
S1P2: Instead of self-compressing when no base is available, fake up
delta windows which just say "insert this blob of new text." This
is just as efficient as any other proposed scheme for sending full
texts.
S1P3: Teach the drivers to send full texts (using faked-up delta
windows) according to a flag. Add a new function to ra_lib to ask
whether deltification is desirable, which can be by libsvn_client to
inform svn_wc_adm_crawl. On the server side, svnserve or
mod_dav_svn can provide the flag to libsvn_repos.
S1P4: Leverage S1P3; simply pass the full text flag to drivers if
configuration says it's appropriate.
S1 variant #1: Keep apply_textdelta but also add apply_fulltext. This
would let S1P[2-4] be more elegant, though not more efficient.
S1 variant #2: Modify apply_textdelta so that it can return a flag
saying "you can send me deltas if you want, but I think bandwidth is
cheap; you should send me full text instead for better performance."
Eliminates the need to pass flags to drivers to solve P3 (though that
mostly gives up on P4). Easiest if we don't use variant #1.
S1 variant #3: Invert apply_textdelta without worrying about full
texts. (By "invert" I mean that the driver provides a callback which
allows apply_textdelta to gather delta windows, instead of the editor
providing a callback which the driver pushes delta windows onto.)
Replace or agument svn_txdelta_to_svndiff with a function that pushes in
the opposite direction. Solves P1, doesn't affect P[2-4]. Compatible
with variant #1 or #2. Inconsistent with the rest of the editor
interface, which uses a push model.
S2: Replace apply_textdelta with apply_text([base stream], target
stream). The ra_svn receiving editor uses a callback to get at base
texts from the FS or WC as appropriate. ra_dav uses a callback to get
at base texts from the WC for update/switch/diff operations (no, the
pain is not limited to ra_svn, you just won't notice it as much inside
ra_dav because of the background pain level). Inconsistent with the
rest of the editor interface which uses a push model.
S2P1: As with S1 variant #3, svn_txdelta_to_svndiff will need to be
agumented or replaced with a function that pushes in the opposite
direction. After that is done, it is easy enough to make ra_dav's
apply_text supply svndiff data on demand from Neon.
S2P2: For checkout and import, pass NULL for the base stream. The
editor can still self-compress if it wants to, or it can send a full
text.
S2P3: The editor can choose to ignore the base stream and send a
full text if it believes bandwidth to be cheap.
S2P4: Editors could be passed a flag at creation time to use full
text instead of deltas, in which case they would ignore the bsae
stream.
S2 variant #1: Replace apply_textdelta with apply_text(target stream);
the base text becomes implicit. ra_dav uses a callback to get at base
texts from the WC for commit as well as update/switch/diff operations.
S2P2 changes a little bit, in that the callback will say "there is no
base text" and then the editor decide whether to self-compress or send
full text.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Feb 11 23:51:52 2003