On Tue, Dec 31, 2002 at 04:17:28PM -0500, Brandon Ehle wrote:
> I want to tackle Subversion Issue 773 because currently all checkout
> operations are failing on our repository because of this bug. I've been
> looking through the bucket-brigade stuff and how dav_svn__log_report()
> works. So far I've got a clear picture of what needs to be done, but
> I'm not sure yet what propfind wasn't written with streaming built in?
> Is it incompatible with bucket-brigade? Was it a lack of time? Is
> there some other technical restriction? Mostly I'm wondering this so
> that I don't make it 70% into the solution and find out the reason why
> propfind didn't support it and have to do a rewrite.
The PROPFIND handling was written back in 1998 before Apache 2.0 and bucket
brigades ever existed. It was also at a point in time where a lot of
properties weren't around, and I simply never heard of a problem with the
memory consumption which would necessitate a streaming option. Also, the
"buffer it in memory" means that you can return a meaningful error message
if something goes wrong. If you've streamed out the properties for half the
resources and *then* something goes wrong, it is rather difficult to notify
the client of that problem (most clients only look at the HTTP status line
rather than the status records in the propstat tags).
Subversion seems to compound the problem by the way the pools are used.
There is no reason that 100 meg of memory should be used for a response that
is a few hundred kilobytes. What is going into that extra memory? My guess
is that it represents transitory data that wasn't cleared from a pool. And a
good chunk of that is probably from the FS and how it doesn't distinguish
between a temporary pool and the pool for returned data. The FS *does* have
new code in there to make that distiction, but additional changes have to
happen to make use of it.
So... I think the first order of business to classify the memory that SVN is
using to determine why memory grows so much. Fix that. Second order of
business is to change the logic in Apache/mod_dav itself. Each resource's
bundle of properties can be gathered into memory and, if successful,
streamed out to the client. If a problem occurs for one (or more) resources,
then they can issue a propstat block that signals an error for that specific
resource.
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Thu Jan 2 23:22:48 2003