On Fri, Jun 17, 2011 at 09:44, <gstein_at_apache.org> wrote:
> Author: gstein
> Date: Fri Jun 17 05:44:02 2011
> New Revision: 1136777
>
> URL: http://svn.apache.org/viewvc?rev=1136777&view=rev
> Log:
> Continued work on the pausing mechanic in the XML parser of ra_serf.
>
> The in-memory blocks no longer use a bucket allocator, since that is
> generally internal to serf's bucket types. Not a big deal, as we can just
> use a simple linked-list of reusable blocks.
>
> This adds code for writing content to the "pending" content, and begins
> the code for injecting that content into the XML parser when ready.
>
> The update editor now contains logic for pausing and un-pausing the parser
> as certain conditions are met. It also makes the appropriate call to
> process the pending content.
>
> However: no logic is changed in the network reading portion, or the normal
> parser injection. In essence, the "pause" state is ignored and the
> "pending" content is never accumulated. Further development and testing is
> needed. This revision is a continued preview of the direction, for
> feedback and code review.
>
[..]
>
> Modified: subversion/trunk/subversion/libsvn_ra_serf/util.c
> URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_ra_serf/util.c?rev=1136777&r1=1136776&r2=1136777&view=diff
> ==============================================================================
[...]
> static svn_error_t *
> +write_to_pending(svn_ra_serf__xml_parser_t *ctx,
> + const char *data,
> + apr_size_t len,
> + apr_pool_t *scratch_pool)
> +{
> + struct pending_buffer_t *pb;
> +
> + /* The caller should not have provided us more than we can store into
> + a single memory block. */
> + SVN_ERR_ASSERT(len <= PARSE_CHUNK_SIZE);
> +
> + if (ctx->pending == NULL)
> + ctx->pending = apr_pcalloc(ctx->pool, sizeof(*ctx->pending));
> +
> + /* We do not (yet) have a spill file, but the amount stored in memory
> + has grown too large. Create the file and place the pending data into
> + the temporary file. */
> + if (ctx->pending->spill == NULL
> + && ctx->pending->memory_size > SPILL_SIZE)
> + {
> + SVN_ERR(svn_io_open_unique_file3(&ctx->pending->spill,
> + NULL /* temp_path */,
> + NULL /* dirpath */,
> + svn_io_file_del_on_pool_cleanup,
> + ctx->pool, scratch_pool));
> + }
> +
> + /* Once a spill file has been constructed, then we need to put all
> + arriving data into the file. We will no longer attempt to hold it
> + in memory. */
> + if (ctx->pending->spill != NULL)
> + {
> + /* NOTE: we assume the file position is at the END. The caller should
> + ensure this, so that we will append. */
> + SVN_ERR(svn_io_file_write_full(ctx->pending->spill, data, len,
> + NULL, scratch_pool));
I think you need 'return' or 'else' here, otherwise data will be
stored in file _AND_ in memory.
> + }
> +
> + /* We're still within bounds of holding the pending information in
> + memory. Get a buffer (already available, or alloc it), copy the data
> + there, and link it into our pending data. */
> + if (ctx->pending->avail != NULL)
> + {
> + pb = ctx->pending->avail;
> + ctx->pending->avail = pb->next;
> + }
> + else
> + {
> + pb = apr_palloc(ctx->pool, sizeof(*pb));
> + }
> + /* NOTE: *pb is uninitialized. All fields must be stored. */
> +
> + pb->size = len;
> + memcpy(pb->data, data, len);
> + pb->next = NULL;
> +
> + /* Start a list of buffers, or append to the end of the linked list
> + of buffers. */
> + if (ctx->pending->tail == NULL)
> + {
> + ctx->pending->head = pb;
> + ctx->pending->tail = pb;
> + }
> + else
> + {
> + ctx->pending->tail->next = pb;
> + ctx->pending->tail = pb;
> + }
> +
> + /* We need to record how much is buffered in memory. Once we reach
> + SPILL_SIZE (or thereabouts, it doesn't have to be precise), then
> + we'll switch to putting the content into a file. */
> + ctx->pending->memory_size += len;
> +
> + return SVN_NO_ERROR;
> +}
> +
> +
[...]
--
Ivan Zhakov
Received on 2011-06-21 00:45:24 CEST