Re: Using perl to delta large files

On May 3, 2005, at 9:49 AM, David Gwyn wrote:

> I am writing a perl script to submit deltas. My current code runs out
> of memory when submitting large files (files in the gigabyte range).
> This happens because I read the entire file contents and then call
> send_string. Does anyone have any suggestions on how I can break the
> file up so it doesn't run into memory problems when I delta a file?
> Would I use send_stream instead of send_string?
>

I don't know the SVN::TxDelta:: API, but I do know the C API. You're
retrieving a txdelta consumer function from the editor, but it looks
like you're not pushing windows at it. Perhaps that's what send_stream
does, I don't know.

Here's the official instructions on how to send txdeltas using C, taken
from notes/txdelta_sanity:

To SEND textdeltas
------------------

  1. txdelta_stream = svn_txdelta (source_stream, target_stream);

  2. Get window handler to do something with delta. If using an
     "editor" to transmit information:

       window_consumer_func = editor->apply_textdelta (file_baton);

     If you want raw vcdiff data for embedding in XML or HTTP
     transactions:

       window_consumer_func = svn_txdelta_to_vcdiff (txdelta_stream);

  3. Loop until there are no more windows:

       txdelta_window = svn_txdelta_next_window (txdelta_stream);
       window_consumer_func (txdelta_window);
       svn_txdelta_free_window (txdelta_window);

     Be sure to call window_consumer_func with a NULL window when there
     are no more windows to consume.

  4. svn_txdelta_free (txdelta_stream);

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Tue May 3 17:31:03 2005