Re: Ruby bindings' excessive memory usage

Hi,

In <B918BEC2-A67F-4E82-AA64-4AFFB166A128@site5.com>
  "Ruby bindings' excessive memory usage" on Mon, 12 Dec 2005 20:58:49 +1100,
  David Felstead <dfelstead@site5.com> wrote:

> Hi all - I've discovered what looks to be a problem with the SVN ruby
> bindings where they begin to consume large amounts of memory whilst
> adding a large number of small files to a repository. My _guess_ is
> that there are APR pools not being disposed of or reused properly,
> but this is complete conjecture on my part.

This problem is caused because Svn::Client::Context#add (and
some Context methods) uses its pool not sub-pool of its
pool. Thus APR pools memory can't be recycled by Ruby's GC
until Svn::Client::Context is alive.

So far, this problem can be avoided by writing like the
following:

  100.times do |i|
    ctx = Svn::Client::Context.new
    ...
    ctx.add(...)
  end

instead of the following:

  ctx = Svn::Client::Context.new
  ...
  100.times do |i|
    ...
    ctx.add(...)
    # you may want to call GC.start.
  end

I will fix this problem by making sub-pool.

> Attached is some code to reproduce the problem - it creates 4000
> small files in a new working copy (/tmp/wc) and calls 'add' for each
> file. If you view memory usage in 'top' it begins to climb at an
> alarming rate, until ruby eventually core dumps. If you attempt to
> perform the same action using the 'svn' command at the command line,
> the memory usage remains stable, so it appears that the core libs are
> not at fault.

In using the 'svn' command case(*), it seems that memory
usage is grown bit by bit.

Regards,

--
kou
--
(*)
#!/usr/bin/env ruby
repos = "/tmp/repos"
wc = "/tmp/wc"
`rm -rf #{repos}`
`rm -rf #{wc}`
`svnadmin create #{repos}`
`svn co file://#{repos} #{wc}`
Dir.chdir(wc)
files = []
1.upto(4000) do |i|
  i = i.to_s
  File.open(i, 'w') do |f|
    f.puts(i * 100)
  end
  files << i
end
`svn add #{files.join(" ")}`
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org