Re: "svnadmin load" a huge file

Les Mikesell wrote:
> >
> >>>>I migrated a large CVS repository (25-50 GB) to SVN years ago on SVN
> >>>>1.3. Our repo had many sections (projects) within it. We had to
> >>>>migrate each project independently so that it's team could coordinate
> >>>>when they migrated to SVN. As such, I dumped each project when ready
> >>>>and then svnadmin loaded each dump into it's own path/root (so as not to
> >>>>overwrite anything previously loaded and unrelated to this project's
> >>>>import).
> >>>>
>
> >
> >It would be fine if the project in question did not contain almost all
> >the files in one directory. You may call the layout silly, but CVS does
> >not seem to mind. OTOH, I would have distributed the files over
> >several subdirectories, but CVS does not handle moving files well.
> >
> >I wonder if cvs2svn is to blame that it produces a dump svnadmin
> >cannot load. Or I am always risking that "svnadmin dump" may one day
> >produce a dump a subsequent "svnadmin load" will be unable to swallow?
> >
> >I mean, if by hook or by crook, by using third party utilities like
> >svndumptool, I will eventually be able to convert this project from
> >CVS to SVN. Is there a chance that a subsequent dump will be again
> >unloadable?
>
> I don't think you are hitting some absolute limit in the software here,
> just running out of RAM on your particular machine. Can you do the
> conversion on a machine with more RAM?

I ran "svnadmin load" on a machine with 1 GB RAM and 25 GB swap (added
so much swap specially for the occasion). svnadmin crashed after
reaching the SIZE about 2.5 GB.

Is 1 GB RAM and 25 GB swap not enough?

I don't know if this gdb output will be useful:

(gdb) where
#0 0x2841e117 in kill () from /lib/libc.so.7
#1 0x2841e076 in raise () from /lib/libc.so.7
#2 0x2841cc4a in abort () from /lib/libc.so.7
#3 0x28116ec5 in abort_on_pool_failure (retcode=Could not find the frame base f
or "abort_on_pool_failure".
)
    at subversion/libsvn_subr/pool.c:49
#4 0x283095fb in apr_palloc (pool=0xba46d018, in_size=204800)
    at memory/unix/apr_pools.c:663
#5 0x280ebad3 in svn_txdelta_target_push (
    handler=0x280eb140 <window_handler>, handler_baton=0x5b26c058,
    source=0xba470738, pool=0xba46d018)
    at subversion/libsvn_delta/text_delta.c:528
#6 0x280d4d62 in svn_fs_fs__set_contents (stream=0xbfbfe7e4, fs=0x28512020,
    noderev=0x27dea528, pool=0x2852a018)
    at subversion/libsvn_fs_fs/fs_fs.c:5066
#7 0x280c9d52 in svn_fs_fs__dag_get_edit_stream (contents=0x2852a138,
    file=0x5b2e61a0, pool=0x2852a018) at subversion/libsvn_fs_fs/dag.c:997
#8 0x280de42e in fs_apply_text (contents_p=0xbfbfe904, root=0x5aef8058,
    path=0x2852a080 "ns2/trunk/tomsk.ru/SOA", result_checksum=0x2852a110,
    pool=0x2852a018) at subversion/libsvn_fs_fs/tree.c:2615
#9 0x280bf44e in svn_fs_apply_text (contents_p=0xbfbfe904, root=0x5aef8058,
    path=0x2852a080 "ns2/trunk/tomsk.ru/SOA",
    result_checksum=0x2852a0e8 "b03cbddfbc11be113cbf675862eb971e",
    pool=0x2852a018) at subversion/libsvn_fs/fs-loader.c:1096
---Type <return> to continue, or q <return> to quit---

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
sip:sudakov_at_sibptus.tomsk.ru