I keep my personal photos and videos in Subversion.
Total repository size is currently 475 GB. There are
some large (1-4 GB) video files. Everything seems to
work fine.
Some details: I'm still running Subversion 1.4.6 on the
server (FSFS, CentOS 5, x86_64). Running 1.6.16
cmdline client on CentOS and latest TortoiseSvn on
Windows XP (32 bit). I should try upgrading the server
at some point, but it does make me nervous, requires
a lot of space for backups etc.
The most annoying thing is probably the lack of true
moves and renames -- it takes a long time to move a
large file because it doesn't just move it on the client
side, it deletes it and recopies the whole thing over
the network (IIUC). I basically never modify any large
files, I just move them around and rename them.
Overall I'm quite happy using Subversion for this.
(I should try to make an effort to test 1.7, since it's
an interesting and somewhat unusual workload...)
HTH,
-Ed
PS TortoiseSvn working copy is 950 GB, 351K files,
30K folders, according to Windows "Properties".
On Mon, May 23, 2011 at 7:36 AM, Julian Foad <julian.foad_at_wandisco.com> wrote:
> On Fri, 2011-05-20, Winston Smith wrote:
>> Sorry if this is the wrong list, but I'm curious about one thing:
>> Are the SVN developers aware of any quirks in SVN in regards
>> to storing a large number (say, 1000) very large binary files
>> (say, 1GB each)? So, the entire repository would be 1TB of size,
>> but my concern is not space, but rather whether SVN would have
>> difficulties (either on the server side or the client side) handling
>> such repositories/workspaces. Thanks for your replies.
>
> I can't give a very definite answer, but, assuming your server and
> network and client hardware is adequately sized for the task, I would
> expect:
>
> * No problem handling many thousands of files in total.
>
> * Putting 1000 or more files all in the same directory can give poor
> performance in time and/or space, on both client and server, so avoid
> doing that.
>
> * Subversion can handle "binary" files as large as 1 GB or even many
> GB. There used to be a 2 GB limit when using old versions of Subversion
> in certain configurations, but that limit is long gone. I have heard
> that the server can be very slow if a new version of a very large file
> consists of a completely different bit pattern from the previous
> version. If your files don't change much, or if you only add new files
> and delete old files instead of checking in modifications to the
> existing files, that won't be an issue.
>
> * Subversion is designed to process large amounts of data "streamily"
> without trying to read it all into RAM at once, so you shouldn't need
> excessive amounts of RAM.
>
> I hope that helps. Please let us know what results you get. If you do
> run into any problem we'd like to know about it and try to fix it.
>
> - Julian
>
>
>
Received on 2011-05-23 18:49:42 CEST