[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Frequent revisions to large compressed files

From: Kynn Jones <kynnjo_at_gmail.com>
Date: Tue, 2 Dec 2008 16:23:01 -0500

Thanks!
Kynn

On Tue, Dec 2, 2008 at 3:21 PM, Ryan Schmidt <
subversion-2008c_at_ryandesign.com> wrote:

>
> On Dec 2, 2008, at 13:09, Kynn Jones wrote:
>
> Hi. I would like to put some large compressed files under version control
>> using Subversion. The largest of these files is about 200MB and the
>> smallest is about 40MB, and they are all compressed using gzip, and I expect
>> that I will need to commit new revisions of these files at the rate of once
>> a day.
>>
>> Does Subversion keep track of changes incrementally? And, if so, would
>> small incremental changes in the *uncompressed* versions of these files
>> translate to small incremental changes in the depository?
>>
>> I suspect that the answer to the last question is "no", because (I guess)
>> that, for it to be "yes", Subversion would have to be smart enough to
>> realize that it is dealing with a compressed file and treat the contents
>> accordingly. This seems like too much to expect...
>>
>> My concern here is that as new revisions of these files are committed to
>> the depository, its disk footprint will balloon rapidly, because, even
>> though the uncompressed revisions are minor, the compressed file will change
>> drastically from one revision to the next, so that even an incremental
>> update will require having almost the entire file saved each time.
>>
>> (I hope the above made sense!)
>>
>
> You're correct in all you say. It in fact depends on the compression
> algorithm; if it causes the old and new files to look totally different,
> then Subversion has no choice but to store all of that difference.
> Subversion does not examine your binary files or attempt to decompress them.
>
> Any words of wisdom on frequently committing large compressed files to
>> Subversion would be much appreciated!
>>
>
> Either store the uncompressed files in the repository (yes the initial
> revision will be larger but subsequent revisions will be only as big as the
> changes so this should be better over time than regular compressed files),
> or compress the files in a way that causes only the differences in the
> original files to show through in the compressed files. For example, for
> gzip, you can use the --rsyncable flag. gzip requires a patch to implement
> this flag; you can get it here:
>
> http://trac.macports.org/ticket/16446
>
>
>
>

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=961128

To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2008-12-02 22:23:57 CET

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.