On Apr 22, 2007, at 11:47 PM, Tim T. wrote:
> On 4/23/07, Blair Zajac <blair@orcaware.com> wrote:
>>
>> On Apr 22, 2007, at 11:23 PM, Tim T. wrote:
>>
>> > On 4/23/07, Blair Zajac <blair@orcaware.com> wrote:
>> >>
>> >> On Apr 22, 2007, at 10:29 PM, Tim T. wrote:
>> >>
>> >> > Hi,
>> >> >
>> >> > Committing large files to Subversion seems to take forever.
>> >> >
>> >> > I handle the infrastructure for a project which needs to
>> archive
>> >> > some 10-20 XML
>> >> > files of about 6-7 Megabytes each. These datafiles are an
>> essential
>> >> > part of the
>> >> > delivery and must be archived.
>> >> >
>> >> > The last update to these datafiles took on the order of two
>> hours,
>> >> > even though I tried
>> >> > to be smart and deleted them first, checking the files in as
>> though
>> >> > they were new,
>> >> > and thus elliminating any line-by-line diffs.
>> >>
>> >> I would recommend keeping those files as related to their
>> >> predecessors. For files that large, after a while, you'll
>> appreciate
>> >> the svndiff algorithm in the repository for space savings.
>> >>
>> >> >
>> >> > Checking the project server ( A Sun Blade 1500, with 1 Gig of
>> >> > memory), I noticed
>> >> > that the post-commit script was eating 100 % CPU and 100% memory
>> >> > during that
>> >> > time. I'm using the standard perlscript by the way, no local
>> mods.
>> >>
>> >> Well, that's the issue, not Subversion itself.
>> >>
>> >> What is you post-commit script running?
>> >
>> > Fairly standard, I think;
>> > svnlook changed to get the changed files, then svn look diff, to
>> > determine what's changed.
>>
>> [cc'ing users]
>>
>> Hi Tim,
>>
>> Please don't take discussions off the mailing list.
> Mea culpa.. It's early here, and I hadn't had my full dose of caffeine
> for the day..
>
>>
>> Doing the svnlook and diff by hands sounds like the hard way to do
>> it. I would use mailer.py to get the email and diffs sent out.
>>
>> >
>> > I really want that diff, to minimize the chance of loosing work,
>> but
>> > I'd assume that this is the causing the problem. I guess I could
>> turn
>> > that off before the next
>> > major file commit..
>>
>> If you really want to minimize the chances of loosing work, the way
>> to do that is to ensure that your post-commit does either a full back
>> of the entire repository or you do incrementals of the last commit
>> onto some other partition, or better yet, other system. Sending
>> emails out of the diffs is another way.
>>
>> I would recommend using mailer.py to do any of this.
>
> Thanks,
>
> I'll take a look at this script, hopefully later today.
>
> I'm already running full backups on a daily basis (to two different
> systems, I've
> learned to be paranoid); If I request that my colleagues keep their
> working copies
> around for at least 24 hours after a commit, I should be be safe
> enough.
It would be best to do a
svnadmin dump --incremental -r $REV > /home/other/disk/than/the/one/
svn/is/on
after each commit, that way you don't have to worry about this
issue. If there is a crash, you can always use the last full and
roll in the few incrementals.
Regards,
Blair
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Mon Apr 23 09:16:03 2007