[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: binary file size limit? (win32 crash+sandbox db corruption)

From: C.A.T.Magic <c.a.t.magic_at_gmx.at>
Date: 2004-03-11 17:44:54 CET

> Is there a *theoretical* limit? Not at all. The svn client and Apache
> server stay at a constant memory footprint, while streaming
> indefinitely-sized files to each other over HTTP. There's no
> theoretical limit to file size when doing HTTP GET and PUT operations.

Are you absolutely sure about the unlimited size?
is apr_off_t guaranteed to be 64bit everywhere?
ok... it seems to be 64bit on Linux and Win32 - nobody
will try and handle gigabyte-sized files on a PDA anyway ;-)


i just did some tests
(Win32, 3.2Ghz cpu, repository access with file:// on a local partition):

adding/commiting a -single- new 700MB file completes in about 10 minutes,

most of the time is spent on copying the file around inside of the wc:
on commit, svn seems to copy the file into the .svn/tmp folder first,
then starts transmitting it to the repos.
when using update, svn copies the file into .svn/tmp first, then again
copies it back out into the users wc area.
so maybe there is some space for speed improvement here
( maybe 'move' can be used instead of 'copy' in some cases? )


 "stability of WindowsXP" :

during the test for adding/committing a 4GB file my system (Win32)
started to get unstable and finally crashed :
reason for the system instability and crash was simple:
   'disk full' - ( WindowsXP, do I need to say more? )
after the reboot now the (fortunately just a sandbox) database
is corrupted in some strange way:

  (freed 8Gb of diskspace meantime)

  svnadmin recover D:\Repository
  Please wait; recovering the repository may take some time...
  svn: DB_RUNRECOVERY: Fatal error, run database recovery

  D:\Repository\db> db_recover
   db_recover: Ignoring log file: log.0000004763: magic number 0, not 40988
   db_recover: Invalid log file: log.0000004763: Invalid argument
   db_recover: First log record not found
   db_recover: PANIC: Invalid argument
   db_recover: DB_ENV->open: DB_RUNRECOVERY: Fatal error, run database

note: the first log file in db/ is log.0000000443,
the others were probably automatically deleted .

even MORE interresting:

>svnadmin verify D:\\Repository
* Verified revision 0.
* Verified revision 1.
* Verified revision 2.
* Verified revision 24.
* Verified revision 25.
* Verified revision 26.

verify does NOT detect an error!

since svn verify worked,
svnadmin dump/load also worked.
I could successfully reinport the DB in a few hours.
the reimported DB is free from errors.
It contained the 700MB file. the 4GB file is not yet there.

I'm not sure what caused this, the crash, the out of disk space
or the 4G file, but my sandbox-database seems in some weird condition.
its still possible to perform updates and add files, verify does not detect
an error,
dump worked
- but the db is reported as corrupt from both, svnadmin -and- db_recover !?

any good reasons?
why is it possible to commit to an already corrupted db?
does berkley db support files >4G ?


To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Mar 11 17:45:16 2004

This is an archived mail posted to the Subversion Users mailing list.