Branko Čibej wrote:
>>How about removing SVN_MAX_OBJECT_SIZE and replacing it with something
>>that makes sense? I don't see how you can have a compile time limit
>>on what is a sensible maximum object size. An object can be anything.
>
> There is a maximum contiguous memory block that can be handled by a
> particular architecture. Usually it is maxsize_t). We use this constant
> to check if the contents will "fit" in memory before trying to read them
> from the database.
And I argue that such a limit it completely useless. Normally you have
much less memory than the architecture limit. Especially so on a 64 bit
machine.
>>The limit is currently used for only one thing in the code, namely to
>>sanity check the size of a rep in the fs code. Renaming it to
>>SVN_MAX_REP_SIZE would make it easier to assign a reasonable size to it.
>
> It isn't exactly the same as a maximum rep size. Notice that we only
> check this in svn_fs__rep_contents, which reads the whole rep into
> memory. The streamy read functions don't have to check for that maximum.
>
> And that's what the question in the FIXME was about: Can we reasonably
> limit this size to, e.g., 100k or not?
Whatever value you choose, you risk to either use a too low value which
has the potential to hit a normal use case on a machine with lots of
memory, or you will choose a too high value, and you will never hit it
because you run out of memory before you hit the limit on a machine with
little memory. The current huge limit is at least harmless, but it is
also almost pointless.
If you want to keep the limit, how about if I add the following FIXME?
/* FIXME: The current value is too huge to be useful. */
> I do have a question for you, though: Did you chech what happens with
> streamy reads (and writes) from the DB now that you've changed the
> stream chunk size? Do we now store large fulltexts in 8kiB chunks
> instead of 100kiB chunks?
I think you're forgetting about the buffer handling in window_consumer
in tree.c. It buffers until the block size is greater than the constant
SVN_FS_WRITE_BUFFER_SIZE which is 512 kiB. Perhaps I can convince you
firther with an exercise? Before and after the patch, the following
recepie produces exactly the same output from "db_dump repos/db/strings
| wc -l". Does that address your concern?
# svnadmin create repos
# svn co file:///home/tori/x/repos wc
Checked out revision 0.
# dd if=/dev/urandom bs=1k count=200 | od > wc/tmp.txt
200+0 records in
200+0 records out
# svn add wc/tmp.txt
A wc/tmp.txt
# svn commit -m "" wc
Adding wc/tmp.txt
Transmitting file data .
Committed revision 1.
# db_dump repos2/db/strings | wc -l
18
> The large-file-integrity tests are specifically designed to generate
> files that use up more than one delta window. Now that you've split
> stream size and delta window size, I believe these tests no longer
> excersize the delta window boundary.
Ahh, now I see what you mean. I should have found that one. Thanks for
the hint. It's fixed in r9284.
/Tobias
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Apr 4 18:46:44 2004