No fear... "svn edit" is on deck. :-) (at least in terms of WC
storage/tracking; may leave client impl to others)
And you won't see me refusing a beer at the Toronado! hehe...
Cheers,
-g
On Fri, Aug 15, 2008 at 6:34 PM, Listman <listman_at_burble.net> wrote:
>
> Sounds amazing Greg!! I'll gladly open an on-going tab for you at the
> Toronado (http://www.toronado.com/) if you can pull this off :)
>
> Perhaps I'm stating the obvious but please lets not forget about the "svn
> edit" command thats been discussed on the list at various points (yes yes,
> optional.. :) - Thats the only way we'll be able to run "svn status" and get
> in the same performance ballpark as Perforce.. I love Subversion but its
> performance with large working copies is such a dog :(
>
>
> On Aug 14, 2008, at 3:12 AM- Aug 14, 2008, Greg Stein wrote:
>
>> Hey all... as you may have guessed from my commit below, I'm starting
>> on the long-needed WC rewrite. Some of you may not know who I am,
>> since I've been away from svn development for so long :-(
>>
>> I was one of the initial developers of svn, way back in mid-2000. My
>> work focused on the DAV network layer, both client and server. I did a
>> bunch of work on other parts of svn, of course, but that was my
>> primary focus. Oh, and getting that f***kin' automake out of our build
>> and creating the current build system. I guess I'd say that I
>> contributed to svn development for maybe the first three years of its
>> life. Then my time kind of petered off, unfortunately.
>>
>> I've recently come into a bunch of free time (woo!), and will work on
>> the next generation WC until the sucker is done. Not sure whether that
>> will be 3 months or 6, but no problem.
>>
>> Other things that I'm interested in:
>> * stripping out Neon support, and just using serf going forward
>> * any serf work to support the above (particularly issues with
>> googlecode.com's svn server)
>> * create a new FS backend based on revlog
>> * offline commits
>>
>> That should keep me busy for the next year, I think :-)
>>
>> Cheers,
>> -g
>>
>> On Wed, Aug 13, 2008 at 1:42 PM, <gstein_at_tigris.org> wrote:
>>>
>>> Author: gstein
>>> Date: Wed Aug 13 13:42:30 2008
>>> New Revision: 32461
>>>
>>> Log:
>>> add some thoughts/notes
>>>
>>> Modified:
>>> trunk/notes/wc-ng-design
>>>
>>> Modified: trunk/notes/wc-ng-design
>>> URL:
>>> http://svn.collab.net/viewvc/svn/trunk/notes/wc-ng-design?pathrev=32461&r1=32460&r2=32461
>>>
>>> ==============================================================================
>>> --- trunk/notes/wc-ng-design Wed Aug 13 13:30:12 2008 (r32460)
>>> +++ trunk/notes/wc-ng-design Wed Aug 13 13:42:30 2008 (r32461)
>>> @@ -260,6 +260,14 @@ then, if these routines are passed file
>>> happy, meaning they still don't need to know where the text base /
>>> source file is...]
>>>
>>> +###GJS: the APIs should use streams so that we can decompress as the
>>> + stream is being read. the diff library will need a callback of some
>>> + kind to perform the rewind, which will effectively just close and
>>> + reopen the stream. if it rewinds *multiple* times, then we may want
>>> + to cache the decompressed version of the file. I'll
>>> + investigate. Given our metadata/base-text storage system, I suspect
>>> + it will be very easy to cache decompressed copies for a while.
>>> +
>>> In order to keep developers sane, it should be extremely clear at any
>>> one time - when operating on a tree - which tree is being operated upon.
>>>
>>> @@ -270,6 +278,7 @@ solely that specific task.
>>> #####XBC Do please note that this suggests yet another instance of
>>> pure polymorphism coded in C. This runs contrary to the
>>> developer sanity requirement.
>>> +###GJS: agreed with XBC.
>>>
>>>
>>> Speed
>>> @@ -283,6 +292,10 @@ the ACTUAL state (concentrated in one pl
>>> it around as long as required, matching it with the WORKING state before
>>> operating on either (not only when deemed important!).
>>>
>>> +###GJS: working copy file counts are unbounded, so we need to be
>>> + careful about keeping "all" stat results in memory. I'll certainly
>>> + keep this in mind, however.
>>> +
>>> Working from the ACTUAL tree will also prove to be a step toward clarity
>>> regarding the exact tree which is being operated upon.
>>>
>>> @@ -306,8 +319,12 @@ There are 3 known models for storing met
>>> groups of users:
>>>
>>> - in-subtree metadata storage (.svn subdir model, as in wc-1.0)
>>> + ###GJS: euh... aren't we axing this? who has *requested* this?
>>> - in-'tree root' metadata storage (working copy central)
>>> - detached metadata storage (user-central)
>>> + - in $HOME/.subversion/
>>> + - in arbitrary location (e.g. $HOME is a (slow) NFS mount, and we
>>> + want the metadata on a local drive, such as /var/...)
>>>
>>> A solution to implementing each of these behaviours in order to satisfy
>>> the wide range of use-cases they solve, would be to define a module
>>> @@ -318,6 +335,31 @@ post-1.0 experiences with public vtables
>>> implementation details are allowed to differ between releases (even patch
>>> releases).
>>>
>>> +###GJS: note that we are talking about both metadata AND base-text
>>> + content. (and yeah, optional and compresses base-texts can be done
>>> + during this rewrite) Also note that we might be able to share
>>> + base-text content across working copies if they are all keyed by
>>> + the MD5 hash into storage directories (under the user-central model)
>>> +
>>> +###GJS: I don't think vtables are needed here. This is simply altering
>>> + the base location, not a whole new implementation. My plan is to
>>> + default to the "tree root" model with a .svn subdirectory. If a
>>> + .svn subdir is not found, then we fall back to looking in the
>>> + $HOME/.subversion/ directory (some subdir under there). If we
>>> + *still* don't find it, then some config options will point us to
>>> + the metadata/base-text location.
>>> +
>>> +###GJS: my plan is to upgrade the working copy if we find a pre-1.6
>>> + working copy. all the data will be lifted from the multiple .svn
>>> + subdirectories, and relocated to the "proper" storage location.
>>> + This will be a non-reversable upgrade, and will preclude pre-1.6
>>> + clients from using that working copy again.
>>> + Note: because of the "destructive" nature of this upgrade, and the
>>> + expected duration, we may want to require the user to perform an
>>> + explicit action in order to complete the upgrade. However, 1.6 will
>>> + not be able to *modify* wc-1.0 metadata -- just read it in order to
>>> + upgrade it to the new storage system.
>>> +
>>>
>>> Transaction duration / memory management
>>> ----------------------------------------
>>> @@ -350,6 +392,10 @@ as well as developer sanity: in the earl
>>> ment process, we'll be able to assess how well the design holds up
>>> under more difficult 'weather'.
>>>
>>> +###GJS: agreed. as much as possible, when I (re)implement the old APIs
>>> + in terms of the new APIs, then I'll write a whitebox test. we'll
>>> + see how long I keep that up :-P
>>> +
>>> Transactional updates
>>> ---------------------
>>>
>>> @@ -493,6 +539,10 @@ Given the assumption of 'little code dup
>>> having several modules which implement the same API (vtable) is
>>> justifiable.
>>>
>>> +###GJS: disagree. I plan to have just one library (libsvn_wc2?) and
>>> + will probably have no vtables. there is very little need for
>>> + distinct implementations, as far as I can tell.
>>> +
>>>
>>> Implementation proposals
>>> ========================
>>> @@ -564,6 +614,21 @@ goal of 'atomic updates' can be realised
>>> in an incomplete yet locked state. This would also invalidate issue
>>> #1879.
>>>
>>>
>>> +Basic Storage Mechanics
>>> +-----------------------
>>> +
>>> +All metadata will be stored into a single SQLite database. This
>>> +includes all of the "entry" fields *and* all of the properties
>>> +attached to the files/directories. SQLite transactions will be used
>>> +rather than the "loggy" mechanics of wc-1.0.
>>> +
>>> +Base text data will be stored in a multi-level directory structure,
>>> +keyed/named by the MD5 of the file. The database will record
>>> +appropriate mappings, content/compression types, and refcounts for the
>>> +base text files (for the shared case).
>>> +
>>> +
>>> +
>>> Other sections
>>> ---------------
>>> +==============
>>> remain to be done
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: svn-unsubscribe_at_subversion.tigris.org
>>> For additional commands, e-mail: svn-help_at_subversion.tigris.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
>> For additional commands, e-mail: dev-help_at_subversion.tigris.org
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-08-16 05:34:41 CEST