On Fri, 2010-09-24 at 01:40 -0400, Greg Stein wrote:
> Hey all,
>
> So I've been thinking of how we upgrade working copies to the new
> NODES table. As before, we have two scenarios to be concerned with:
>
> 1. formats 4..10: releases 1.0 thru 1.6
> 2. formats 11..19: 1.7-dev
>
> For (1), we revamp the old-entries-writing in entries.c, which is
> invoked as part of upgrade_to_wcng(). Rather than writing to the
> *_NODE tables, we will write to NODES and ACTUAL_NODE. This doesn't
> feel too hard, except for the testing part. Our current
> entries-writing code was tested via entry_modify() and friends when
> those functions were still in use. The only way to test this code path
> now is to create a large set of working copies with "all" state
> variations. Not fun. Maybe there is a way to automate this, or use our
> test suite to generate working copy datasets?
>
> For (2), I would prefer to do this in code, but being such a difficult
> series of transforms (prior auto-upgrades were quite simple), and that
> we've already had precedent for an external script... I'm thinking of
> just doing that.
>
> Thoughts?
Sounds good.
I think we should produce a test framework that can give us a WC
containing all the different possible WC states. Then we can write
tests against this framework, some tests that test specific state, and
other tests that apply the same operations to all the states and check
that it works in all the states that it should.
I have been thinking this for a while. As yet I've just got a
rudimentary constructor for a WC containing (nearly) all *base* states.
Not changes, yet. Here it is:
[[[
Add test suite functions to create a WC containing all valid base
states. I would define a "base state" as a state in which there is
nothing to commit and no unresolved conflicts.
* subversion/tests/cmdline/basic_tests.py
(add_to_version_control, get_complex_wc_repository_state,
make_complex_wc_base_state, make_complex_wc_checkout):
New functions.
(check_out_all_wc_base_states): New dummy test, to exercise these.
> Index: subversion/tests/cmdline/basic_tests.py
> ===================================================================
> --- subversion/tests/cmdline/basic_tests.py (revision 1000757)
> +++ subversion/tests/cmdline/basic_tests.py (working copy)
> @@ -2551,6 +2551,191 @@ def delete_and_add_same_file(sbox):
> None,
> wc_dir)
>
> +
> +#----------------------------------------------------------------------
> +
> +# Functions for testing a WC that contains all possible WC states.
> +
> +def add_to_version_control(state_desc, wc_dir):
> + """Create each item in STATE_DESC as a new versioned node in the existing
> + working copy WC_DIR.
> + STATE_DESC is a dictionary of { path string : StateItem object }."""
> +
> + # First, the directories. Be sure to create parents before their children.
> + dir_fullpaths = []
> + for path, state in sorted(state_desc.items()):
> + if state.contents is None: # a directory
> + fullpath = os.path.join(wc_dir, path)
> + dir_fullpaths += [fullpath]
> + svntest.actions.run_and_verify_svn(None, None, [],
> + 'mkdir', '--parents', *tuple(dir_fullpaths))
> + # Now all the files.
> + file_fullpaths = []
> + for path, state in state_desc.items():
> + if state.contents is not None: # a file
> + fullpath = os.path.join(wc_dir, path)
> + file_fullpaths += [fullpath]
> + open(fullpath, 'wb').write(state.contents)
> + svntest.actions.run_and_verify_svn(None, None, [],
> + 'add', *tuple(file_fullpaths))
> +
> + # Now set each node's properties. Create a dict {(pname,pval)=>[paths]},
> + # and run one propset per key, for speed.
> + props_map = {}
> + for path, state in state_desc.items():
> + fullpath = os.path.join(wc_dir, path)
> + for pname, pval in state.props.items():
> + props_map[(pname, pval)] = props_map.get((pname, pval), []) + [fullpath]
> + for (pname, pval), paths in props_map.items():
> + svntest.actions.run_and_verify_svn(None, None, [],
> + 'propset', pname, pval, *tuple(paths))
> +
> +def get_complex_wc_repository_state():
> + """Return a repository state suitable for use as a base for a complex WC.
> + Return ({ dict of StateItem objects }, { dict of authz rules }).
> +
> + - exclude/D/... 'D' will be excluded from the WC
> + - exclude/f 'f' will be excluded from the WC
> + - unauthz/D/... 'D' will be unauthorized for reading
> + - unauthz/f 'f' will be unauthorized for reading
> + - notpres/D/... 'D' will be updated to a 'not present' state in WC
> + - notpres/f 'f' will be updated to a 'not present' state in WC
> + - normal/D-empty dir, empty, no props
> + - normal/D/... dir with children and props
> + - normal/f-empty file, empty, no props
> + - normal/f file, with text, with props that affect it
> + ### TODO:
> + - extern/ dir, svn:externals maps 'D' to a dir, 'f' to a file
> + - symlinks
> + """
> +
> + Item = svntest.wc.StateItem
> + p = { 'p': 'v' }
> + p_complex = { 'svn:keywords': 'Rev',
> + 'svn:eol-style': 'native',
> + 'svn:executable': '*',
> + 'svn:needs-lock': '*' }
> + p_extern = { 'svn:externals': '^/normal/D D\n^/normal/f f' }
> + f = "This is a file."
> + f_complex = "Expanded keyword: $Rev$."
> +
> + state_items = {
> + 'exclude' : Item(),
> + 'exclude/D' : Item(props=p),
> + 'exclude/f' : Item(props=p, contents=f),
> + 'unauthz' : Item(),
> + 'unauthz/D' : Item(props=p),
> + 'unauthz/f' : Item(props=p, contents=f),
> + 'notpres' : Item(),
> + 'notpres/D' : Item(props=p),
> + 'notpres/f' : Item(props=p, contents=f),
> + 'normal' : Item(),
> + 'normal/D-empty' : Item(),
> + 'normal/D' : Item(props=p),
> + 'normal/D/D' : Item(props=p),
> + 'normal/D/f' : Item(props=p, contents=f),
> + 'normal/f-empty' : Item(contents=""),
> + 'normal/f' : Item(props=p_complex, contents=f_complex),
> + #'extern' : Item(props=p_extern),
> + }
> +
> + authz_rules = { '/': '*=rw' }
> + authz_rules['/unauthz/D'] = '*='
> + authz_rules['/unauthz/f'] = '*='
> +
> + return state_items, authz_rules
> +
> +def make_complex_wc_base_state(sbox):
> + """Create a repository revision to use as a base for a complex WC.
> + SBOX.repo_url must point to an existing repository.
> + ### Currently, SBOX.wc_dir must point to a WC at rev 1.
> + Return the dictionary of StateItems objects.
> + """
> +
> + wc_dir = sbox.wc_dir
> +
> + items, authz_rules = get_complex_wc_repository_state()
> +
> + # Make the WC empty, just for ease of checking the expected result.
> + svntest.actions.run_and_verify_update(wc_dir, None, None, None,
> + None, None, None, None, None, False,
> + '-r0', os.path.join(wc_dir, 'A'),
> + os.path.join(wc_dir, 'iota'))
> +
> + # Create the desired state in the WC ...
> + add_to_version_control(items, wc_dir)
> +
> + # ... and commit it to the repository.
> + expected_output = svntest.wc.State(wc_dir, {})
> + for path in items.keys():
> + expected_output.add({ path: Item(verb='Adding') })
> + expected_status = svntest.wc.State(wc_dir, {
> + '': Item(status=' ', wc_rev='1') })
> + for path in items.keys():
> + expected_status.add({ path: Item(status=' ', wc_rev='2') })
> + svntest.actions.run_and_verify_commit(wc_dir,
> + expected_output, expected_status,
> + None, wc_dir)
> +
> + # Change the repository authz to hide the paths intended to be hidden.
> + svntest.main.write_authz_file(sbox, authz_rules)
> +
> +def make_complex_wc_checkout(sbox):
> + """Check out a WC with all kinds of base states but no local mods.
> +
> + ### For now, assumes the WC is as left by make_complex_wc_base_state().
> +
> + - exclude/D/... 'D' will be excluded from the WC
> + - exclude/f 'f' will be excluded from the WC
> + - unauthz/D/... 'D' will be absent due to authz (status=absent)
> + - unauthz/f 'f' will be absent due to authz (status=absent)
> + - notpres/D/... 'D' will be updated to r0 (so status=not-present)
> + - notpres/f 'f' will be updated to r0 (so status=not-present)
> + - normal/D-empty depth=empty
> + - normal/D/... depth=infinity
> + - normal/f-empty file, empty, no props
> + - normal/f file, with text, with props that affect it
> + ### TODO:
> + - extern/D external dir => ^/extern/D-target
> + - extern/f external file => ^/extern/f-target
> + - symlink
> + - switched
> + - status=not-present due to updating to a *later* rev
> + """
> +
> + make_complex_wc_base_state(sbox)
> +
> + wc_dir = sbox.wc_dir
> + exclude_paths = (os.path.join(wc_dir, 'exclude', 'D'),
> + os.path.join(wc_dir, 'exclude', 'f'))
> + unauthz_paths = (os.path.join(wc_dir, 'unauthz', 'D'),
> + os.path.join(wc_dir, 'unauthz', 'f'))
> + extern_paths = (os.path.join(wc_dir, 'extern'), )
> + notpres_paths = (os.path.join(wc_dir, 'notpres', 'D'),
> + os.path.join(wc_dir, 'notpres', 'f'))
> +
> + # Exclude paths
> + svntest.actions.run_and_verify_update(wc_dir, None, None, None,
> + None, None, None, None, None, False,
> + '--set-depth=exclude', *exclude_paths)
> + # Get the correct authz-absent and externals states.
> + svntest.actions.run_and_verify_update(wc_dir, None, None, None,
> + None, None, None, None, None, False,
> + '-r0', *(unauthz_paths + extern_paths))
> + svntest.actions.run_and_verify_update(wc_dir, None, None, None,
> + None, None, None, None, None, False,
> + *(unauthz_paths + extern_paths))
> + # Make nodes 'not-present'
> + svntest.actions.run_and_verify_update(wc_dir, None, None, None,
> + None, None, None, None, None, False,
> + '-r0', *notpres_paths)
> +
> +def check_out_all_wc_base_states(sbox):
> + """check out all WC base states"""
> + sbox.build()
> + make_complex_wc_checkout(sbox)
> + # This doesn't currently test anything, it just creates WC base states.
> +
> #----------------------------------------------------------------------
>
> ########################################################################
> @@ -2611,6 +2796,7 @@ test_list = [ None,
> SkipUnless(meta_correct_library_being_used,
> svntest.main.is_ra_type_dav),
> delete_and_add_same_file,
> + check_out_all_wc_base_states,
> ]
>
> if __name__ == '__main__':
]]]
Any comments on whether or how to pursue that direction?
- Julian
Received on 2010-09-24 17:16:48 CEST