Introduce the commit argument --include-externals. This affects commit's recursion regarding file and dir externals "in opposite ways", to converge towards consistent behavior across the two. NOTE: This affects only recursion, explicit targets are never skipped. During commit, never recurse to file or dir externals, except when --include-externals is passed. When --include-externals is passed, commit all file and dir externals, and also find file and dir externals that are "hidden" inside unversioned dirs [1] as well as dir externals "hidden behind" nested WCs. However, there are exceptions that prevent externals from being included in recursion: - Never recurse to pegged externals - Never recurse to (dir) externals coming from a different repos - [1] Because the commit API can't handle file externals inside unversioned dirs properly yet, some additional SQL currently carefully excludes them again (see STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW). Only use each target path's most immediate WC-root to find externals that are defined to be descendants of the target. Do not scan for externals definitions across WC-roots. So --include-externals does not include nested externals. Bumps svn_client_commit5() to &6(), introducing separate flag arguments INCLUDE_DIR_EXTERNALS and INCLUDE_FILE_EXTERNALS. Splitting the flags in two is practical when evaluating DEPTH, and may also be used to obtain previous behavior (only include file externals; when [1] goes away though, it will only be near previous behavior, because "hidden" file externals will be included). However, the commandline client only gets the general --include-externals option. Two fine grained ones could easily be added later. This patch does *not* verify whether the externals found in svn:externals definitions actually exist in the WC, nor whether they are checked out from the external's desired URL. It simply adds them as targets and lets the harvesting figure it out. A missing external will abort the commit. Any other situation will attempt to commit whatever is at that path. This behavior may be refined, but concrete testing so far has shown no harmful effect from committing the externals' paths unchecked, noting that they would anyway be included in recursion if they are no externals at all. Todo: try to make this break somehow and then fix it. See also http://svn.haxx.se/dev/archive-2011-08/0620.shtml Previously, all file externals were included by recursion. Recursion never reached into unversioned dirs, so "hidden" file externals were omitted. Dir externals were previously committed when passed as explicit targets, only. * subversion/include/private/svn_wc_private.h (svn_wc__committable_external_info_t): New struct for below function: (svn_wc__committable_externals_below): New function. * subversion/include/svn_client.h, * subversion/libsvn_client/commit.c (svn_client_commit6): New function, adding INCLUDE_FILE_EXTERNALS and INCLUDE_DIR_EXTERNALS arguments over svn_client_commit5(). Add all committable externals as explicit targets before harvesting committables. (svn_client_commit5): Call svn_client_commit6() with INCLUDE_FILE_EXTERNALS = TRUE and INCLUDE_DIR_EXTERNALS = FALSE. * subversion/libsvn_client/commit_util.c (harvest_committables): New argument IS_EXPLICIT_TARGET, pass as FALSE during recursion. Skip all file externals unless IS_EXPLICIT_TARGET == TRUE. (svn_client__harvest_committables: Pass IS_EXPLICIT_TARGET as TRUE, because outermost target. (harvest_copy_committables): Pass IS_EXPLICIT_TARGET as TRUE, but has no effect ATM since COPY_MODE inside harvest_committables() will be TRUE and all file externals will anyway be excluded. * subversion/libsvn_wc/externals.c (svn_wc__committable_externals_below): New function. * subversion/libsvn_wc/wc_db.h, * subversion/libsvn_wc/wc_db.c (svn_wc__db_committable_externals_below): New function wrapping STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW. * subversion/libsvn_wc/wc-queries.sql (STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW): New select. * subversion/svn/commit-cmd.c (svn_cl__commit): Call svn_client_commit6() instead of &5(). * subversion/svn/cl.h (svn_cl__opt_state_t): Add INCLUDE_EXTERNALS for --include-externals. * subversion/svn/main.c (svn_cl__longopt_t, svn_cl__options, svn_cl__cmd_table, main): Add OPT_INCLUDE_EXTERNALS, enable --include-externals for 'commit'. * subversion/tests/cmdline/externals_tests.py (include_externals): New test, PASS. (include_immediate_dir_externals): New test, XFAIL. (test_list): Add above two. --This line, and those below, will be ignored-- conf: # dub:/home/neels/pat/.pat-base/config-default conf: http://archive.apache.org/dist/apr/apr-1.4.5.tar.bz2 conf: http://archive.apache.org/dist/apr/apr-util-1.3.12.tar.bz2 conf: http://www.sqlite.org/sqlite-autoconf-3070701.tar.gz conf: http://archive.apache.org/dist/httpd/httpd-2.2.19.tar.bz2 conf: http://www.webdav.org/neon/neon-0.29.6.tar.gz conf: http://ftp2.de.freebsd.org/pub/FreeBSD/distfiles/bdb/db-5.1.25.tar.gz conf: ftp://ftp.fu-berlin.de/unix/gnu/libiconv/libiconv-1.14.tar.gz conf: fsfs conf: local Index: subversion/include/private/svn_wc_private.h =================================================================== --- subversion/include/private/svn_wc_private.h (revision 1164285) +++ subversion/include/private/svn_wc_private.h (working copy) @@ -131,6 +131,39 @@ svn_wc__read_external_info(svn_node_kind apr_pool_t *result_pool, apr_pool_t *scratch_pool); +/** See svn_wc__committable_externals_below(). */ +typedef struct svn_wc__committable_external_info_t { + + /* The local absolute path where the external should be checked out. */ + const char *local_abspath; + + /* Set to either svn_node_file or svn_node_dir. */ + svn_node_kind_t kind; + +} svn_wc__committable_external_info_t; + +/* Return in *EXTERNALS a list of svn_wc__committable_external_info_t * + * containing info on externals defined to be checked out below LOCAL_ABSPATH, + * returning only those externals that are not fixed to a specific revision. + * + * If there are no such externals, set *EXTERNALS to NULL. + * + * NOTE: This only returns the externals known by the WC that LOCAL_ABSPATH + * directly belongs to; i.e. when calling this on a node inside a checked-out + * dir external, this only returns those externals that are defined by + * svn:externals props belonging to the dir external's checkout, and does not + * return the externals defined in the parent WC. + * IOW, A parent WC may define further externals nested inside a hypothetical + * dir external. These can only be returned when passing a LOCAL_ABSPATH that + * directly belongs to the parent WC. */ +svn_error_t * +svn_wc__committable_externals_below(apr_array_header_t **externals, + svn_wc_context_t *wc_ctx, + const char *local_abspath, + svn_boolean_t immediates_only, + apr_pool_t *result_pool, + apr_pool_t *scratch_pool); + /* Gets a mapping from const char * local abspaths of externals to the const char * local abspath of where they are defined for all externals defined at or below LOCAL_ABSPATH. Index: subversion/include/svn_client.h =================================================================== --- subversion/include/svn_client.h (revision 1164285) +++ subversion/include/svn_client.h (working copy) @@ -1948,6 +1948,16 @@ svn_client_import(svn_client_commit_info * #TRUE, changes to descendants are only committed if they are itself * included via @a depth and targets. * + * If @a include_file_externals and @a include_dir_externals are true, all + * file respectively dir externals as defined by an svn:externals property are + * included in commit recursion, with these exceptions: any externals that + * - have a fixed peg revision or + * - that come from a different repository root URL (dir externals) + * are never included in recursion. + * ### TODO: currently, file externals hidden inside an unversioned dir are + * ### also skipped, because we can't commit those yet. + * ### See STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW. + * * When @a commit_as_operations is #TRUE it is possible to delete a node and * all its descendants by selecting just the root of the deletion. If it is * set to #FALSE this will raise an error. @@ -1961,6 +1971,26 @@ svn_client_import(svn_client_commit_info * * Use @a pool for any temporary allocations. * + * @since New in 1.8. + */ +svn_error_t * +svn_client_commit6(const apr_array_header_t *targets, + svn_depth_t depth, + svn_boolean_t keep_locks, + svn_boolean_t keep_changelists, + svn_boolean_t commit_as_operations, + svn_boolean_t include_file_externals, + svn_boolean_t include_dir_externals, + const apr_array_header_t *changelists, + const apr_hash_t *revprop_table, + svn_commit_callback2_t commit_callback, + void *commit_baton, + svn_client_ctx_t *ctx, + apr_pool_t *pool); + +/** + * Similar to svn_client_commit6(), but passes @a include_file_externals as + * TRUE and @a include_dir_externals as FALSE. * @since New in 1.7. */ svn_error_t * Index: subversion/libsvn_client/commit.c =================================================================== --- subversion/libsvn_client/commit.c (revision 1164285) +++ subversion/libsvn_client/commit.c (working copy) @@ -1167,11 +1167,13 @@ check_url_kind(void *baton, } svn_error_t * -svn_client_commit5(const apr_array_header_t *targets, +svn_client_commit6(const apr_array_header_t *targets, svn_depth_t depth, svn_boolean_t keep_locks, svn_boolean_t keep_changelists, svn_boolean_t commit_as_operations, + svn_boolean_t include_file_externals, + svn_boolean_t include_dir_externals, const apr_array_header_t *changelists, const apr_hash_t *revprop_table, svn_commit_callback2_t commit_callback, @@ -1231,6 +1233,105 @@ svn_client_commit5(const apr_array_heade if (rel_targets->nelts == 0) APR_ARRAY_PUSH(rel_targets, const char *) = ""; + /* Easy part of applying DEPTH to externals. */ + if (depth == svn_depth_empty) + { + /* If user passes an external as explicit target, the external *will* be + * handled; just stop all recursion to externals: */ + include_file_externals = include_dir_externals = FALSE; + } + else if (depth != svn_depth_infinity) + { + include_dir_externals = FALSE; + /* We slip in dir externals as explicit targets. When we do that, + * depth_immediates should become depth_empty for dir externals targets. + * But adding the dir external to the list of targets makes it get + * handled with depth_immediates itself, and thus will also include the + * immediate children of the dir external. So do dir externals only with + * depth_infinity or not at all. + * + * ### TODO: Maybe rework this into separate target lists, + * ### "duplicating" REL_TARGETS: one for the user's targets and one + * ### for the overlayed externals targets, and pass an appropriate + * ### depth for the externals targets in a separate call to + * ### svn_client__harvest_committables(). The only effect though is + * ### correct handling of this very specific case: during 'svn commit + * ### --depth=immediates --include-externals', commit dir externals + * ### (only immediate children of a target) with depth_empty instead + * ### of not at all. No other effect. So not doing that for now. */ + } + + /* If including externals, query every target for committable externals + * and append them as first class targets. All externals paths thus obtained + * are below BASE_ABSPATH, so we can just append more REL_TARGETS. */ + if (include_file_externals || include_dir_externals) + { + int rel_targets_nelts_fixed; + + /* Iterate *and* grow REL_TARGETS */ + rel_targets_nelts_fixed = rel_targets->nelts; + + for (i = 0; i < rel_targets_nelts_fixed; i++) { + apr_array_header_t *externals; + int j; + const char *target; + + svn_pool_clear(iterpool); + + target = svn_dirent_join(base_abspath, + APR_ARRAY_IDX(rel_targets, i, const char *), + iterpool); + + /* On svn_depth_files, all dirs are already filtered out by setting + * INCLUDE_DIR_EXTERNALS = FALSE above. So if depth < infinity + * here, simply allow only direct children of the target. */ + + SVN_ERR(svn_wc__committable_externals_below( + &externals, ctx->wc_ctx, target, + (depth != svn_depth_infinity), + iterpool, iterpool)); + + if (externals != NULL) + { + const char *rel_target; + + for (j = 0; j < externals->nelts; j++) + { + svn_wc__committable_external_info_t *xinfo = + APR_ARRAY_IDX(externals, j, + svn_wc__committable_external_info_t *); + + /* Does this external kind match the include_file/dir flags? */ + if ((xinfo->kind == svn_node_file + && ! include_file_externals) + || (xinfo->kind == svn_node_dir + && ! include_dir_externals)) + { + continue; + } + + /* "Slip in" another rel_target, to handle this external. + * Needs to be relative to BASE_ABSPATH. */ + rel_target = svn_dirent_skip_ancestor(base_abspath, + xinfo->local_abspath); + + SVN_ERR_ASSERT(rel_target != NULL && *rel_target != '\0'); + + /* ### TODO: We should maybe verify that the working copy + * actually has this external checked out: + * - dir ext'l: that it is from the same URL we're expecting + * it to be from. + * - file ext'l: there must be a BASE node for the checked-out + * external with a matching URL. */ + + APR_ARRAY_PUSH(rel_targets, const char *) = + apr_pstrdup(pool, rel_target); + } + } + } + + } + SVN_ERR(determine_lock_targets(&lock_targets, ctx->wc_ctx, base_abspath, rel_targets, pool, iterpool)); @@ -1601,3 +1702,24 @@ svn_client_commit5(const apr_array_heade return svn_error_trace(reconcile_errors(cmt_err, unlock_err, bump_err, pool)); } + +svn_error_t * +svn_client_commit5(const apr_array_header_t *targets, + svn_depth_t depth, + svn_boolean_t keep_locks, + svn_boolean_t keep_changelists, + svn_boolean_t commit_as_operations, + const apr_array_header_t *changelists, + const apr_hash_t *revprop_table, + svn_commit_callback2_t commit_callback, + void *commit_baton, + svn_client_ctx_t *ctx, + apr_pool_t *pool) +{ + return svn_client_commit6(targets, depth, keep_locks, keep_changelists, + commit_as_operations, + TRUE, FALSE, + changelists, revprop_table, commit_callback, + commit_baton, ctx, pool); +} + Index: subversion/libsvn_client/commit_util.c =================================================================== --- subversion/libsvn_client/commit_util.c (revision 1164285) +++ subversion/libsvn_client/commit_util.c (working copy) @@ -410,6 +410,9 @@ bail_on_tree_conflicted_ancestor(svn_wc_ when harvesting committables; that is, don't add a path to COMMITTABLES unless it's a member of one of those changelists. + IS_EXPLICIT_TARGET should always be passed as TRUE (except when + harvest_committables() calls itself in recursion). + If CANCEL_FUNC is non-null, call it with CANCEL_BATON to see if the user has cancelled the operation. @@ -428,6 +431,7 @@ harvest_committables(svn_wc_context_t *w apr_hash_t *changelists, svn_boolean_t skip_files, svn_boolean_t skip_dirs, + svn_boolean_t is_explicit_target, svn_client__check_url_kind_t check_url_func, void *check_url_baton, svn_cancel_func_t cancel_func, @@ -543,10 +547,24 @@ harvest_committables(svn_wc_context_t *w svn_dirent_local_style(local_abspath, scratch_pool)); } - /* ### in need of comment */ - if (copy_mode - && is_update_root - && db_kind == svn_node_file) + /* Handle file externals. + * (IS_UPDATE_ROOT is more generally defined, but at the moment this + * condition matches only file externals.) + * + * Don't copy files that svn:externals brought into the WC. So in copy_mode, + * even explicit targets are skipped. + * + * Exclude file externals from recursion. Hande file externals only when + * passed as explicit target. Note that svn_client_commit6() passes all + * committable externals in as explicit targets iff they count. + * + * Also note that dir externals will never be reached recursively by this + * function, since svn_wc__node_get_children_of_working_node() (used below + * to recurse) does not return switched working copies. */ + if (is_update_root + && db_kind == svn_node_file + && (copy_mode + || ! is_explicit_target)) { return SVN_NO_ERROR; } @@ -841,6 +859,7 @@ harvest_committables(svn_wc_context_t *w changelists, (depth < svn_depth_files), (depth < svn_depth_immediates), + FALSE, /* is_explicit_target */ check_url_func, check_url_baton, cancel_func, cancel_baton, notify_func, notify_baton, @@ -1125,6 +1144,7 @@ svn_client__harvest_committables(svn_cli FALSE /* COPY_MODE_ROOT */, depth, just_locked, changelist_hash, FALSE, FALSE, + TRUE /* IS_EXPLICIT_TARGET */, check_url_func, check_url_baton, ctx->cancel_func, ctx->cancel_baton, ctx->notify_func2, ctx->notify_baton2, @@ -1218,6 +1238,7 @@ harvest_copy_committables(void *baton, v FALSE, /* JUST_LOCKED */ NULL, FALSE, FALSE, /* skip files, dirs */ + TRUE, /* IS_EXPLICIT_TARGET (don't care) */ btn->check_url_func, btn->check_url_baton, btn->ctx->cancel_func, Index: subversion/libsvn_wc/externals.c =================================================================== --- subversion/libsvn_wc/externals.c (revision 1164285) +++ subversion/libsvn_wc/externals.c (working copy) @@ -1116,6 +1116,21 @@ svn_wc__read_external_info(svn_node_kind } svn_error_t * +svn_wc__committable_externals_below(apr_array_header_t **externals, + svn_wc_context_t *wc_ctx, + const char *local_abspath, + svn_boolean_t immediates_only, + apr_pool_t *result_pool, + apr_pool_t *scratch_pool) +{ + return svn_error_trace( + svn_wc__db_committable_externals_below(externals, + wc_ctx->db, local_abspath, + immediates_only, + result_pool, scratch_pool)); +} + +svn_error_t * svn_wc__externals_defined_below(apr_hash_t **externals, svn_wc_context_t *wc_ctx, const char *local_abspath, Index: subversion/libsvn_wc/wc_db.c =================================================================== --- subversion/libsvn_wc/wc_db.c (revision 1164285) +++ subversion/libsvn_wc/wc_db.c (working copy) @@ -3058,6 +3058,66 @@ svn_wc__db_external_read(svn_wc__db_stat svn_error_compose_create(err, svn_sqlite__reset(stmt))); } + +svn_error_t * +svn_wc__db_committable_externals_below(apr_array_header_t **externals, + svn_wc__db_t *db, + const char *local_abspath, + svn_boolean_t immediates_only, + apr_pool_t *result_pool, + apr_pool_t *scratch_pool) +{ + svn_wc__db_wcroot_t *wcroot; + svn_sqlite__stmt_t *stmt; + const char *local_relpath; + svn_boolean_t have_row; + svn_wc__committable_external_info_t *info; + svn_wc__db_kind_t db_kind; + apr_array_header_t *result = NULL; + + SVN_ERR_ASSERT(svn_dirent_is_absolute(local_abspath)); + + SVN_ERR(svn_wc__db_wcroot_parse_local_abspath(&wcroot, &local_relpath, db, + local_abspath, scratch_pool, scratch_pool)); + VERIFY_USABLE_WCROOT(wcroot); + + SVN_ERR(svn_sqlite__get_statement(&stmt, wcroot->sdb, + STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW)); + + SVN_ERR(svn_sqlite__bindf(stmt, "isi", wcroot->wc_id, local_relpath, + (apr_int64_t)(immediates_only ? 1 : 0))); + + SVN_ERR(svn_sqlite__step(&have_row, stmt)); + + if (have_row) + result = apr_array_make(result_pool, 0, + sizeof(svn_wc__committable_external_info_t *)); + + while (have_row) + { + info = apr_palloc(result_pool, sizeof(*info)); + + local_relpath = svn_sqlite__column_text(stmt, 0, NULL); + info->local_abspath = svn_dirent_join(wcroot->abspath, local_relpath, + result_pool); + + db_kind = svn_sqlite__column_token(stmt, 1, kind_map); + if (db_kind == svn_wc__db_kind_file) + info->kind = svn_node_file; + else if (db_kind == svn_wc__db_kind_dir) + info->kind = svn_node_dir; + else + SVN_ERR_MALFUNCTION(); + + APR_ARRAY_PUSH(result, svn_wc__committable_external_info_t *) = info; + + SVN_ERR(svn_sqlite__step(&have_row, stmt)); + } + + *externals = result; + return svn_error_trace(svn_sqlite__reset(stmt)); +} + svn_error_t * svn_wc__db_externals_defined_below(apr_hash_t **externals, svn_wc__db_t *db, Index: subversion/libsvn_wc/wc_db.h =================================================================== --- subversion/libsvn_wc/wc_db.h (revision 1164285) +++ subversion/libsvn_wc/wc_db.h (working copy) @@ -1164,6 +1164,26 @@ svn_wc__db_external_read(svn_wc__db_stat apr_pool_t *result_pool, apr_pool_t *scratch_pool); +/* Return in *EXTERNALS a list of const char * local abspaths of externals + * defined to be checked out below LOCAL_ABSPATH, returning all except those + * externals that are fixed to a specific revision. + * + * NOTE: This only returns the externals known by the WC that LOCAL_ABSPATH + * directly belongs to; i.e. when calling this on a node inside a checked-out + * dir external, this only returns those externals that are defined by + * svn:externals props belonging to the dir external's checkout, and does not + * return the externals defined in the parent WC. + * IOW, A parent WC may define further externals nested inside a hypothetical + * dir external. These are only returned when passing a LOCAL_ABSPATH that + * directly belongs to the parent WC. */ +svn_error_t * +svn_wc__db_committable_externals_below(apr_array_header_t **externals, + svn_wc__db_t *db, + const char *local_abspath, + svn_boolean_t immediates_only, + apr_pool_t *result_pool, + apr_pool_t *scratch_pool); + /* Gets a mapping from const char * local abspaths of externals to the const char * local abspath of where they are defined for all externals defined at or below LOCAL_ABSPATH. Index: subversion/libsvn_wc/wc-queries.sql =================================================================== --- subversion/libsvn_wc/wc-queries.sql (revision 1164285) +++ subversion/libsvn_wc/wc-queries.sql (working copy) @@ -958,6 +958,38 @@ SELECT presence, kind, def_local_relpath FROM externals WHERE wc_id = ?1 AND local_relpath = ?2 LIMIT 1 +/* Select all committable externals, i.e. only unpegged ones on the same + * repository as the target path ?2, that are defined by WC ?1 to + * live below the target path. It does not matter which ancestor has the + * svn:externals definition, only the local path at which the external is + * supposed to be checked out is interesting here. + * ?1 wc_id + * ?2 the target path, local relpath inside ?1 + * + * ### NOTE: This statement carefully removes those file externals that live + * inside an unversioned dir, because the commit API can't handle those yet. + * Once that has been fixed, the conditions checking for an existing versioned + * parent on file externals (see "--->8---") becomes obsolete. */ +-- STMT_SELECT_COMMITTABLE_EXTERNALS_BELOW +SELECT e.local_relpath, e.kind +FROM externals AS e +WHERE e.wc_id = ?1 + AND e.def_revision IS NULL + AND e.repos_id = (SELECT repos_id FROM nodes + WHERE local_relpath = ?2) + AND ( ((NOT ?3) + AND (?2 = '' + /* Want only the cildren of e.local_relpath; + * externals can't have a local_relpath = ''. */ + OR IS_STRICT_DESCENDANT_OF(e.local_relpath, ?2))) + OR + ((?3) + AND e.parent_relpath = ?2) ) + AND /* ------>8------- */ + (EXISTS (SELECT 1 FROM nodes + WHERE wc_id = e.wc_id + AND local_relpath = e.parent_relpath)) + -- STMT_SELECT_EXTERNALS_DEFINED SELECT local_relpath, def_local_relpath FROM externals Index: subversion/svn/cl.h =================================================================== --- subversion/svn/cl.h (revision 1164285) +++ subversion/svn/cl.h (working copy) @@ -230,6 +230,7 @@ typedef struct svn_cl__opt_state_t svn_boolean_t internal_diff; /* override diff_cmd in config file */ svn_boolean_t use_git_diff_format; /* Use git's extended diff format */ svn_boolean_t allow_mixed_rev; /* Allow operation on mixed-revision WC */ + svn_boolean_t include_externals; /* Recurses (in)to file & dir externals */ } svn_cl__opt_state_t; Index: subversion/svn/commit-cmd.c =================================================================== --- subversion/svn/commit-cmd.c (revision 1164285) +++ subversion/svn/commit-cmd.c (working copy) @@ -166,11 +166,13 @@ svn_cl__commit(apr_getopt_t *os, } /* Commit. */ - err = svn_client_commit5(targets, + err = svn_client_commit6(targets, opt_state->depth, no_unlock, opt_state->keep_changelists, TRUE /* commit_as_operations */, + opt_state->include_externals, /* file externals */ + opt_state->include_externals, /* dir externals */ opt_state->changelists, opt_state->revprop_table, ! opt_state->quiet Index: subversion/svn/main.c =================================================================== --- subversion/svn/main.c (revision 1164285) +++ subversion/svn/main.c (working copy) @@ -125,6 +125,7 @@ typedef enum svn_cl__longopt_t { opt_internal_diff, opt_use_git_diff_format, opt_allow_mixed_revisions, + opt_include_externals, } svn_cl__longopt_t; @@ -342,6 +343,12 @@ const apr_getopt_option_t svn_cl__option "Use of this option is not recommended!\n" " " "Please run 'svn update' instead.")}, + {"include-externals", opt_include_externals, 0, + N_("Also commit file and dir externals reached by\n" + " " + "recursion. This does not include externals with a\n" + " " + "fixed revision. (See the svn:externals property)")}, /* Long-opt Aliases * @@ -460,7 +467,7 @@ const svn_opt_subcommand_desc2_t svn_cl_ " If any targets are (or contain) locked items, those will be\n" " unlocked after a successful commit.\n"), {'q', 'N', opt_depth, opt_targets, opt_no_unlock, SVN_CL__LOG_MSG_OPTIONS, - opt_changelist, opt_keep_changelists} }, + opt_changelist, opt_keep_changelists, opt_include_externals} }, { "copy", svn_cl__copy, {"cp"}, N_ ("Duplicate something in working copy or repository, remembering\n" @@ -2030,6 +2037,9 @@ main(int argc, const char *argv[]) case opt_allow_mixed_revisions: opt_state.allow_mixed_rev = TRUE; break; + case opt_include_externals: + opt_state.include_externals = TRUE; + break; default: /* Hmmm. Perhaps this would be a good place to squirrel away opts that commands like svn diff might need. Hmmm indeed. */ Index: subversion/tests/cmdline/externals_tests.py =================================================================== --- subversion/tests/cmdline/externals_tests.py (revision 1164285) +++ subversion/tests/cmdline/externals_tests.py (working copy) @@ -2020,6 +2020,475 @@ def commit_file_external(sbox): expected_status, None, None, None, None, None, False, wc_dir) +def include_externals(sbox): + "commit with --include-externals" + # svntest.factory.make(sbox, """ + # svn mkdir --parents Xpegged X/Y + # svn ci + # svn up + # svn ps svn:externals "^/iota@1 Xpegged/xiota" . + # """) + # exit(0) + sbox.build() + wc_dir = sbox.wc_dir + + X_Y = os.path.join(wc_dir, 'X', 'Y') + Xpegged = os.path.join(wc_dir, 'Xpegged') + + # svn mkdir --parents Xpegged X/Y + expected_stdout = verify.UnorderedOutput([ + 'A ' + Xpegged + '\n', + 'A ' + wc_dir + '/X\n', + 'A ' + X_Y + '\n', + ]) + + actions.run_and_verify_svn2('OUTPUT', expected_stdout, [], 0, 'mkdir', + '--parents', Xpegged, X_Y) + + # svn ci + expected_output = svntest.wc.State(wc_dir, { + 'X' : Item(verb='Adding'), + 'X/Y' : Item(verb='Adding'), + 'Xpegged' : Item(verb='Adding'), + }) + + expected_status = actions.get_virginal_state(wc_dir, 1) + expected_status.add({ + 'X' : Item(status=' ', wc_rev='2'), + 'X/Y' : Item(status=' ', wc_rev='2'), + 'Xpegged' : Item(status=' ', wc_rev='2'), + }) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, wc_dir) + + # svn up + expected_output = svntest.wc.State(wc_dir, {}) + + expected_disk = svntest.main.greek_state.copy() + expected_disk.add({ + 'Xpegged' : Item(), + 'X' : Item(), + 'X/Y' : Item(), + }) + + expected_status.tweak(wc_rev='2') + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # svn ps svn:externals "^/iota@1 Xpegged/xiota" . + expected_stdout = ["property 'svn:externals' set on '" + wc_dir + "'\n"] + + actions.run_and_verify_svn2('OUTPUT', expected_stdout, [], 0, 'ps', + 'svn:externals', ''' + ^/iota@1 Xpegged/xiota + -r1 ^/A/B/E Xpegged/xE + ^/A/mu X/xmu + ^/A/B/lambda X/Y/xlambda + ^/A/D/G X/xG + ^/A/D/H X/Y/xH + ''', wc_dir) + + # svntest.factory.make(sbox, prev_disk=expected_disk, + # prev_status=expected_status, + # commands = """ + # svn ci + # svn up + # echo mod >> Xpegged/xE/alpha + # echo mod >> X/xmu + # echo mod >> X/Y/xlambda + # echo mod >> X/xG/pi + # echo mod >> X/Y/xH/chi + # svn status + # # Expect no externals to be committed + # svn ci + # # Expect no externals to be committed, because pegged + # svn ci --include-externals Xpegged + # # Expect no externals to be committed, because of depth + # svn ci --depth=immediates --include-externals + # # Expect only unpegged externals to be committed (those in X/) + # svn ci --include-externals + # svn up + # # new mods to check more cases + # echo mod >> X/xmu + # echo mod >> X/Y/xlambda + # echo mod >> X/xG/pi + # echo mod >> X/Y/xH/chi + # svn status + # # Expect no externals to be committed, because of depth + # svn ci --include-externals --depth=empty X + # # Expect only file external xmu to be committed, because of depth + # svn ci --include-externals --depth=files X + # svn status + # svn up + # echo mod >> X/Y/xlambda + # echo mod >> X/xG/pi + # svn status + # # Expect explicit targets to be committed + # svn ci X/Y/xlambda X/xG + # svn status + # """) + + X = os.path.join(wc_dir, 'X') + X_xG = os.path.join(wc_dir, 'X', 'xG') + X_xG_pi = os.path.join(wc_dir, 'X', 'xG', 'pi') + X_xmu = os.path.join(wc_dir, 'X', 'xmu') + X_Y_xH_chi = os.path.join(wc_dir, 'X', 'Y', 'xH', 'chi') + X_Y_xlambda = os.path.join(wc_dir, 'X', 'Y', 'xlambda') + Xpegged = os.path.join(wc_dir, 'Xpegged') + Xpegged_xE_alpha = os.path.join(wc_dir, 'Xpegged', 'xE', 'alpha') + + # svn ci + expected_output = svntest.wc.State(wc_dir, { + '' : Item(verb='Sending'), + }) + + expected_status.tweak('', wc_rev='3') + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, wc_dir) + + # svn up + expected_output = svntest.wc.State(wc_dir, { + 'X/xmu' : Item(status='A '), + 'X/xG/tau' : Item(status='A '), + 'X/xG/rho' : Item(status='A '), + 'X/xG/pi' : Item(status='A '), + 'X/Y/xlambda' : Item(status='A '), + 'X/Y/xH/psi' : Item(status='A '), + 'X/Y/xH/chi' : Item(status='A '), + 'X/Y/xH/omega' : Item(status='A '), + 'Xpegged/xiota' : Item(status='A '), + 'Xpegged/xE/alpha' : Item(status='A '), + 'Xpegged/xE/beta' : Item(status='A '), + }) + + expected_disk.add({ + 'Xpegged/xE' : Item(), + 'Xpegged/xE/beta' : Item(contents="This is the file 'beta'.\n"), + 'Xpegged/xE/alpha' : Item(contents="This is the file 'alpha'.\n"), + 'Xpegged/xiota' : Item(contents="This is the file 'iota'.\n"), + 'X/Y/xlambda' : Item(contents="This is the file 'lambda'.\n"), + 'X/Y/xH' : Item(), + 'X/Y/xH/chi' : Item(contents="This is the file 'chi'.\n"), + 'X/Y/xH/psi' : Item(contents="This is the file 'psi'.\n"), + 'X/Y/xH/omega' : Item(contents="This is the file 'omega'.\n"), + 'X/xmu' : Item(contents="This is the file 'mu'.\n"), + 'X/xG' : Item(), + 'X/xG/tau' : Item(contents="This is the file 'tau'.\n"), + 'X/xG/rho' : Item(contents="This is the file 'rho'.\n"), + 'X/xG/pi' : Item(contents="This is the file 'pi'.\n"), + }) + + expected_status.tweak(wc_rev='3') + expected_status.add({ + 'Xpegged/xiota' : Item(status=' ', wc_rev='1', switched='X'), + 'Xpegged/xE' : Item(status='X '), + 'X/Y/xH' : Item(status='X '), + 'X/Y/xlambda' : Item(status=' ', wc_rev='3', switched='X'), + 'X/xmu' : Item(status=' ', wc_rev='3', switched='X'), + 'X/xG' : Item(status='X '), + }) + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # echo mod >> Xpegged/xE/alpha + main.file_append(Xpegged_xE_alpha, 'mod\n') + + # echo mod >> X/xmu + main.file_append(X_xmu, 'mod\n') + + # echo mod >> X/Y/xlambda + main.file_append(X_Y_xlambda, 'mod\n') + + # echo mod >> X/xG/pi + main.file_append(X_xG_pi, 'mod\n') + + # echo mod >> X/Y/xH/chi + main.file_append(X_Y_xH_chi, 'mod\n') + + # svn status + expected_status.tweak('X/Y/xlambda', 'X/xmu', status='M ') + + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + + # Expect no externals to be committed + # svn ci + expected_output = svntest.wc.State(wc_dir, {}) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, wc_dir) + + # Expect no externals to be committed, because pegged + # svn ci --include-externals Xpegged + expected_output = svntest.wc.State(wc_dir, {}) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--include-externals', Xpegged) + + # Expect no externals to be committed, because of depth + # svn ci --depth=immediates --include-externals + expected_output = svntest.wc.State(wc_dir, {}) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--depth=immediates', '--include-externals', wc_dir) + + # Expect only unpegged externals to be committed (those in X/) + # svn ci --include-externals + expected_output = svntest.wc.State(wc_dir, { + 'X/xmu' : Item(verb='Sending'), + 'X/Y/xlambda' : Item(verb='Sending'), + 'X/Y/xH/chi' : Item(verb='Sending'), + 'X/xG/pi' : Item(verb='Sending'), + }) + + expected_status.tweak(status=' ') + expected_status.tweak('X/Y/xlambda', 'X/xmu', wc_rev='4') + expected_status.tweak('X/Y/xH', 'X/xG', 'Xpegged/xE', status='X ') + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--include-externals', wc_dir) + + # svn up + expected_output = svntest.wc.State(wc_dir, { + 'A/D/G/pi' : Item(status='U '), + 'A/D/H/chi' : Item(status='U '), + 'A/mu' : Item(status='U '), + 'A/B/lambda' : Item(status='U '), + }) + + expected_disk.tweak('Xpegged/xE/alpha', + contents="This is the file 'alpha'.\nmod\n") + expected_disk.tweak('A/D/H/chi', 'X/Y/xH/chi', + contents="This is the file 'chi'.\nmod\n") + expected_disk.tweak('A/D/G/pi', 'X/xG/pi', + contents="This is the file 'pi'.\nmod\n") + expected_disk.tweak('A/mu', 'X/xmu', + contents="This is the file 'mu'.\nmod\n") + expected_disk.tweak('A/B/lambda', 'X/Y/xlambda', + contents="This is the file 'lambda'.\nmod\n") + + expected_status.tweak(wc_rev='4') + expected_status.tweak('Xpegged/xiota', wc_rev='1') + expected_status.tweak('Xpegged/xE', wc_rev=None) + expected_status.tweak('X/Y/xH', wc_rev=None) + expected_status.tweak('X/xG', wc_rev=None) + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # new mods to check more cases + # echo mod >> X/xmu + main.file_append(X_xmu, 'mod\n') + + # echo mod >> X/Y/xlambda + main.file_append(X_Y_xlambda, 'mod\n') + + # echo mod >> X/xG/pi + main.file_append(X_xG_pi, 'mod\n') + + # echo mod >> X/Y/xH/chi + main.file_append(X_Y_xH_chi, 'mod\n') + + # svn status + expected_status.tweak('X/Y/xlambda', 'X/xmu', status='M ') + + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + + # Expect no externals to be committed, because of depth + # svn ci --include-externals --depth=empty X + expected_output = svntest.wc.State(wc_dir, {}) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--include-externals', '--depth=empty', X) + + # Expect only file external xmu to be committed, because of depth + # svn ci --include-externals --depth=files X + expected_output = svntest.wc.State(wc_dir, { + 'X/xmu' : Item(verb='Sending'), + }) + + expected_status.tweak(status=' ') + expected_status.tweak('X/xmu', wc_rev='5') + expected_status.tweak('Xpegged/xE', 'X/Y/xH', 'X/xG', status='X ') + expected_status.tweak('X/Y/xlambda', status='M ') + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--include-externals', '--depth=files', X) + + # svn status + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + + # svn up + expected_output = svntest.wc.State(wc_dir, { + 'A/mu' : Item(status='U '), + }) + + expected_disk.tweak('A/mu', 'X/xmu', + contents="This is the file 'mu'.\nmod\nmod\n") + expected_disk.tweak('X/Y/xlambda', + contents="This is the file 'lambda'.\nmod\nmod\n") + expected_disk.tweak('X/Y/xH/chi', + contents="This is the file 'chi'.\nmod\nmod\n") + expected_disk.tweak('X/xG/pi', + contents="This is the file 'pi'.\nmod\nmod\n") + + expected_status.tweak(wc_rev='5') + expected_status.tweak('Xpegged/xiota', wc_rev='1') + expected_status.tweak('Xpegged/xE', wc_rev=None) + expected_status.tweak('X/Y/xH', wc_rev=None) + expected_status.tweak('X/xG', wc_rev=None) + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # echo mod >> X/Y/xlambda + main.file_append(X_Y_xlambda, 'mod\n') + + # echo mod >> X/xG/pi + main.file_append(X_xG_pi, 'mod\n') + + # svn status + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + + # Expect explicit targets to be committed + # svn ci X/Y/xlambda X/xG + expected_output = svntest.wc.State(wc_dir, { + 'X/Y/xlambda' : Item(verb='Sending'), + 'X/xG/pi' : Item(verb='Sending'), + }) + + expected_status.tweak(status=' ') + expected_status.tweak('X/Y/xlambda', wc_rev='6') + expected_status.tweak('X/Y/xH', 'X/xG', 'Xpegged/xE', status='X ') + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, X_Y_xlambda, X_xG) + + # svn status + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + +@XFail() +def include_immediate_dir_externals(sbox): + "commit --include-externals --depth=immediates" + # See also comment inside svn_client_commit6(). + + # svntest.factory.make(sbox,""" + # svn mkdir X + # svn ci + # svn up + # svn ps svn:externals "^/A/B/E X/XE" wc_dir + # svn ci + # svn up + # + # svn ps some change X/XE + # echo mod >> X/XE/alpha + # + # svn st X/XE + # # Expect only the propset on X/XE to be committed. + # # Should be like svn commit --include-externals --depth=empty X/XE + # svn commit --include-externals --depth=immediates X + # """) + + sbox.build() + wc_dir = sbox.wc_dir + + X = os.path.join(wc_dir, 'X') + X_XE = os.path.join(wc_dir, 'X', 'XE') + X_XE_alpha = os.path.join(wc_dir, 'X', 'XE', 'alpha') + + # svn mkdir X + expected_stdout = ['A ' + X + '\n'] + + actions.run_and_verify_svn2('OUTPUT', expected_stdout, [], 0, 'mkdir', X) + + # svn ci + expected_output = svntest.wc.State(wc_dir, { + 'X' : Item(verb='Adding'), + }) + + expected_status = actions.get_virginal_state(wc_dir, 1) + expected_status.add({ + 'X' : Item(status=' ', wc_rev='2'), + }) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, wc_dir) + + # svn up + expected_output = svntest.wc.State(wc_dir, {}) + + expected_disk = svntest.main.greek_state.copy() + expected_disk.add({ + 'X' : Item(), + }) + + expected_status.tweak(wc_rev='2') + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # svn ps svn:externals "^/A/B/E X/XE" wc_dir + expected_stdout = ["property 'svn:externals' set on '" + wc_dir + "'\n"] + + actions.run_and_verify_svn2('OUTPUT', expected_stdout, [], 0, 'ps', + 'svn:externals', '^/A/B/E X/XE', wc_dir) + + # svn ci + expected_output = svntest.wc.State(wc_dir, { + '' : Item(verb='Sending'), + }) + + expected_status.tweak('', wc_rev='3') + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, wc_dir) + + # svn up + expected_output = svntest.wc.State(wc_dir, { + 'X/XE/alpha' : Item(status='A '), + 'X/XE/beta' : Item(status='A '), + }) + + expected_disk.add({ + 'X/XE' : Item(), + 'X/XE/alpha' : Item(contents="This is the file 'alpha'.\n"), + 'X/XE/beta' : Item(contents="This is the file 'beta'.\n"), + }) + + expected_status.tweak(wc_rev='3') + expected_status.add({ + 'X/XE' : Item(status='X '), + }) + + actions.run_and_verify_update(wc_dir, expected_output, expected_disk, + expected_status, None, None, None, None, None, False, wc_dir) + + # svn ps some change X/XE + expected_stdout = ["property 'some' set on '" + X_XE + "'\n"] + + actions.run_and_verify_svn2('OUTPUT', expected_stdout, [], 0, 'ps', 'some', + 'change', X_XE) + + # echo mod >> X/XE/alpha + main.file_append(X_XE_alpha, 'mod\n') + + # svn st X/XE + actions.run_and_verify_unquiet_status(wc_dir, expected_status) + + # Expect only the propset on X/XE to be committed. + # Should be like svn commit --include-externals --depth=empty X/XE + # svn commit --include-externals --depth=immediates X + expected_output = svntest.wc.State(wc_dir, { + 'X/XE' : Item(verb='Sending'), + }) + + actions.run_and_verify_commit(wc_dir, expected_output, expected_status, + None, '--include-externals', '--depth=immediates', X) + + ######################################################################## # Run the tests @@ -2061,6 +2530,8 @@ test_list = [ None, file_externals_different_repos, file_external_in_unversioned, commit_file_external, + include_externals, + include_immediate_dir_externals, ] if __name__ == '__main__':