[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Status very long for big repository and status caching.

From: Max Bowsher <maxb_at_ukf.net>
Date: 2004-12-19 17:42:05 CET

Laurent CHASTEL wrote:
> Hello,
>
> I'm the "source code manager" of the firm I works for.
> I have implemented a solution using SVN.
> One part of our project was under CVS, I moved all to SVN a PowerBuilder
> application (2500 objects) and a database (5000 objects : tables, stored
> procedures, ...)
> We use tools that are SCC compliants, As there was no SCC dll for SVN (the
> project begins in august), I developped one.
> The SCC imposed us to have all files in one folder (one for database, one
> for PB)
>
> We tried to use the solution during this week, and it's impossible to use.
> We need 10 to 15 min to get the list of objects under SVN control for the
> database folder and the 10 min again when performing an operation.
>
> I have implemented the retrieve of the list by using the function
> svn_wc_status.
> I only want local status : is the file under SVN ? and is it locally
> modified ?
> The execution of this function is the longuest part of the treatment.
> Did I used the good function ? (You could find the source code I write at
> the end of the message)
...

Above you claim to be using svn_wc_status, but your code below is using
svn_client_status.

You are using the svn API in a horribly inefficient way.
For every file, you are:

1. Locking the WC
2. XML-parsing .svn/entries
3. Getting the status for every item in the directory.
4. Throwing away all but one of the statuses.
5. Unlocking the WC.

So, basically you are doing the amount of work that needs to be done,
squared.

As you have found out, 5000 squared is quite a large number.

You should be using svn_wc_adm_open2, then calling svn_wc_status for each
item you want the status of for the operation in progess, then calling
svn_wc_adm_close, OR you should call svn_client_status just *once*, and
fetch the status for each item from the hash you built in the
svn_client_status callback.

Max.

> The source code :
>
> svn_wc_status_t *getSVNStatus(CString fileName)
> {
> AFX_MANAGE_STATE(AfxGetStaticModuleState()); //LC
> CsccsvnApp *theApp = (CsccsvnApp *) AfxGetApp( );
> CSCCEditor *editor = theApp->getEditor(); // get editor
>
> apr_hash_t * statushash;
> apr_array_header_t * statusarray;
> const sort_item * item;
> svn_error_t * m_err;
> svn_wc_status_t * status;
> apr_pool_t * m_pool = svn_pool_create (NULL);
> CString internalpath;
> CString resToken;
> int curPos= 0;
> svn_client_ctx_t ctx;
>
> bool update = false; // do not change => will create connection to server,
> failed because no authentification defined
> bool noignore = false;
>
> if (fileName.GetLength() <= 0) return NULL;
>
> memset (&ctx, 0, sizeof (ctx));

BAD! Read the API docs for svn_client_create_context()

> m_err = svn_config_get_config (&(ctx.config), NULL, m_pool);
> if (m_err)
> {
> svn_pool_destroy (m_pool); // free the allocated memory
> return NULL;
> } // if (m_err)
>
> //we need to convert the path to subversion internal format
> //the internal format uses '/' instead of the windows '\'
> internalpath = fileName.GetString();
> internalpath.Replace("\\", "/");

Also bad. See svn_path_internal_style().

> statushash = apr_hash_make(m_pool);
> svn_revnum_t youngest = SVN_INVALID_REVNUM;
> svn_opt_revision_t rev;
> rev.kind = svn_opt_revision_unspecified;
> struct hashbaton_t hashbaton;
> hashbaton.hash = statushash;
> hashbaton.pool = m_pool;
> m_err = svn_client_status (&youngest, internalpath, &rev, getstatushash,
> &hashbaton, FALSE, TRUE, update, noignore, &ctx, m_pool);
>
> // Error present if function is not under version control
> if ((m_err != NULL) || (apr_hash_count(statushash) == 0))
> {
> return NULL;

Bad! Didn't destroy the pool!

> // delete => bug in pb7
> }
>
> // Convert the unordered hash to an ordered, sorted array
> statusarray = sort_hash (statushash, sort_compare_items_as_paths,
> m_pool);
>
> //only the first entry is needed (no recurse)
> item = &APR_ARRAY_IDX (statusarray, 0, const sort_item);
>
> status = (svn_wc_status_t *) item->value;
> svn_pool_destroy (m_pool);
>
> return status;
> }

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sun Dec 19 17:43:28 2004

This is an archived mail posted to the Subversion Dev mailing list.