[PATCH] improve speed of non-verbose svn list

So I followed through on my recently discussed plans regarding
providing a way for the client to indicate which parts of the
svn_dirent_t they really care about when calling svn_client_ls.
Here's a patch that implements two new functions svn_client_ls4 and
svn_ra_get_dir2 that provide this capability and changes the command
line client to use svn_client_ls4. When we're not in verbose or XML
mode for list we only ask for the kind field of the dirent to be
filled in, and as a result the speed of the operation is greatly
improved.

It turns out that this requires no changes to mod_dav_svn, as DAV
already provides us enough functionality to control what properties
we're interested in. Note that there is still some inefficiency in
the way has_props is calculated, and the addition of the
deadprops-count liveprop (from Jean-Marc Godbout's patch in issue
2151) would certainly help in that case, but it isn't necessary to
improve the non-verbose case, so I've left that for another day.

The svnserve changes here are the minimum required to make this work.
It does result in a speed increase simply from not calculating the
information on the server side, but we are still using extra bandwidth
to transmit even the portions of the dirent we don't care about. If
someone is motivated enough it would be possible to fix this, but it
is a more complex change.

I haven't done any work on libsvn_ra_local here, so it is still
exactly the same speed it was before.

I've tested svnserve with both old and new servers and clients, and
everything works as expected. Since this hasn't required any
mod_dav_svn changes there are no compatibility problems there.

While investigating this I may have found some ways to speed up the
general case for ra_dav (which is still slow much slower than ra_svn
in the verbose case), but regardless I still think these changes have
merit, since no matter how fast we make the calculation of the dirent
fields it's still silly to calculate and send them when they are never
even used.

Any comments on this would be greatly appreciated.

-garrett

[[[
Speed up non-verbose 'svn list' by a scary amount by allowing the server
to avoid calculating parts of the svn_dirent_t objects that are never used.

* subversion/libsvn_ra/ra_loader.c
  (svn_ra_get_dir2): new wrapper function.

* subversion/libsvn_ra/ra_loader.h
  (svn_ra__vtable_t): add get_dir2 function.

* subversion/include/svn_types.h
  (SVN_DIRENT_KIND,
   SVN_DIRENT_SIZE,
   SVN_DIRENT_HAS_PROPS,
   SVN_DIRENT_CREATED_REV,
   SVN_DIRENT_TIME,
   SVN_DIRENT_LAST_AUTHOR): new bitfield values for each field in a dirent.
  (SVN_DIRENT_ALL): combination of all the other bitfield values.

* subversion/include/svn_client.h
  (svn_client_ls4): new function, like svn_client_ls3, but with a dirent
   fields parameter.
  (svn_client_ls3): deprecated in favor of svn_client_ls4.

* subversion/include/svn_ra.h
  (svn_ra_get_dir2): new version of svn_ra_get_dir, takes a dirent fields
   parameter.
  (svn_ra_get_dir): deprecate in favor of svn_ra_get_dir2.

* subversion/libsvn_ra_local/ra_plugin.c
  (svn_ra_local__get_dir2): new function, currently ignores the dirent
   fields parameter.
  (svn_ra_local__get_dir): call into svn_ra_local__get_dir, passing it
   SVN_DIRENT_ALL for the dirent_fields parameter.
  (ra_local_vtable): add svn_ra_Local__get_dir2.

* subversion/libsvn_client/ls.c
  (get_dir_contents): add a dirent_fields parameter, use svn_ra_get_dir2,
   update recursive call.
  (svn_client_ls4): new list function, adds a dirent_fields parameter which
   is passed to get_dir_contents.
  (svn_client_ls3): backwards compat wrapper, calls svn_client_ls4 with the
   dirent_fields set to SVN_DIRENT_ALL.

* subversion/clients/cmdline/ls-cmd.c
  (svn_cl__ls): use svn_client_ls4, only asking for the dirent fields we
   actually need.

* subversion/libsvn_ra_svn/client.c
  (ra_svn_get_dir2): new version of get_dir, pass new dirent_fields
   parameter along to the server.
  (ra_svn_get_dir): pass through to new function, passing SVN_DIRENT_ALL
   for dirent_fields.
  (ra_svn_vtable): add ra_svn_get_dir2.

* subversion/libsvn_ra_dav/ra_dav.h
  (svn_ra_dav__get_dir2): new prototype.

* subversion/libsvn_ra_dav/session.c
  (dav_vtable): add svn_ra_dav__get_dir2.

* subversion/libsvn_ra_dav/fetch.c
  (svn_ra_dav__get_dir2): new version of get_dir, takes a dirent fields
   arg, dynamically figure out which props we need to ask for to fill in
   the dirent fields that were asked for. fall back to asking for all
   of them in the has_props case, since we currently have no other way
   to get that data. only fill in the parts of the dirent we were asked
   to get. create and destroy a new subpool, which is used to allocate
   the temporary neon properties array.
  (svn_ra_dav__get_dir): compat wrapper, passes SVN_DIRENT_ALL as the
   dirent_flags arg to svn_ra_dav__get_dir2.

* subversion/svnserve/serve.c
  (get_dir): accept an optional dirent fields parameter from the client,
   defaulting to SVN_DIRENT_ALL if it's not present. only initialize the
   parts of the dirents that the client asked for.
]]]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org