Re: svn commit: r30173 - in branches/in-memory-cache/subversion: include libsvn_subr

On Wed, Apr 2, 2008 at 2:20 PM, Blair Zajac <blair_at_orcaware.com> wrote:
> glasser_at_tigris.org wrote:
>
> > Author: glasser
> > Date: Tue Apr 1 16:24:34 2008
> > New Revision: 30173
> >
> > Log:
> > On the in-memory-cache branch:
> >
> > Add memcached-base caches. (Not currently used anywhere.) Hardcode
> > in use of a single server (localhost:11211).
> >
> > This is using the apr_memcache code that currently lives on apr-util
> > trunk and will be eventually released in apr-util 1.3. It was
> > imported from a separate apr_memcache package; we should be able to
> > use either version, but I haven't done the configuration for that yet
> > (Dan Christian sent me a patch to do that, though).
> >
>
> For memcached I suggest exposing the memcached flags value.
>
> Uses for the key:
>
> 1) It's a very useful tool for not having to encode additional information
> in the key, which makes key manipulation faster.
>
> 2) Hash some unique value into N bits of the flags. Each
> serializer/deserializer pair will pick its own unique value. When you
> version up a structure that you are caching, say add an additional field,
> you bump the value and even if the memcached server is not bounced, you can
> ignore its results since the returned flags doesn't match.
>
> 3) Use some bits of the flags to indicate compression. You can not
> compress some short keys and compress longer ones.
>
> So I suggest modeling this API to be similar to the *gasp* new Java API
> which is a great piece of work:
>
> http://bleu.west.spy.net/~dustin/projects/memcached/apidocs/
>
> Have a new type, say cached_data that contains a
>
> struct svn_cached_data
> {
> apr_size_t flags;
> const char *data;
> apr_size_t data_len;
> }
>
> So something like
>
>
> typedef svn_error_t *(svn_cache_deserialize_func_t)(void **out,
> svn_cached_data *data,
> apr_pool_t *pool);
>
> typedef svn_error_t *(svn_cache_serialize_func_t)(svn_cached_data **data,
> void *in,
> apr_pool_t *pool);
>
> I think you'll find not having the flags available will be a drawback
> sometime in the future, so I strongly suggest putting it now.
>
> I realize that an in-process cache won't need a flags value, but better to
> have the API expose the flags for memcached and ignore it for in-process
> cache then not have it.
>
> BTW, I'm doing the same thing in my Subversion server, but at a higher
> level. We're caching in memcached a (repos-uuid, rev, path) with a (node-id)
> value and then doing lookups (repos-uuid, node-id) for the real data.

All current uses of svn_cache_t are already serializing/deserializing
complex data structures into strings, so that already gives you room
to add flags. I'd rather keep the flags for the use of the
svn_cache-memcached implementation itself (eg, to specify that the
value has been compressed).

--dave

-- 
David Glasser | glasser@davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org