[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svn commit: r979193 - in /subversion/branches/performance/subversion: include/private/svn_cache.h libsvn_subr/cache-membuffer.c

From: Lieven Govaerts <svnlgo_at_mobsol.be>
Date: Sat, 28 Aug 2010 10:42:09 +0200

Hi Stefan,

building on Mac OS X 64-bits raises some warnings (and an error, but
that's in another revision).

Comments inline.

Lieven

On Mon, Jul 26, 2010 at 10:30 AM, <stefan2_at_apache.org> wrote:
> Author: stefan2
> Date: Mon Jul 26 08:30:08 2010
> New Revision: 979193
>
> URL: http://svn.apache.org/viewvc?rev=979193&view=rev
> Log:
> Provide a memcached-like implementation of svn_cache_t that does not have
> the same latency and reliability issues. Detailed descriptions can be found in
> the .c file.
>
[..]

> +/* Get the entry references for the given ENTRY.
> + */
> +static APR_INLINE apr_uint32_t
> +get_index(membuffer_cache_t *cache, entry_t *entry)
> +{
> + †return entry - (entry_t *)cache->directory;
> +}

In a 64-bit build this function triggers a "implicit conversion
shortens 64-bit value into a 32-bit value" warning.

[..]

> +/* If necessary, enlarge the insertion window until it is at least
> + * SIZE bytes long. SIZE must not exceed the data buffer size;
> + */
> +static void
> +ensure_data_insertable(membuffer_cache_t* cache, apr_size_t size)
> +{
> + †int average_hits;
> + †int threashold;
> + †entry_t *entry;
> +
> + †/* Make sure that this function actually terminates.
> + † */
> + †assert(cache->data_size >= size);
> +
> + †while (1)
> + † †{
> + † † †/* first offset behind the insertion window
> + † † † */
> + † † †apr_uint64_t end = cache->next == -1
> + † † † † † † † † † † † ? cache->data_size
> + † † † † † † † † † † † : get_entry(cache, cache->next)->offset;
> +
> + † † †/* leave function as soon as the insertion window is large enough
> + † † † */
> + † † †if (end - cache->current_data >= size)
> + † † † †return;
> +
> + † † †/* try to enlarge the insertion window
> + † † † */
> + † † †if (cache->next == -1)
> + † † † †{
> + † † † † †/* We reached the end of the data buffer; restart at the beginning.
> + † † † † † * Due to the randomized nature of our LFU implementation, very
> + † † † † † * large data items may require multiple passes. Therefore, SIZE
> + † † † † † * should be restricted to significantly less than data_size.
> + † † † † † */
> + † † † † †cache->current_data = 0;
> + † † † † †cache->next = cache->first;
> + † † † †}
> + † † †else
> + † † † †{
> + † † † † †/* Roll the dice and determine a threashold somewhere from 0 up

Typo.

> + † † † † † * to 2 times the average hit count.
> + † † † † † */
> + † † † † †average_hits = cache->hit_count / cache->used_entries;

Here too.

> + † † † † †if (average_hits < 1)
> + † † † † † †average_hits = 1;
> +
[..]
> +svn_error_t*
> +svn_cache__membuffer_cache_create(membuffer_cache_t **cache,
> + † † † † † † † † † † † † † † † † †apr_size_t total_size,
> + † † † † † † † † † † † † † † † † †apr_size_t directory_size,
> + † † † † † † † † † † † † † † † † †svn_boolean_t thread_safe,
> + † † † † † † † † † † † † † † † † †apr_pool_t *pool)
> +{
> + †membuffer_cache_t* c = apr_palloc(pool, sizeof(*c));
> + †int i, k;
> +
> + †/* We use this sub-pool to allocate the data buffer and the dictionary
> + † * so that we can release that memory easily upon OOM.
> + † */
> + †apr_pool_t *sub_pool = svn_pool_create(pool);
> +
> + †/* prevent pathological conditions: ensure a certain minimum cache size
> + † */
> + †if (total_size < 2 * sizeof(entry_group_t))
> + † †total_size = 2 * sizeof(entry_group_t);
> +
> + †/* adapt the dictionary size accordingly, if necessary:
> + † * It must hold at least one group and must not exceed the cache size.
> + † */
> + †if (directory_size > total_size - sizeof(entry_group_t))
> + † †directory_size = total_size - sizeof(entry_group_t);
> + †if (directory_size < sizeof(entry_group_t))
> + † †directory_size = sizeof(entry_group_t);
> +
> + †/* allocate buffers and initialize cache members
> + † */
> + †c->group_count = directory_size / sizeof (entry_group_t);

And here.

> + †c->directory = apr_palloc(sub_pool, c->group_count * sizeof(entry_group_t));
> + †c->first = -1;
> + †c->last = -1;
> + †c->next = -1;
> +
[..]

> +static svn_error_t*
> +membuffer_cache_set(membuffer_cache_t *cache,
> + † † † † † † † † † †const void *key,
> + † † † † † † † † † †apr_size_t key_len,
> + † † † † † † † † † †void *item,
> + † † † † † † † † † †svn_cache__serialize_func_t serializer,
> + † † † † † † † † † †apr_pool_t *pool)
> +{
> + †apr_uint32_t group_index;
> + †unsigned char to_find[KEY_SIZE];
> + †entry_t *entry;
> + †char *buffer;
> + †apr_size_t size;
> + †svn_error_t *err;
> +
> + †/* find the entry group that will hold the key.
> + † */
> + †group_index = get_group_index(cache, key, key_len, to_find, pool);
> + †if (group_index == -1)
> + † †return err;
> +
> + †/* Serialize data data.
> + † */
> + †SVN_ERR(serializer(&buffer, &size, item, pool));
> +
> + †/* The actual cache data access needs to sync'ed
> + † */
> + †SVN_ERR(lock_cache(cache));
> +
> + †if (cache->data_size / 4 > size)
> + † †{
> + † † †/* if necessary, enlarge the insertion window.
> + † † † */
> + † † †ensure_data_insertable(cache, size);
> +
> + † † †/* Remove old data for this key, if it that exists.
> + † † † * Get an unused entry for the key and and initialize it with
> + † † † * the serialized item's (future) posion within data buffer.
> + † † † */
> + † † †entry = find_entry(cache, group_index, to_find, TRUE);
> + † † †entry->size = size;

And here too.

> + † † †entry->offset = cache->current_data;
> +
> + † † †/* Copy the serialized item data into the cache.
> + † † † */
Received on 2010-08-28 10:43:27 CEST

This is an archived mail posted to the Subversion Dev mailing list.