On Mon, Nov 5, 2012 at 3:16 PM, Stefan Sperling <stsp_at_elego.de> wrote:
> On Mon, Nov 05, 2012 at 02:54:07PM +0100, Stefan Fuhrmann wrote:
> > On Sun, Nov 4, 2012 at 10:40 AM, Stefan Sperling <stsp_at_elego.de> wrote:
> > > I just came across something that reminded me of this thread.
> > > It seems PostgreSQL is doing something quite similar to what we
> > > want to do here:
> > >
> > > When the first PostgreSQL process attaches to the shared memory
> segment,
> > > it
> > > checks how many processes are attached. If the result is anything
> other
> > > than
> > > "one", it knows that there's another copy of PostgreSQL running which
> is
> > > pointed at the same data directory, and it bails out.
> > > http://rhaas.blogspot.nl/2012/06/absurd-shared-memory-limits.html
> > >
> >
> > IIUIC, the problems they are trying to solve are:
> >
> > * have only one process open / manage a given data base
> > * have SHM of arbitrary size
> >
> > Currently, we use named SHM to make the value of
> > two 64 bit numbers per repo visible to all processes.
> > Also, we don't have a master process that would
> > channel access to a given repository.
> >
> > The "corruption" issue is only about how to behave
> > if someone wrote random data to one of our repo
> > files. That's being addressed now (don't crash, have
> > a predictable behavior in most cases).
> >
> > > If this works for postgres I wonder why it wouldn't work for us.
> > > Is this something we cannot do because APR doesn't provide the
> > > necessary abstractions?
> > >
> >
> > The postgres code / approach may be helpful when
> > we try to move the whole membuffer cache into a
> > SHM segment.
>
> Ah, I see.
>
> Next question: Why don't we use a single SHM segment for the revprop cache?
>
> Revprop values are usually small so mapping a small amount of memory
> would suffice. And using a single SHM segment would make updated values
> immediately visible in all processes, wouldn't it? And we wouldn't need the
> generation number dance to make sure all processes see up-to-date values.
> Whichever process updates a revprop value would update the corresponding
> section of shared memory.
>
First of all, I want to point out that we now have a
working implementation for 1.8 and what we are
discussing here is probably targeted at future releases.
If we want revprop-only caches (to keep things simple),
we still need to handle the following basic trade-off:
Lifetime (effectiveness) ./. size. To be effective with
e.g. serf, the cache content should survive single
requests i.e. live longer than an fs_t. We also need
several MB (~200B/rev) per repo for decent hit rates.
OTOH, there may be hundreds of repositories on a
server and it is very hard to re-size the revprop cache
when the number of revs in a repo grows. It is thus
not quite feasible to keep fairly-sized per-repository
caches around indefinitely - even if they only contain
revprops.
That means that we should have one (or some small
number) shared cache for all repositories and let e.g.
some external process manage its lifetime etc. But
that is technically no different from having our membuffer
cache use shared memory instead of being process
local - which is a good thing.
The downside is that we need to address the following
3 issues when moving membuffer to SHM. From the
easiest to the hardest:
* make generations an integral feature of the cache
(e.g. by tagging index entries and bumping the
values upon "replace")
This is necessary to get rid of the revprop generations.
Race between revprop readers 1 and writer 2:
1: lookup revprop in cache -> miss
1: read revprop from disk -> "old content"
2: store new revprop on disk and in cache
1: store "old content" in cache
Note that after the 3rd step, the new content may or
may not be cached, i.e. we can't check for it in step 4.
* Have some SHM not bound to a repository or a
parent / child (fork) process relationship. Make it
work on most platforms.
* Portable, robust (lock owners may die), very low
overhead (~1musec) many-readers-one-writer locks
on the cache content. I have some ideas on how
to do that but this will be very hard to do correctly.
I'd like to see all that solved and SHM being used
for membuffer - which has been designed with that
goal in mind. It's the robustness part that makes it
so much harder to do than I thought back then.
-- Stefan^2.
--
Certified & Supported Apache Subversion Downloads:
*
http://www.wandisco.com/subversion/download
*
Received on 2012-11-06 11:48:28 CET