On Wed, Mar 21, 2012 at 2:19 PM, Ashod Nakashian
<ashodnakashian_at_yahoo.com> wrote:
> All,
>
> I'm happy to share[1] with you the design document for the Compressed
> Pristines feature. The document is public and anyone can comment on any part
> (select, right-click and comment away). If you'd like to get *editing*
> permission, please email me and I'll add you to the list of editors.
>
> I'm sure there will be much to criticize and debate, I'd love to hear all
> input, but being pragmatic, I also would like to a) experiment and figure
> out the best approach in practice, backed with real data and consensus and
> b) to finish this feature rather than debate forever (it's been debated for
> almost a decade this December!).
>
> As such, what's not clear, I've left out or written TBD notes and at the
> same time I've already made experimental changes locally to have a more
> learned information rather than an academic design (this, not to mention
> reading 100s of dev-list mails). I made a serious attempt at specifying as
> much of the hard facts/reqs/goals as possible to narrow the scope and avoid
> feature-creep.
>
> I'd like to take this feature on a "lightweight branch" and start committing
> code and getting reviews (and contributions!!) while we finalize the design
> and decide on the details (those who can create branches and grant commit
> rights please let me know when is the right time to do this - I'm ready and
> have code to commit and develop further).
>
> I thank everyone who will help us get this finally done in advance and look
> forward to hearing from you all.
> -Ash
>
> [1] https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit
So, I've read through the design document, and the various threads,
and have a couple of comments / questions which I don't think have
been addressed. My first impression, though is to give you major
kudos for going through the effort to research and think about this
complex and subtle problem. Now my thoughts...
As mentioned elsewhere, I too was surprised by the choice of a custom
container, though I think you make a good argument for it. One
simplification I was thinking about is this: what if the container
only needed to support add and batch-delete operations? These are the
current contraints of the existing pristine store; would they
introduce additional simplicity into your design?
In some respects, it looks like you're solving *two* problems:
compression and the internal fragmentation due to large FS block
sizes. How orthogonal are the problems? Could they be solved
independently of each other in some way? I know that compression
exposes the internal fragmentation issue, but used alone it certainly
doesn't make things *worse* does it?
Finally, in all the above let's not let "the perfect be the enemy of
the good." If something *simple* will give us demonstrable
performance improvements now, can we do so without limiting out
ability to do a more complex and complete solution later?
Anyway, good work, and here's hoping it yield fruit.
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Received on 2012-03-23 19:55:00 CET