Re: Compressed Pristines (Design Doc)

All, I'm happy to share[1] with you the design document for the Compressed Pristines feature. The document is public and anyone can comment on any part (select, right-click and comment away). If you'd like to get *editing* permission, please email me and I'll add you to the list of editors. I'm sure there will be much to criticize and debate, I'd love to hear all input, but being pragmatic, I also would like to a) experiment and figure out the best approach in practice, backed with real data and consensus and b) to finish this feature rather than debate forever (it's been debated for almost a decade this December!). As such, what's not clear, I've left out or written TBD notes and at the same time I've already made experimental changes locally to have a more learned information rather than an academic design (this, not to mention reading 100s of dev-list mails). I made a serious attempt at specifying as much of the hard facts/reqs/goals as possible to narrow the scope and avoid feature-creep. I'd like to take this feature on a "lightweight branch" and start committing code and getting reviews (and contributions!!) while we finalize the design and decide on the details (those who can create branches and grant commit rights please let me know when is the right time to do this - I'm ready and have code to commit and develop further).  I thank everyone who will help us get this finally done in advance and look forward to hearing from you all. -Ash [1] https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit >________________________________ > From: Hyrum K Wright <hyrum.wright@wandisco.com> >To: Ashod Nakashian <ashodnakashian@yahoo.com> >Cc: Philip Martin <philip.martin@wandisco.com>; Greg Stein <gstein@gmail.com>; "dev@subversion.apache.org" <dev@subversion.apache.org> >Sent: Monday, March 12, 2012 5:31 PM >Subject: Re: Compressed Pristines > >On Mon, Mar 12, 2012 at 7:11 AM, Ashod Nakashian ><ashodnakashian@yahoo.com> wrote: >> ----- Original Message ----- >> >>> From: Philip Martin <philip.martin@wandisco.com> >>> To: Ashod Nakashian <ashodnakashian@yahoo.com> >>> Cc: Greg Stein <gstein@gmail.com>; "dev@subversion.apache.org" <dev@subversion.apache.org> >>> Sent: Monday, March 12, 2012 2:40 PM >>> Subject: Re: Compressed Pristines >>> >>> Ashod Nakashian <ashodnakashian@yahoo.com> writes: >>> >>>>  * Are there any documentation/design/discussions on this feature that >>>>  I could study? >>> >>> There has been some discussion in the past on the dev list.  I don't >>> think it is written down anywhere else. >>> >>>>  * Who should coordinate and be contacted on decision points? >>> >>> The dev list. >>> >>>>  * I know this feature was planned for 1.8. Is that still reasonable? >>>>  (I can't find a release date for 1.8) Will 1.8 wait for this or has >>>>  this feature been demoted to a low-priority in general? The reason I >>>>  ask is to have a vague idea as to how close this feature is on the >>>>  critical path to future releases. >>> >>> We don't plan like that.  The features that will go into 1.8 are the >>> features people choose to write. >> >> Got it. Thanks. >> >> Here is my plan. I'll study whatever discussion took place on dev-list. My main technical concern is related to any complications that aren't obvious (due to design or requirements elsewhere in svn). I'll come up with an architectural overview for the feature design and a breakdown of major milestones. When ready, I'll share them on this list and open it for discussion. Based on the results, code changes can commence. > >A few observations based upon my past poking around this area. > >The current implementation of the pristine store is designed to be >"streamy."  That is, external users aren't supposed to know or care >where the actual contents live, or how they are accessed, but simply >get a stream from which they can read and write the contents.  In >principle, this should make compressing said contents relatively easy, >as we could just insert a compressing stream in this pipeline and >everything would automagically work.  Since this hasn't yet happened, >you can probably guess that it isn't as easy as that. :) > >The primary issue when I looked at this problem was that the streamy >abstraction is broken in several places, such as when we install the >new pristine file.  There are also certain consumers, such as a >external diff tools, that require an uncompressed on-disk file to >operate on, and we currently just provide the pristine as that file. >Compressed pristines would require recreating the uncompressed version >when such a tool is invoked.  Whether this is a useful tradeoff, I >don't know. > >Generally, though, I'm +1 for compressed pristines, as that was one of >the design goals of wc-ng in the first place.  (Oh, and extra points >for selectively compressing pristines based upon mime-type.) > >So start digging in, asking on the dev@ list and #svn-dev on Freenode, >and sending in patches.  You'll find a community of folks eager for >your input, and willing to help you. > >Hope that helps, >-Hyrum > > >-- > >uberSVN: Apache Subversion Made Easy >http://www.uberSVN.com/ > > >
Received on 2012-03-21 20:19:40 CET