Fwd: Subversion Compressed Pristines Design

Again, pulling commentary up out of the HTML:

* I'm unclear on what you mean by "no limitations", as your design doc
clearly states 16MB file size limits, and 64k of those files. Thus, 1TB of
content. Now... elsewhere, you also state that a single (pack) file may
exceed the normal limit, for purposes of keeping an entire pristine within
a single file. These two design points are a bit contradictory.

* In any case... 1TB is certainly not large enough for the pristine content
store. I've personally witnessed 10GB working copies. I will bet others on
this list have seen larger. 10GB is just two orders of magnitude less than
1TB. We need better future proofing. I'd like to see a pristine store that
can accommodate 1PB, minimum. The current store has no limit, so if one is
to be applied, then it better be very generous.

Cheers,
-g

---------- Forwarded message ----------
From: ashodnakashian (Google Docs) <
d+MTAyNzIyNDgyMjQ4NDY3NTc1OTA0-MTEyNTUyMDI1ODMxNTcwMjc4OTI1_at_docs.google.com>
Date: Thu, Mar 22, 2012 at 00:03
Subject: Subversion Compressed Pristines Design
To: gstein_at_gmail.com

ashodnakashian added comments to Subversion Compressed Pristines
Design<https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit?disco=AAAAAEf_oP4>
[image: Greg Stein]
*Greg Stein*
activity

This section reminds me of "generational scavenging". Also similar to
LevelDB's multi-level storage system.
[image: ashodnakashian]
*ashodnakashian*

Sounds like something you like - always good thing :)

Yes, that is very similar to what I had in mind. In fact, I event imagined
that by adding file revision numbers in the index file, we can store
historic revisions (at least some of them) and give the user the ability to
accumulate a number of historic revisions for faster diff/restore with them
at the expense of storage. But that is out of scope, so I left out
speculating about it.
Reply<mailTo:Reply+%3Cd+AORGpRfVaERvJu3_BahprsnE-7LYbYl014EB-M3fz7C4zPy2YwYhcBIsOFPa79OoMyIDL5T_r6Ljyeg.google.com%3E?subject=Subversion%20Compressed%20Pristines%20Design>
Mute<https://docs.google.com/document/docos/mute?id=1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc&title=Subversion+Compressed+Pristines+Design&di=AAAAAEf_oP8&t=DISCUSSION>Open
»<https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit?disco=AAAAAEf_oP8>
[image: Greg Stein]
*Greg Stein*
1TB

That isn't large enough. Some people have REALLY huge working copies. Think
"video".

(yes, I'd agree that it's likely nobody has one this large, but I wouldn't
be surprised at all by working copies in the multi-100GB range)

We need to design well ahead of the curve here.
[image: ashodnakashian]
*ashodnakashian*

There are no limitations inherent anywhere. I've made a very conscious
effort to be forward compatible and resilient to arbitrary limitations.
However you can't make everything 128-bit "just in case".

So my initial "sane" goal is 1TB (I found this to be a "reasonable" minimum
to support.) Having said that, The number of pack store files and the limit
on each are both subject to tuning based on hard numbers. So I'd like to do
benchmarks and we should find out the right balance *for the target minimum
that we agree on*.
Reply<mailTo:Reply+%3Cd+AORGpRfLPQNs2rOQ1m8oY3YU-Jp5Le-DIjWvcn45sVTt_JV3uQETeCCBA1L1uCi7cHYyX8uwN0hssJ0.google.com%3E?subject=Subversion%20Compressed%20Pristines%20Design>
Mute<https://docs.google.com/document/docos/mute?id=1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc&title=Subversion+Compressed+Pristines+Design&di=AAAAAEf_oPk&t=DISCUSSION>Open
»<https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit?disco=AAAAAEf_oPk>
[image: Greg Stein]
*Greg Stein*
access

That is not a design goal, so it is not a limitation. We do not want any
external tools/processes monkeying around with files under .svn
[image: ashodnakashian]
*ashodnakashian*

Fair enough.
Reply<mailTo:Reply+%3Cd+AORGpRdnjc3sF0YP0NWT9iqMjj-ZvlTaH5dmE8jzB1ljSifz40Z8dbwRzEV6ZSwGJqaf7Ib4Sw9aLWY.google.com%3E?subject=Subversion%20Compressed%20Pristines%20Design>
Mute<https://docs.google.com/document/docos/mute?id=1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc&title=Subversion+Compressed+Pristines+Design&di=AAAAAEf_oP4&t=DISCUSSION>Open
»<https://docs.google.com/document/d/1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc/edit?disco=AAAAAEf_oP4>
You received this email because you are a participant in the updated
comment threads.Change<https://docs.google.com/document/docos/notify?id=1ktIsewfMBMVBxbn-Ng8NwkNwAS_QJ6eC7GOygsbBeEc&title=Subversion+Compressed+Pristines+Design>what
Google Docs sends you.You
can not reply to this email.