AW: Compressed Pristines (Design Doc)

Hi,

I just want to shed light on three arguments against a new custom archive format.

Compressing the files using a standard format (like gz or xz) file-by-file has the advantage of better debuggability. Developers can easily (de)compress or otherwise those files using their standard utilities when trying to debug problems. Using a custom format always makes that process more difficult.

In addition, increasingly more file systems support features like block_suballocation, tail packing or tail merging[1]. This drastically reduces the space loss due to files being smaller than the block size.

And the third argument is the simplicity of implementation. Just checking .svn/pristines/ab/abcd.gz in addition to .svn/pristines/ab/abcd when searching for a pristine file is much easier to implement.

I'm not opposed in general to storing pristines in an archive, but the disadvantages should be weighted in when making the decision.

A different, somehow related idea is a common pristine store somewhere in the users directory, shared by several working copies. Especially when checking out several working copies of the same project (or similar branches), this could save a lot of network traffic.

Best regards

Markus Schaber
[1]: http://msdn.microsoft.com/en-us/library/windows/desktop/ee681827%28v=vs.85%29.aspx claims tail packing support for NTFS. http://en.wikipedia.org/wiki/Block_suballocation claims support for Btrfs, ReiserFS, Reiser4, FreeBSD UFS2. And AFAIR, XFS has a similar feature.

-- 
___________________________
We software Automation.
3S-Smart Software Solutions GmbH
Markus Schaber | Developer
Memminger Str. 151 | 87439 Kempten | Germany | Tel. +49-831-54031-0 | Fax +49-831-54031-50
Email: m.schaber@3s-software.com | Web: http://www.3s-software.com 
CoDeSys internet forum: http://forum.3s-software.com
Download CoDeSys sample projects: http://www.3s-software.com/index.shtml?sample_projects
Managing Directors: Dipl.Inf. Dieter Hess, Dipl.Inf. Manfred Werner | Trade register: Kempten HRB 6186 | Tax ID No.: DE 167014915 
-----Ursprüngliche Nachricht-----
Von: Mark Phippard [mailto:markphip_at_gmail.com] 
Gesendet: Donnerstag, 22. März 2012 16:23
An: Ashod Nakashian
Cc: Daniel Shahaf; dev_at_subversion.apache.org
Betreff: Re: Compressed Pristines (Design Doc)
On Thu, Mar 22, 2012 at 11:18 AM, Ashod Nakashian <ashodnakashian_at_yahoo.com> wrote:
>>Design-wise I'm a bit surprised that the choice ended up being rolling 
>>a custom file format.
>
> Personally I know not of any library that can deliver the requirements 
> that we need (outlined in the doc). Again, if the requirements are in 
> question, let's simplify them. If there is such a library, suggesting 
> it will save us a lot of time and effort. Otherwise, using a Tar-like container will just not cut it. On the other hand, the proposed custom format is rather simple and its code shouldn't be complex. In fact, I suspect Tar is more complex (considering it must store more information than we do).
I am not sure what Daniel meant, but I had always just assumed we would simply compress the files in the existing pristines.  I think your document does a nice job explaining why that is not good enough.
In that sense, I would also say that I was surprised by the choice of a custom file format, but that does not mean I would question it.  I think your document does a nice job in revealing some of the subtle complexities of this feature.  That gives me more hope on progress towards a solution.
--
Thanks
Mark Phippard
http://markphip.blogspot.com/