[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

RE: EXTERNAL: Who else is using SVN for large-binary-asset storage?

From: Marlow, Andrew <Andrew.Marlow_at_fisglobal.com>
Date: Fri, 24 Apr 2020 17:19:53 +0000

Hello everyone,

The answer is "yes". I have come across investment banks that store boost releases in subversion. Sometimes it's a full boost release and that can be quite large. They take a long time to checkout but at the time it seemed better than the alternatives available at the time, e.g. placing the artifacts on an SFTP site or Windows share (ugh).

-----Original Message-----
From: Karl Fogel <kfogel_at_opentechstrategies.com>
Sent: 24 April 2020 17:40
To: Subversion Users <users_at_subversion.apache.org>
Subject: EXTERNAL: Who else is using SVN for large-binary-asset storage?

Are there other companies out there using SVN for large-binary-blob storage?

I'm wondering if it might be possible to put together a mini-consortium of companies to fund the completion of Issue #525:

  https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FSVN-525&amp;data=02%7C01%7CAndrew.Marlow%40fisglobal.com%7C6729a1c6e8434e58ae4b08d7e86e7f0d%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637233433584704974&amp;sdata=WDAoL3RF5rOb0nx7Ig1wFIZkrisKSw%2BgKC1nGZ2GaPM%3D&amp;reserved=0
  "allow working copies without text-base/"

Our company keeps medium-large (say, ~5GB) binary blobs under version control in a dedicated Subversion repository, and it works quite well. Subversion can handle files of that size just fine, and it enables us to do path-based authorization (yay) and partial-tree checkouts. [1]

But the presence of text-base files in .svn/pristine/ is a real downer :-). The text-base files are mostly pointless in this case, and they double the client-side disk space usage.

There is no useful diff between two revisions of these binary blobs: there's no human-readable diff *and* there's rarely any machine-useable diff either (e.g., for reducing network time when receiving an update or committing a new revision). So the only benefit from the text-base files is to make 'svn revert' faster. We'd be happy to have 'svn revert' re-fetch the file from the repository if it meant we could reduce our storage cost on the client side by half.

(Plus you'd only lose local-revert capability on files where you know you don't need it, since presumably the no-text-base behavior would be optional per file and controlled via an 'svn:no-pristine' property or something like that.)

Is anyone else in a similar situation? If we join forces, we could probably fund one or more Subversion developers to finally get Issue #525 done. I'd be happy to do the organizing (I'm still reasonably familiar with Subversion development and who does what, though I haven't been an active developer in the project in many years).

Please CC me on any replies, as I'm not subscribed to users@.

Best regards,
-Karl

[1] We investigated using Git too, but, though Git good for many things, it is not well-suited for this particular job. The Git Large-File Storage extension (https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit-lfs.github.com%2F&amp;data=02%7C01%7CAndrew.Marlow%40fisglobal.com%7C6729a1c6e8434e58ae4b08d7e86e7f0d%7Ce3ff91d834c84b15a0b418910a6ac575%7C0%7C0%7C637233433584714968&amp;sdata=XLfHamWk4don%2F1w77lC8zcC85ORUS%2FCnkuB8ft53XGs%3D&amp;reserved=0) doesn't address most of our needs either; it's solving a different problem, I guess.
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. FIS is a trading name of the following companies: Advanced Portfolio Technologies Ltd (No: 6312142) | Clear2Pay Limited (No: 5792457) | Decalog (UK) Limited (No: 2567370) | FIS Apex (International) Limited (No: 2999960) | FIS Apex (UK) Limited (No. 3573008) | FIS Consulting Services (UK) Limited (No: 2486794) | FIS Derivatives Utility Services (UK) Limited (No: 9398140) | FIS Energy Solutions Limited (No: 1889028) | FIS Global Execution Services Limited (No. 3127109) | FIS Global Trading (UK) Limited (No: 2523114) | FIS Investment Systems (UK) Limited (No: 1366010) | FIS Sherwood Systems Group
 Limited (No: 982833) | FIS Systems Limited (No: 1937159) | FIS Treasury Systems (Europe) Limited (No: 2624209) | FIS Treasury Systems (UK) Limited (No: 2893376) | GL Settle Limited (No: 2396127) | Integrity Treasury Solutions Europe Limited (No: 3289271) | Monis Software Limited (No: 2333925) | Reech Capital Limited (No: 3649490) | Solutions Plus Consulting Services Limited (No: 3839487) | Valuelink Information Services Limited (No: 3827424) all registered in England & Wales with their registered office at 25 Canada Square, London E14 5LQ | FIS Global Execution Services Limited is authorised and regulated by the Financial Conduct Authority | Certegy Card Services Limited (No: 3517639) | Certegy France Limited (No: 2557650) | eFunds International Limited (No: 1930117) | Fidelity Information Services Limited (No: 2225203) | FIS Payments (UK) Limited (No: 4215488) | Metavante Technologies Limited (No: 2659326) all registered in England & Wales with their registered office at 1st Floor Tricorn House, 51-53 Hagl
ey Road, Edgbaston, Birmingham, West Midlands, B16 8TU, United Kingdom | FIS Payments (UK) Limited is authorised and regulated by the Financial Conduct Authority; some services are covered by the Financial Ombudsman Service (in the UK). Clear2Pay Limited, Registered in Scotland (No SC157659), Registered Office: Clear2Pay House, Pitreavie Court, Pitreavie Business Park Queensferry Rd, Dunfermline, Fife, SS, KY11 8UU, Scotland | FIS eProcess Intelligence LLC (UK Branch), UK Establishment Registered in England & Wales (No: FC16527/Branch No. BR000355), Registered Branch Office: 25 Canada Square, London, E14 5LQ; FIS eProcess Intelligence LLC is a limited liability company formed in the USA registered on file with the Office of the Delaware Secretary of State, Division of Corporations (File No. 2032143), Head Office: 601 Riverside Avenue, Jacksonville Florida, FL32204, USA | FIS Investment Systems LLC, UK Establishment Registered in England & Wales (No: FC033836/Branch No. BR018923), Registered Branch Office: 25
 Canada Square, London, E14 5LQ; FIS Investment Systems LLC is a limited liability company formed in the USA registered on file with the Office of the Delaware Secretary of State, Division of Corporations (File No. 0881255), Head Office: 377 E. Butterfield Road, Suite 800, Lombard, IL 60148, USA | Calls to and from the companies may be recorded for quality purposes. | All of the named companies are part of FIS (Fidelity National Information Services, Inc.).
Received on 2020-04-24 19:20:07 CEST

This is an archived mail posted to the Subversion Users mailing list.