[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Who else is using SVN for large-binary-asset storage?

From: Karl Fogel <kfogel_at_opentechstrategies.com>
Date: Fri, 24 Apr 2020 13:43:43 -0500

On 24 Apr 2020, Mark Phippard wrote:
>I think this would be a good idea in that it might be one of the last
>remaining niches where SVN is a better tool for the job than a DVCS.
>I do not think I could contribute though.
>
>I just wanted to throw another item on the pile. I recall an old
>thread (have not been able to find it) where it was shown that a
>massive performance win on large binary blobs would be if we could
>skip all of the xdelta stuff and just stream the binary.  If I recall
>correctly, you can even see and demo this today using WebDAV and just
>doing a PUT or whatever is right request with the entire file.  The
>server already knows how to handle it and store the file the same as
>it would if it had come via a SVN client.  I think there were some
>complications with how svndiff0/svndiff1 etc are expected by a
>client, but if there were some way to have a property on a file that
>caused us to skip all of this, including storing the extra pristine
>copy, it could be a big win for managing large binaries with SVN.
>
>It seems like we could make revert fetch the file from the server
>again to restore a binary.
>
>If I can find any of those old threads I will share them.  So far the
>only one I found was about how using a larger xdelta window size
>could give better compression, but the thread I recall was about not
>doing it at all.  It also assume that the xdelta is of no real value
>because it does not shrink the amount of bytes that have to be
>transferred.

Ah, thanks for this reminder! I also recall those results (and I guess they're not surprising). I'll make sure we keep it in mind if this project happens. If you happen to dig up any of the old threads, that'd be great, but even if you don't, the above information is enough for a developer to know the possibility exists.

Best regards,
-Karl
Received on 2020-04-24 21:15:35 CEST

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.