svnsync fails to replicate large revision

Hello,

I'm not sure if this is a "bug", a "configuration issue", or simply a network issue. I'm hoping it is just a configuration issue.

I'm creating an off-site replica of our SVN server. Our current server is running the latest CollabNet SVNEdge server. Subversion version 1.8.19-4419.51

There was a large commit that fails after an indeterminate amount of time - sometimes 45 minutes, sometime 4.5 hours. I've been tinkering with Timout and KeepAlive settings to no avail.

The error I continue to see when the failure happens is:
[Fri Jul 13 11:22:33.773429 2018] [dav:error] [pid 1906] [client XX:38654] Provider encountered an error while streaming a REPORT response. [500, #0]
[Fri Jul 13 11:22:33.773445 2018] [dav:error] [pid 1906] [client XX:38654] Problem replaying revision [500, #104]
[Fri Jul 13 11:22:33.773448 2018] [dav:error] [pid 1906] [client XX:38654] Error writing base64 data: Connection reset by peer [500, #104]

The initial requests are seen here:

[13/Jul/2018:09:45:46 -0700] gregc IA rev-proplist r18116 9

With the failure showing up as such (note the timestamp of the two messages):

[13/Jul/2018:11:20:01 -0700] jenkins IA get-dir / /trunk r44281 props 0

[13/Jul/2018:09:45:54 -0700] gregc IA replay / r18116 5799

Relevant settings:
Bulk updates allowed. I've tried preferring but that didn't seem to fix the issue
Timeout: I've set it between 12,000 and 120,000. Issue not fixed
KeepAlive: Yes (always)
MaxKeelAliveRequests: I've set it between 10,000 and 100,000
KeepAliveTimeout: I've set it between 15 and 900
Compression set to 5
Server cache: 64 MB, enabled for Full Texts, Deltas, and RevProps

I have had to remove "mod_deflate" from loading - I believe this was due to crashes when transferring files > 4GB.

The server seems to be syncing at about 500KBps (4Mbps). The file "db/revs/18/18116" is 441 MB on disk: FSFS V6, Repo format 5, Not packed.

This is supposedly the message the client is exiting with (garbled with the output of "time"?):
57:43.26
.svnsync: E175002: REPORT request on '/svn/IA/!svn/rev/18116' failed
Command exited with non-zero status 1
1158.67user 425.49system 57:43.26elapsed 45%CPU (0avgtext+0avgdata 35588maxresident)k
216336inputs+1877472outputs (19major+808715minor)pagefaults 0swaps

That is all the relevant information I can think of in this instance. Seemingly the client is terminating the "svnsync synchronize" request. Or is this something in the server settings that is causing this issue? Or could this be something else?

Thank you very much!

[143421_logo_final_25]
Colin Foster
Software Engineer
Tel 262-409-9898
colin.foster_at_in-advantage.com<mailto:colin.foster_at_in-advantage.com>
www.in-advantage.com