[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [Sidebar] - Interesting discussion on impact of "simple" HTTP requests

From: Mark Phippard <markphip_at_gmail.com>
Date: Tue, 5 Jun 2012 11:28:52 -0400

On Tue, Jun 5, 2012 at 11:25 AM, Greg Stein <gstein_at_gmail.com> wrote:
> On Tue, Jun 5, 2012 at 10:49 AM, Mark Phippard <markphip_at_gmail.com> wrote:
>>...
>> The net effect is just the impact that a seemingly small HTTP request
>> can have on a highly trafficked site where those requests start to add
>> up.  The main problem here is that the Eclipse update process looks
>> for a small index file that is currently always a HTTP 404.  This
>
> Heh. Reminds me of the traffic spike that occurred when a Google Code
> project used one of its version control for a configuration file. It
> was a proxy add-on for Firefox to navigate the Chinese firewall.
> Suddenly, every user who had the add-on installed was pinging
> code.google.com for this one file, retrieved via a Subversion URL.
> Daniel Berlin moved fast and introduced a cache that dropped our
> server load.
>
>> results in 1.5GB of network packets per day which basically amounts to
>> 152 minutes per day of their available bandwidth.  The 404 itself is
>> not the problem, the bandwidth usage would be worse if the file
>> existed.  The issue is more the design of the protocol and the
>> discovery it does each time it runs.  This is probably similar to all
>> of the HEAD and PROPFIND requests we do in our protocol.  They are
>> seeing similar bandwidth issues resulting from the usage of HEAD in
>> their protocol.  It looks like the webmaster would like them to
>> explore doing more with a single request.  Someone that understands
>> how our REPORT requests work might want to throw some commentary into:
>
> See above: cache.
>
> Not "lump things into one response".
>
> A reverse proxy (or several) in front of the primary (Subversion)
> server can 404 that request before introducing any load on the origin
> server.

Keep in mind that this is not about server load, they are looking at
this from bandwidth. So a cache in front of the server would not help
them at all. Eclipse.org has a 10MB Internet link that is almost
always saturated.

Also, if it was not clear, Subversion is not involved here. This is a
plain Apache server.

-- 
Thanks
Mark Phippard
http://markphip.blogspot.com/
Received on 2012-06-05 17:29:24 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.