[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: stopping webcrawlers using robots.txt

From: Thomas Beale <thomas_at_deepthought.com.au>
Date: 2006-07-12 15:39:00 CEST

Our system administrator messed around with this, and produced the
following solution, which works. The key was to change Location
directives to Directory directives, i.e. to match directory patterns not
URL patterns.

<VirtualHost 1.2.3.4>
         ServerAdmin webmaster@xxxx.org

         ServerName svn.xxxx.org

         DocumentRoot /usr/local/var/svn

         RewriteEngine On
         RewriteRule .*robots\.txt$ /generic-root/robots.txt [PT]

         Alias /generic-root /usr/local/var/generic-root

         <Directory /usr/local/var/generic-root>
                 SetHandler default-handler
                 allow from all
         </Directory>

         <Directory /usr/local/var/svn>
                 DAV svn
                 SVNParentPath /usr/local/var/svn

                 # authorisation
                 AuthzSVNAccessFile /etc/subversion/access-control

                 # authentication
                 AuthType Basic
                 AuthName "development Subversion Repository"
                 AuthUserFile /etc/subversion/authentication

                 # anonymous access rules
                 Satisfy Any
                 Require valid-user
         </Directory>
</VirtualHost>

Thomas Beale wrote:
>
> Hi,
>
> I have looked around but not found the answer to the question: how to
> make /robots.txt visible in an apache virtual host config for a
> subversion server. How would I tell Apache to allow requests to read
> /robots.txt given the following configuration? (Or - how can I just
> block robots going into the SVN repositories)?
>
> <VirtualHost 1.2.3.4>
> ServerAdmin webmaster@xxxx.org
>
> ServerName svn.xxxx.org
>
> <Location />
> DAV svn
> SVNParentPath /usr/local/var/svn
>
> # authorisation
> AuthzSVNAccessFile /etc/subversion/access-control
>
> # authentication
> AuthType Basic
> AuthName "development Subversion Repository"
> AuthUserFile /etc/subversion/authentication
>
> # anonymous access rules
> Satisfy Any
> Require valid-user
> </Location>
> </VirtualHost>
>
>
>
> thanks,
>
> - thomas beale

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Wed Jul 12 15:46:04 2006

This is an archived mail posted to the Subversion Users mailing list.