[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: stopping webcrawlers using robots.txt

From: Evert | Rooftop <evert_at_rooftopsolutions.nl>
Date: 2006-07-10 00:07:15 CEST

I'm guessing you could still create an alias for robots.txt.. but im not
a 100% sure..

We simply use authentication everywhere.. I can't really understand why
you would want to open you repository for everyone, except robots..

Evert

Thomas Beale wrote:
> Bob Proulx wrote:
>>> ...
>>> <Location />
>>> DAV svn
>>> SVNParentPath /usr/local/var/svn
>>>
>>
>> Oh, I see. You have configured your subversion repository as the only
>> visible paths in your web server. In my opinion you have made a bad
>> choice of repository location. By putting it in the root directly you
>> have prevented your ability to do anything else with your web server.
>> At that point I think you are unable to do what you want.
> we chose that URL so as to be able to have http://svn.openEHR.org
> (i.e. our main URL is http://www.openEHR.org). Does no-one else do
> this? It seems an obvious thing to do....
>> So to
>> answer your question, in your configuration you can't. I suppose you
>> could check in robots.txt into the top level of your repository. But
>> then it would be part of your project and so forth.
>>
> no, I think that would just be plain confusing, and I don't think it
> would work anyway....but surely there is a way to tell apache to deal
> in a certain way with a request straight off '/' , even with this
> configuration?
>
> I don't think changing the URL is an option - it is published all over
> the place - everyone in our community knows it. I know we could put
> some kind of redirect in, but that seems a pretty poor option as
> well....I am quite surprised to find that the configuration which
> seems to be preferred by the subversion manual is not compatible with
> managing web robots...
>
> - thomas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Mon Jul 10 13:34:50 2006

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.