There are search engines for Tor hidden services, analogous to Google, Yahoo, etc. for the greater Internet. How can sites avoid being indexed by these search engines? Does a conventional "robots.txt" work?

1 Answer 1


robots.txt only works if the search engine provider listens to it, which they are not guaranteed to do.

You will only be indexed if the spider that the search engine is using can not find you, and the only way to have them not find you is to never have a link to your website posted to another website. (which is impossible to guarantee too)

You can try robots.txt and you can try not having any links anywhere like the hidden wiki. If that does not work, contact the search engine and ask them to take you off

  • 2
    I asked the developers of the Ahmia search engine and they agreed that they respect the robots.txt for hidden services.
    – Jens Kubieziel
    Commented May 21, 2014 at 16:48

You must log in to answer this question.