How can I stop Google and others indexing me?

I was asked this question today. Usually it’s the other way, how can I get to the top. But in some cases, clients need to stop the Googlebot crawling certain files and directories.

The easiest way to do this is to get your web designer to create a robots.txt file. The full specification for robots.txt is here. So, to stop Googlebot and the other search engines entering a directory called ‘/private’ you’d add the following line:

Disallow: /private/