
You can instruct Google not to include content from your site or remove content that is currently indexed in their search engine.
For instance to prevent robots from crawling your site, place a robots.txt file in your server root:
Open a text editor such as wordpad and copy and paste the following text:
User-agent: *
Disallow: /
Save the file as robots.txt and place it in your server root as such:
(http://yourserver.com/robots.txt)
- To block a directory and everything in it, follow the directory name with a forward slash.
Disallow: /junk-directory/
- To block a page, list the page.
Disallow: /private_file.html
- To remove a specific image from Google image search, add the following:
User-agent: Googlebot-Image Disallow: /images/dogs.jpg
- To remove all images on your site from Google image search:
User-agent: Googlebot-Image Disallow: /
- To block files of a specific file type (for example, .gif), use the following:
User-agent: Googlebot Disallow: /*.gif$
So you can therefore specify which part of your site you need preventing from indexing, To find out more about content indexation go to google Webmaster Guidelines.