googlee

You can instruct Google  not to include content from your site or remove content that is currently  indexed in their search engine.

For instance to prevent robots from crawling your site, place a robots.txt file in your server root:

Open a text editor such as wordpad and copy and paste the following text:

User-agent: *

Disallow: /

Save the file as robots.txt and place it in your server root as such:

(http://yourserver.com/robots.txt)

  • To block a directory and everything in it, follow the directory name with a forward slash.
    Disallow: /junk-directory/
  • To block a page, list the page.
    Disallow: /private_file.html
  • To remove a specific image from Google image search, add the following:
    User-agent: Googlebot-Image
    Disallow: /images/dogs.jpg
  • To remove all images on your site from Google image search:
    User-agent: Googlebot-Image
    Disallow: /
  • To block files of a specific file type (for example, .gif), use the following:
    User-agent: Googlebot
    Disallow: /*.gif$

So you can therefore specify which part of your site you need preventing from indexing, To find out  more about content indexation go to  google Webmaster Guidelines.


This post is tagged , ,

Leave a Reply





Configure Widgets

Categories

Configure Widgets