Where to put robots.txt file? [closed]

问题:

Where should put robots.txt?

domainname.com/robots.txt

or

domainname/public_html/robots.txt

I placed the file in domainname.com/robots.txt, but it's not opening when I type this in browser.

alt text http://shup.com/Shup/358900/11056202047-My-Desktop.png

回答1:

Where the file goes in your filesystem depends on what host you're using, so it's hard for us to give a specific answer about that.

The best description is: put it wherever the index.html (or index.php or whatever) file is that represents your homepage. If that's domainname/public_html/index.html, for example, put it in domainname/public_html/robots.txt.

回答2:

i think the better way to describe it is to have it in the root web folder of your domain... so http://example.com/robots.txt you can also put your sitemap.xml in the root or refer to it with a Sitemap: http://example.com/fldr/smap.xml line in your robots.txt.

dont forget: you can use Google Webmaster Tools to check to make sure you haven't restricted anything you didnt mean to(you also get to see queries and links woohoo!).

suggestion: id consider using the <META NAME="ROBOTS" CONTENT="INDEX, NOFOLLOW"> if possible because you will still earn linkjuice for links on the page but it wont show up in googles index while a robots.txt directive can leave a plain url with do description in SERPs but will loose all value of links pointed to it because its robots.txted out (its ranking b/c of anchor text so get credit for it)

回答3:

In the root of your web directory (where you put the files that show up on your website)

回答4:

In this case you should put it in domainname/public_html/robots.txt, as the public.html folder is where your index file will be.