{var%20f='http://v.t.sina.com.cn/share/share.php?appkey=1515056452',u=z||d.location,p=['&url=',e(u),'&title=',e(t||d.title),'&source=',e(r),'&sourceUrl=',e(l),'&content=',c||'gb2312','&pic=',e(p||'')].join('');function%20a(){if(!window.open([f,p].join(''),'mb',['toolbar=0,status=0,resizable=1,width=440,height=430,left=',(s.width-440)/2,',top=',(s.height-430)/2].join('')))u.href=[f,p].join('');};if(/Firefox/.test(navigator.userAgent))setTimeout(a,0);else%20a();})(screen,document,encodeURIComponent,'','','https://www.manongdao.com/data/attach/logo/logo.png', '推荐 唯我独甜 的问题《Stop Google from indexing》','https://www.manongdao.com/q-138689.html','页面编码gb2312|utf-8默认gb2312'));){kind=link}
Is there a way to stop Google from indexing a site?
标签：
google-index
相关问题
- Is There any way to change the snippet created by
- How to index dynamic pages to google using html5 p
- Double slash in URL path - bad practice?
- How do I tell search engines not to index content
- Google indexing of my AngularJS application
相关文章
- How to index dynamic pages to google using html5 p
- 我怎样才能让谷歌缓存中删除旧的网页，并开始索引新的(how can i make google ca
- 如何索引动态页面谷歌使用HTML5 pushState的方法？(How to index dynam
- Double slash in URL path - bad practice?
- How do I tell search engines not to index content
- Google indexing of my AngularJS application
- Stop Google from indexing
There are several way to stop crawlers including Google to stop crawling and indexing your website.
At server level through header
At root domain level through robots.txt file
At page level through robots meta tag
However, I must say if your website has outdated and not existing pages/urls then you should wait for sometime Google will automatically deindex those urls in next crawl - read https://support.google.com/webmasters/answer/1663419?hl=en
Bear in mind that microsoft's crawler for Bing, despite their claim to obey robots.txt, does not always do so.
Our server stats indicate that they have a number of IP's that run crawlers that do not obey robots.txt as well as a number of ones that do.
I gotta add my answer here, as accepted answer doesn't really touch the problem properly. Also remember that preventing Google from crawling doesn't mean you can keep your content private.
My answer is based on few sources: https://developers.google.com/webmasters/control-crawl-index/docs/getting_started https://sites.google.com/site/webmasterhelpforum/en/faq--crawling--indexing---ranking
robots.txtfile controls crawling, but not indexing! Those two are completely different actions, performed separately. Some pages may be crawled but not indexed, and some may even be indexed but never crawled. The link to non-crawled page may exist on other websites, which will make Google indexer to follow it, and try to index.Question is about indexing which is gathering data about the page so it may be available through search results. It can be blocked adding meta tag:
or adding HTTP header to response:
If the question is about crawling then of course you could create
robots.txtfile and put following lines:Crawling is an action performed to gather information about the structure of one specific website. E.g. you've added the site through Google Webmaster Tools. Crawler will take it on account, and visit your website, searching for
robots.txt. If it doesn't find any, then it will assume that it can crawl anything (it's very important to havesitemap.xmlfile as well, to help in this operation, and specify priorities and define change frequencies). If it finds the file, it will follow the rules. After successful crawling it will at some point run indexing for crawled pages, but you can't tell when...Important: this all means that your page can still be shown in Google search results regardless of
robots.txt.I hope at least some users will read this answer, and have it clear, as it's crucial to know what actually happens.
use a nofollow meta tag:
To specify nofollow at the link level, add the attribute rel with the value nofollow to the link:
You can disable this server wide by adding the below setting in globally in apache conf or the same parameters can be used in vhost for disabling it for particular vhost only.
Once this is done you can test it by verifying apache headers returned.
Also you can add the meta robots in this way:
And another extra layer is to modify .htaccess, but you need to check it deeply.