I’m hoping someone can help me with this problem I’ve been trying to solve for the past few days. I want to hide Magento’s Layered Navigation from the search engines entirely, but make it available to users. For SEO reasons, I don’t want to settle for NoFollowing all the links, or using noindex follow meta tags, or even blocking it entirely with Robots.txt. The most effective way of handling this would be only showing the layered Navigation to users with Cookies enabled, since Google doesn’t use cookies. The same effect could probably be achieved with JavaScript as well, but I’ve chosen the Cookie method.
So far I’ve managed to implement a crude piece of JS to check if cookies are enabled once the page has loaded (adapted from another thread on this forum). If cookies are enabled, it does nothing and layered nav displays, but if cookies are not enabled, I want to remove the “catalog.leftnav” block. I can’t for the life of me figure out how to do this from my JS script. All I’ve been able to achieve is removing the div element, or setting style.display to none etc., and while all of these techniques remove the links from the frontend, Google can still see them all. Here’s an example of the code I have so far in template/catalog/layer/filter.phtml
<div id="shop-by-filters">
<ol>
<?php foreach ($this->getItems() as $_item): ?>
<li>
<?php if ($_item->getCount() > 0): ?>
<a href="<?php echo $this->urlEscape($_item->getUrl()) ?>" rel="nofollow"><?php echo $_item->getLabel() ?></a>
<?php else: echo $_item->getLabel() ?>
<?php endif; ?>
<?php if ($this->shouldDisplayProductCount()): ?>
(<?php echo $_item->getCount() ?>)
<?php endif; ?>
</li>
<?php endforeach ?>
</ol>
</div>
<script type="text/javascript">
if (navigator.cookieEnabled) {
return true;
} else if (navigator.cookieEnabled === undefined) {
document.cookie = "testcookie";
if (cookie_present("testcookie"))
return true;
} else {
var elem = document.getElementById('shop-by-filters');
elem.parentNode.removeChild(elem);
}
</script>
Can anyone help me with this, or is there a better way of going about it? Please keep in mind that I am still trying to get my head around Magento, so I might need some instructions if the implementation is complicated.
Thank you.
Brendon
Just in case someone still finds this old question:
Hiding something from Google (or trying to do so) based on Session/Cookie/User-Agent is never a good idea. Google does not like to be cheated but wants to see the site just like any other visitor.
robots.txt, rel=nofollow etc. also do not fully fix this issue or have at least some downsides or limitations.
A better and more elegant solution for hiding layered navigation for crawlers and fix SEO issues caused by the huge number of layered navigation URLs would be using PRG Pattern.
This works like a charm, i. e. not changing the UX regarding Layered Navigation and 100% reliable in terms of preventing crawlers from wasting crawl budget on useless duplicate content URLs.
Simply said, it's about replacing the GET request to a layered navigation/filter URL with a POST request (which search engine crawlers do not follow) before redirecting the user to the original layered navigation/filter URL.
For further details and reading, please see
- Detailed explanation incl. sample request flow
- Why robots.txt, rel=nofollow etc. are no satisfying solutions here
- PRG Pattern Magento 2 Extension
- PRG Pattern Demo
I'm not sure if the Google robot will reliably parse your javascript.
You may be better off hiding the layered nav based on the current session with php.
<?php if (Mage::getSingleton('customer/session')): ?>
...your nav code...
<?php endif ?>
First of all, Javascript will do nothing to stop Google from indexing that content.
Why don't you want 'to settle for NoFollowing all the links'? That is exactly what NoFollow is for. You can also tell Google to not pay attention to the qualifiers/query strings in Webmaster Tools.
If for some reason you really wanted to hide that block from Google, edit the template and string compare $_SERVER['HTTP_USER_AGENT'] against Google's very public list of user agents here http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1061943
EDIT -- string compare
<?php if (stripos($_SERVER['HTTP_USER_AGENT'], 'Googlebot') !== false): ?>
<div id="shop-by-filters">
<ol>
<?php foreach ($this->getItems() as $_item): ?>
<li>
<?php if ($_item->getCount() > 0): ?>
<a href="<?php echo $this->urlEscape($_item->getUrl()) ?>" rel="nofollow"><?php echo $_item->getLabel() ?></a>
<?php else: echo $_item->getLabel() ?>
<?php endif; ?>
<?php if ($this->shouldDisplayProductCount()): ?>
(<?php echo $_item->getCount() ?>)
<?php endif; ?>
</li>
<?php endforeach ?>
</ol>
</div>
<?php endif; ?>
It is a slick subject. We used this code to hide the layered navigation from Google, but we are not sure is it working...
<div id="filters-no-follow"></div>
<?php
function prepare_for_echo($string) {
$no_br = trim(preg_replace('/\s+/', ' ', $string));
$no_slashes = str_replace('\'', '\\\'', $no_br);
return $no_slashes;
}
?>
<script>
function please_enable_cookies() {
var f = document.getElementById('filters-no-follow');
f.innerHTML = '<div class="no-cookies-error">Enable cookies to choose filters.</div>';
}
function please_load_filters() {
var f = document.getElementById('filters-no-follow');
f.innerHTML = '<?php if ( !empty($filtersHtml) || !empty($stateHtml) ): ?>'
+ '\n<div class="block block-layered-nav">'
+ '\n <div class="block-title">'
+ '\n <strong><span><?php echo prepare_for_echo($this->__('Shop By')); ?></span></strong>'
+ '\n </div>'
+ '\n <div class="block-content">'
+ '\n <?php echo prepare_for_echo($this->getStateHtml()); ?>'
+ '\n <?php if ($this->canShowOptions()): ?>'
+ '\n <p class="block-subtitle"><?php echo prepare_for_echo($this->__('Shopping Options')); ?></p>'
+ '\n <dl id="narrow-by-list">'
+ '\n <?php echo prepare_for_echo($filtersHtml); ?>'
+ '\n </dl>'
+ '\n <?php endif; ?>'
+ '\n </div>'
+ '\n</div>'
+ '\n<?php endif; ?>';
}
function are_cookies_enabled()
{
var cookieEnabled = (navigator.cookieEnabled) ? true : false;
if (typeof navigator.cookieEnabled == "undefined" && !cookieEnabled)
{
document.cookie="testcookie";
cookieEnabled = (document.cookie.indexOf("testcookie") != -1) ? true : false;
}
return (cookieEnabled);
}
if(are_cookies_enabled()) {
please_load_filters();
} else {
please_enable_cookies();
}
</script>
