Does Google crawl AJAX content? [closed]

问题:

On the home page of my site I use JQuery's ajax function to pull down a list of recent activity of users.

The recent activity is displayed on the page, and each line of the recent activity includes a link to the user profile of the user who did the activity.

Will Google actually make the ajax call to pull down this info and use it in calculating page relevancy / link juice flow?

I'm hoping that it does not because the user profile pages are not very Google index worthy, and I don't want all those links to the User profile pages diluting my home page's link juice flow away from other more important links.

回答1:

No, it will not crawl AJAX content by default.

http://code.google.com/web/ajaxcrawling/ has instructions on how to make AJAX content crawlable, but those are explicit steps you need to take, it isn't automatic

回答2:

Maybe. If you want to guarantee Google won't spider your JSON responses, put them in robots.txt. That isn't security, though; indeed, it's the first place a cracker will look for "interesting" pages. And other crawlers may ignore it.

回答3:

Google is definitely crawling content in our page that is only referenced within an AJAX request.

I'm wondering if Google is going through the page source, and looking for potentially valid URLs, and testing to see if they've got content.

Here is what our request looks like... which might offer some insight into what's going on.

'$(document).ready(function() {
    $("#theDiv").block({ message: 'Getting latest content...' });
    $.ajax({
        url: '/content/pages/articles?count=4&part=true',
        success: function(data) {
            $('#theDiv').html(data);
            $("#theDiv").unblock();
        }
    });
});`

回答4:

Yes, Google crawls dynamic content created using javascript. It can recognize the DOM after loading, including modifications to the title tag. It can also follow links created with the onclick event handler.

Read about it here: http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157

It will respect nofollow however, for your profile links.

回答5:

Google now has a way for you to enable crawling on AJAX pages. If your links contain "#!", then google will change that to "?_escaped_fragment_=" and request that document from your server. However, when it shows that page in the search results, it will show the original URL with the "#!".

http://code.google.com/web/ajaxcrawling/docs/learn-more.html

回答6:

Google itself doesn't crawl AJAX content, but it proposed a scheme to make AJAX content crawl. I wrote a an article to explain that how it works. It also includes very simple code to demonstrate Google Crawling Scheme for AJAX content. Find it here: http://www.tayyabali.com/seo-ajax-content-asp-net-code-to-crawl-ajax

回答7:

Here is some info on AJAX crawling from the makers of JQuery Address plugin.

回答8:

For Ruby users there is a gem which handles this relatively painlessly.... google_ajax_crawler Github repo here, writeup about using it for backbonejs at thecodeabode

回答9:

in the last month google has stopped to support ajax! https://googlewebmastercentral.blogspot.fr/2015/10/deprecating-our-ajax-crawling-scheme.html