Facebook Scraper and Angular App with HTML5Mode Ha

Question

I added seo support in my Angular app

http://staging.lovented.com

I configured Html5Mode including Hashbang urls

 $locationProvider.html5Mode(true).hashPrefix('!');

By doing this both urls are valid

http://staging.lovented.com/contest/extension_test_contest

http://staging.lovented.com/#!/contest/extension_test_contest

Then I setup the meta tags like

<meta name="description" content="{{description}}">

 <!--Facebook Meta Tags-->
<meta property="og:title" content="{{title}}" />
<meta property="og:url" content="{{siteUrl}}#!{{contentUrl}}" />
<meta property="og:description" content="{{description}}" />
<meta property="og:image" content="{{serviceUrl}}{{imageUrl}}" />

If I go to this page http://staging.lovented.com/contest/extension_test_contest

the meta tags set dynamically, the way google crawl the site by appending escaped fragment at the end of url and system will take the snapshot. You can click on the below link and see the source of the page. Everything is there.

http://staging.lovented.com/contest/extension_test_contest?_escaped_fragement_=

But the way facebook crawler would work, it looks for #! in the url and replace it with escaped fragment so for fb share I would share this url http://staging.lovented.com/#!/contest/extension_test_contest

But If I run this url in Facebook Debugger, it not seems to scrape the site.

https://developers.facebook.com/tools/debug/og/object/

So, Google crawler would definitely work but I am not sure why Fb not scrape my page. Any suggestion please?

Answer 1

It is because facebook is not using <meta name="fragment" content="!"> when scraping, so if there is no #! in url - facebook will fetch it as a regular page without adding _escaped_fragment_ query parameter…

The same is true for other social networks(at least it was, when I was doing SEO for my application)…

To handle this issue you can add detecting based on user agent. There is good examples how this can be done from prerender.io:

Apache: https://gist.github.com/thoop/8072354

Nginx: https://gist.github.com/thoop/8165802