Verifying Googlebot in Rails

Question

I am looking to implement First Click Free in my rails application. Google has this information on how to verify a if a googlebot is viewing your site here.

I have been searching to see if there is anything existing for Rails to do this but I have been unable to find anything. So firstly, does anyone know of anything? If not, could anyone point me in the right direction of how to go about implementing what they have suggested in that page about how to verify?

Also, in that solution, it has to do a lookup every time to try and detect google, that seems like its going to be a big performance hit if I have to do it every page load? I could cache the IP if it has been verified in the past but Google have stated that their IP's change so at some point it may no longer belong to them. Although it probably doesn't happen regularly so it may not be that big of an issue.

Many thanks!!

Answer 1

I've built a Ruby gem for that recently, it's called "legitbot".

You may learn if a Web request comes from a supported bot using

bot = Legitbot.bot(userAgent, ip)

"legitbot" does this looking into User-agent and searching for a bot signature, i.e. how bots identify themselves. This doesn't guarantee that the Web request IP really comes from e.g. Googlebot. To make sure it is, call

bot.detected_as # => "Google"
bot.valid? # => true
bot.fake? # => false

Supported bots are Googlebot, Yandex bots, Bing, Baidu, DuckDuckGo.

Answer 2

Check out the browser gem: https://github.com/fnando/browser

What I'd do is use the

browser.bot?

method to check if your site is being accessed by a bot or not. If you care about the Googlebot specifically, you could check if

browser.name

includes googlebot. Keep in mind that this gem just checks the user agent sent by the client's browser, which could of course be spoofed. Sounds like that isn't a huge concern for your purposes.