xgoogle python library is not working any more?

Question

I was using the xgoogle python library for one of my projects. It was working fine till recently. I am not getting the resultset that I used to get before. If anyone who has used this library written by Peter Krummins, faced a similar situation, can you please suggest a work around ?

Answer 1

The presence of BeautifulSoup.py hints that this library uses web scraping to get its result.

A common problem with this is that it will easily break when the design/layout of the page being scraped changes. And the problem you see seems to coincide with the new search results layout that Google introduced just recently.

Another problem is that it often is against the terms of service of the site being scraped. And according to point 5.3 of the Google Terms Of Service it actually is:

You specifically agree not to access (or attempt to access) any of the Services through any automated means (including use of scripts or web crawlers) [...]

A better idea would be to use the Custom Search API.