How can I get more than 100 results from GitHub AP

Question

I'm using GitHub API Gem and trying to get statistics about contributors's additions, deletions, and commit counts. The problem is that I get only 100 results and can't get access to other pages. It seems to be very common question but I couldn't find the answer.

For instance, let's look at rails/rails repo. There's 1 990 contributors:

  repo = Github::Repos.new user: 'rails', repo: 'rails'
  repo.stats.contributors

What I get is first 100 results.

I tried to query pagination information included in the link header. My output in rails console:

irb(main):001:0> repo = Github::Repos.new
=> #<Github::Repos:0xa6941dc *@current_options ommited* >

irb(main):002:0> res = repo.stats.contributors user: 'rails', repo: 'rails'
=> #<Github::ResponseWrapper *@body omitted* >

irb(main):003:0> res.links
=> #<Github::PageLinks:0xa2a966c @next=nil, @last=nil>

Nothing.

Passing auto_pagination option doesn't change anything for me.

What am I missing?

Answer 1

I'm unsure what you mean by passing in the auto_pagination option since that seems to be something that's configured when creating a new GitHub instance, e.g.,

github = GitHub.new do |config|
  config.auto_pagination = true
end
github.repos.contributors 'rails', 'rails'

Frankly though, I would suggest using the official GitHub API gem -- octokit.rb. It does the pagination work for you and intelligently knows when it can bump the number of items per page to 100.

Answer 2

Faced same issue, my solution was to fall back to git command line:

Count commits:

$ git log --author="X Y" --oneline --shortstat|wc -l
224

Count deltas:

$ git log --author="X Y" --pretty=tformat: --numstat | awk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s, removed lines: %s, total lines: %s\n", add, subs, loc }' -

added lines: 1861, removed lines: 1243, total lines: 618

Answer 3

I tried many things and ended up with the underlying GitHub API HTTP methods. For example:

curl https://api.github.com/repos/rails/rails/stats/contributors 

Got nothing. So, I dropped an e-mail to GitHub Support. Here's the answer from Wynn Netherland:

Thanks for getting in touch. That particular method doesn't support pagination, so we're effectively capping the contributors data to 100 as you've discovered. I can't promise if/when we'll be able to show more data on that one since it's sort of an expensive endpoint for us. Keep an eye on the API developer docs for updates.

Thanks Wynn. So, GitHub Repo Statistics API isn't supporting pagination. There is no way to get contributors list with more than 100 results.