Open Source
gh api --paginate --slurp --jq 'length' Counts Pages, Not Issues
James Joyner DEV Community
28 views
gh api --paginate --slurp --jq 'length' is the command everyone reaches for to count issues in a repository. It does not count issues. It counts pages.
I found this while writing pagination recipes for a toolkit, and then found the same wrong example in a lesson I had published — with 1,797 search impressions a month on it. So this is partly a correction and partly the mental model that stops it happening again.
What --slurp actually does
From gh api --help (gh 2.100.0):
Each page is a separate JSON array or object. Pass --slurp to wrap all pages of JSON arrays or objects into an outer JSON array.
Read it twice. Without --slurp, --paginate emits one JSON array per page and applies your --jq filter to each page in turn. With --slurp, you get one array whose elements are the pages — an array of arrays. length on that is the number of pages.
You can see the shape without a token. Simulate two pages from the public API and wrap them the way --slurp does (jq -s slurps its inputs into one array, which is exactly the --slurp semantics):
curl -s 'https://api.github.com/repos/cli/cli/pulls?per_page=5&page=1' > p1.json
curl -s 'https://api.github.com/repos/cli/cli/pulls?per_page=5&page=2' > p2.json
jq -s 'length' p1.json p2.json
# 2 ← pages
jq -s 'add | length' p1.json p2.json
# 10 ← pull requests
add concatenates the page arrays into one list. Then length counts items. Every whole-collection filter after --slurp starts with add.
The three flags, and how they combine
Flag
What it does
Needs
--paginate
Follows Link: rel="next" (REST) or pageInfo.hasNextPage (GraphQL with $endCursor)
—
--jq EXPR
Runs the filter per page, unless --slurp is present
—
--slurp
Wraps all pages in one outer array before --jq runs
--paginate
The rule: anything that counts, sorts, or deduplicates across pages needs --slurp and then add.
# Wrong — one number per page
gh api --paginate repos/OWNER/REPO/issues --jq 'length'
# Wrong — number of pages
gh api --paginate --slurp repos/OWNER/REPO/issues --jq 'length'
# Right
gh api --paginate --slurp 'repos/OWNER/REPO/issues?per_page=100' --jq 'add | length'
Two more things in that last line. per_page=100 is the maximum for almost every list endpoint; the default of 30 makes --paginate do three times the requests. And the path is quoted because & would otherwise background the command.
The second trap on the same endpoint
/issues returns pull requests too. Every PR is an issue in GitHub's data model, with a pull_request key set. So the count above is still not "open issues":
gh api --paginate --slurp 'repos/OWNER/REPO/issues?per_page=100' \
--jq 'add | map(select(.pull_request == null)) | length'
Endpoints that wrap their list
Some endpoints return an object — { total_count, workflow_runs: [...] } — not a bare array. add on those merges objects, which is not what you want. Reach into the list first:
gh api --paginate --slurp 'repos/OWNER/REPO/actions/runs?per_page=100&status=failure' \
--jq 'map(.workflow_runs) | add | length'
Same for search/* (map(.items) | add), artifacts, and anything else with a total_count.
A recipe you can keep
Open pull requests older than two weeks, oldest first, drafts excluded — run against a repository you have access to:
gh api --paginate --slurp 'repos/OWNER/REPO/pulls?per_page=100&state=open' \
--jq 'add
| map(select(.draft | not))
| map({number, title, author: .user.login, created_at})
| sort_by(.created_at)
| .[]
| "\(.number)\t\(.created_at[:10])\t\(.author)\t\(.title)"'
gh api --jq prints strings raw, so the output goes straight into cut, sort, or a spreadsheet.
Where the rest of these live
I put ten of these patterns — issues counted by label, failed runs this week, repositories in an org missing a CODEOWNERS file, GraphQL cursor pagination, idempotent label writes — in a free file: gh api pagination recipes. Every jq filter in it was run against the live API before publication, which is how the counting bug got caught.
The full lesson on gh api — placeholders, -f versus -F, GraphQL, response headers, the safety habits — is here: gh api: the full GitHub API from the command line. It is corrected now.
Read original: https://dev.to/jjoyneriv/gh-api-paginate-slurp-jq-length-counts-pages-not-issues-1i1b
← Previous
Your First AWS Production Architecture: From a Simple Application to a Scalable System
Next →
Someone Force-Pushed Over main. Here Is the Reflog Nobody Knows They Have.
Related
What 435 directory submissions taught us about launching software
Open Source
27
Dev.to (EN Zone)
One global threshold is how you delete valid data
Open Source
30
DEV Community
Pacing the Frontier – Tahuna: AI Training Infrastructure, Now Open Source [P]
Open Source
33
Reddit r/MachineLearning
I made an alternative to the archived filebrowser
Open Source
30
Dev.to (EN Zone)
Comments0
No comments yet — be the first