Job search results flooded by one company: a per-company cap, measured on 270,722 jobs

We keep an index of open jobs read from company career sites. On 27 October 2026, when it covered Greenhouse, Lever, Ashby and Personio, it held 270,722 jobs from 8,476 career sites; every figure below is from that snapshot. SmartRecruiters has joined since, and on 31 October 2026 the index held 469,340 jobs from 9,326 sites. We ran the most ordinary search there is against it, remote jobs posted in the last seven days, newest first, 100 rows, and got back 100 rows from nine companies. Ninety-two of them came from one board.

Nothing was broken. Every row was a real, open, remote job posted that week. The search did exactly what it was told, and the result was useless to anyone who wanted to see who is hiring. Here is why that happens on any job index, and the thirty lines we added instead of the obvious fix.

A job index is not flat

How jobs are spread across boards, on the 27 October snapshot:

• The median board has 9 open jobs. The mean is 32.
• 4,528 boards, more than half, have 10 jobs or fewer.
• 20 boards have more than 1,000.
• Half of all 270,722 jobs sit on 357 boards, about 4% of them.

The top of the list:

Those eight boards, about one in a thousand, hold 31,218 jobs, 11.5% of the index. They are not all the same kind of thing. Some are employers hiring at a scale most companies never reach. One, lever:jobgether, is a remote-jobs aggregator's board: it reposts the jobs of other companies on its own applicant-tracking board. For several of the others, the slug alone does not tell you what the business is.

Why "newest first" hands the page to one board

More than 10,000 jobs from more than 1,200 career sites matched "remote, posted in the last seven days". Sorting those by date and keeping the first 100 gives the page to whoever posts most often. The aggregator's board had posted 92 remote jobs within about three hours, the newest in the index, so it took 92 of the 100 rows; the other eight came from eight companies, one each.

Any ranking that rewards recency or volume does this.

Why we did not write a blocklist

The obvious fix is a list: mark the aggregators and agencies, drop them from searches. We decided against it for four reasons.

1. It is guesswork. A row from a reposting service, a staffing firm and an employer with thousands of openings looks the same: a title, a location, a date, an apply link, on a normal ATS board. Telling them apart means researching each business and making a call about it that we would have to stand behind.
2. It goes stale. One discovery pass this month added 4,236 boards with open jobs to the index. A blocklist only covers the boards someone has already looked at.
3. The jobs are real. Some people want exactly those postings. Dropping a board outright removes them from every search, including the ones that asked for them.
4. The problem is not the kind of board, it is the share. A very large employer can fill a search the same way. What we actually want to prevent is one source taking the page, whatever that source is.

So the rule we wrote does not know what any board is. It only counts.

The cap

The rule, as it runs in the search:

• Matches are sorted newest first.
• Each board may take at most a tenth of the rows asked for (at least 2) while other boards still have matches. Asking for 100 rows means at most 10 per board.
• Rows over a board's limit are not thrown away. They go to an overflow list, and if the first pass does not fill the run, the overflow fills it in date order.
• The final rows are sorted by date again.
• The cap is off when the user names companies, because then the user asked for that company.

The key is the board, system:slug, not the company name, so it never depends on matching names across systems. In TypeScript:

type Row = { id: string; posted_at?: string; first_seen_at?: string }; function when(r: Row): string { return r.posted_at ?? r.first_seen_at ?? ''; } function newestFirst(a: Row, b: Row): number { return when(b).localeCompare(when(a)); } function boardOf(r: Row): string { return r.id.slice(0, r.id.lastIndexOf(':')); // "lever:acme:123" gives "lever:acme" } export function spread(matches: Row[], maxItems: number, perBoard = Math.max(2, Math.ceil(maxItems / 10))): Row[] { const sorted = [...matches].sort(newestFirst); if (perBoard <= 0) return sorted.slice(0, maxItems); const taken = new Map<string, number>(); const first: Row[] = []; const overflow: Row[] = []; for (const r of sorted) { if (first.length >= maxItems) break; const n = taken.get(boardOf(r)) ?? 0; if (n < perBoard) { first.push(r); taken.set(boardOf(r), n + 1); } else if (overflow.length < maxItems) overflow.push(r); } const picked = [...first, ...overflow.slice(0, Math.max(0, maxItems - first.length))]; return picked.sort(newestFirst); } 

The overflow matters more than it looks. Without it, a narrow search where only three boards match would return 30 rows when 100 were asked for, while 70 matching rows sat unused. With it, the cap only decides which rows come first when there are more matches than rows. It never makes a search return less than it could.

What it did to the same search

Same index, same input, cap on at its default:

Two boards reached their limit of ten; the other 80 rows came from 65 companies. The aggregator is still in the result, at the same share as any other large board, so nobody who wanted its postings lost them. The cost is visible in the last line: the 100 rows are no longer the 100 newest, they are the newest spread across companies, and the oldest of them was posted about two and a half hours earlier than before. For a seven-day search that is a good trade. For a "last hour" feed it might not be, which is why the cap can be set or switched off.

The run's log says which rule applied, in the form <n> jobs match; returning the 100 newest, at most 10 per company while others match, so a result that looks thin can be explained from the log alone.

What we would tell you before you build a job search

1. Look at the distribution before you choose a ranking. A median of 9 and a top board of 5,019 means a rank by date or by match count will tend to hand the page to the biggest boards.
2. Cap the source, do not judge it. A count needs no research, never goes stale, and treats an aggregator and a giant employer the same way.
3. Keep the overflow. A cap that can shorten a result is a bug waiting for a narrow query.
4. Switch it off when the user names a company. "All of Stripe's jobs" should be all of them.

The same idea, an equal share that lifts once everyone has had a turn, came up earlier when four remote job feeds raced each other, in a post about a run that returned 200 rows or 104. And if you are building the board list itself, half the boards you find by guessing a slug belong to a different company.

Try it

The index and the cap are in the Career Site Jobs Search Actor on Apify. The field is Max jobs per company: empty means a tenth of the run, a number sets your own limit, 0 turns it off, and naming companies turns it off for you. Every row links back to the job on the employer's own career page.

The same index also runs as a feed. Career Jobs Feed returns only the jobs that are new since your last run, each one once per feed name, with the search's keyword, place, system and pay filters: schedule it daily and the de-duplication is done before the data reaches you. In the feed, Max jobs per company is off unless you set it, because a feed should hand you every new job by default. Its prefilled run ("data engineer", "analytics engineer") returned 38 new jobs from 31 companies on 31 October 2026, six of them one company's same title in six places.

Dodo Data is a small job-data shop. Claude, Anthropic's AI, does the engineering.