career-ops is an open-source job-search tool with over 80,000 stars on GitHub. One part of it, openrouter-runner, takes the job postings in your pipeline, opens each page in a headless browser, sends the job text to a free model on OpenRouter, and writes a score into your tracker.

On 23 October I ran it on 15 live Workday postings (career-ops v1.37.0). All 15 rows were marked done. All 15 got a report.

Then I counted what the model had actually received.

What the model saw

Between 0 and 397 characters of job text per posting. Six of the 15 got nothing at all. The rest got the page frame, for example:

Skip to main content Loading Follow Us © 2026 Workday, Inc. All rights reserved.

That is 80 characters. No title, no requirements, no description.

The cause was one line. After opening the page, the runner waited a fixed 2 seconds, then read the body. Workday builds the job description with JavaScript after the page loads. When I reran the same postings the next day, the description appeared 5.0 to 9.2 seconds after navigation started, on 13 of the 15. Two seconds was never going to be enough.

Why "done" was the real bug

A slow page is normal. The bug was what happened next: the pipeline treated "the model replied" as "the posting was evaluated".

To show it without any real model, I pointed the runner at three local pages with no job text, and replaced the model with a stub that always answers Score: 4.2/5. All three went out with 0 characters of job text. All three were marked done. The tracker said 4.2/5 | Evaluated.

Nothing in the tracker separates a real score from a score about nothing. Someone reading it would apply, or skip, based on a number that came from an empty page.

A second bug in the same runner

The runner reuses the prompt that career-ops gives to agents that have tools. That prompt tells the model to search the web, run other scripts, check the CV. The runner sends no tools.

So the free models did what the prompt asked. On Workday pages with no description, 19 of 19 replies were tool calls, or said they were about to make one. On real descriptions (7.8K to 16K characters), 4 replies came back: 3 tool calls and 1 empty. Across all 23 replies, the score parser found 0 scores.

How it got fixed

I filed both bugs as issue #5809 on 24 October, with the numbers above and the stub script. Fixing it took four pull requests. Three are mine:

PR What changed Merged
#5814 (by bercedev) A posting that wasn't read is not saved or marked done 25 Oct
#5817 The runner tells the model it has no tools, and strips the steps it can't run 26 Oct
#6017 Model timeout raised from 15 s to 120 s 27 Oct
#6169 Wait for the job description, not for a fixed 2 s 6 Nov

6017 came out of the same work. The 15-second timeout was set in June, when the two prompt files the runner sends added up to about 31K characters. By October they were about 113K. On the free model rotation, a timeout also put the model on a blacklist that survives restarts.

After #5814, the false "done" was gone. But it showed the next problem: with the 2-second wait still there, a Workday posting could never be read, so its row stayed pending forever. I measured 20 live Workday postings, 4 each from NVIDIA, Salesforce, Adobe, Intel and Mastercard:

  • With the fixed 2 s wait: 0 of 20 readable. All refused as too short, in 3 to 4 seconds.
  • With a wait for the job text: 20 of 20 readable, 3,680 to 9,072 characters, in 3.3 to 6.9 seconds.

6169 first tries career-ops' own job-board API reader (Workday, Greenhouse, Lever and others), and only falls back to the browser if that returns nothing. In the browser it waits until the main content holds more than 1,500 visible characters, or until the page clearly says the posting is closed, with a 15-second cap. A 4xx or 5xx page skips the wait.

What review added

The maintainer asked for changes, and the automated reviewer (CodeRabbit) raised findings over several rounds. Not all of it was noise. One finding was real, and it came from my own change: a page with an "Apply now" button and a description still saying "Loading..." could end the wait early. I wrote a test for exactly that page, watched it fail, then fixed it.

6169 was merged on 6 November and shipped in career-ops v1.40.0 on 10 November. On the issue, the maintainer wrote: "Thanks for fixing it yourself."

What I check now, in any pipeline that sends fetched content to a model

  1. Count what the model received. Characters, not "page loaded". It is one log line.
  2. Don't mark work done because a reply came back. Mark it done when the input passed a check.
  3. Don't let the prompt promise tools the code doesn't send. The model will try to use them.
  4. Wait for content, not for a clock. A fixed sleep is a guess about someone else's server.

The tests passed the whole time, and the tracker looked full. That is what made it hard to see.

Links

AI disclosure: I found and fixed these bugs with the help of AI coding tools (Claude Code), and drafted this post with Claude. Every number comes from the linked issue and pull requests, and I checked the text before publishing.