Agent ready isn't a docs folder. It's a scoreboard.

For 126 nights between March and September, a script at Meta handed the same UI tasks to fresh AI agents four ways (three on some nights): with Meta's open source design system, with that system plus Tailwind, with shadcn, and with plain HTML. Then it type-checked and scored what came back.

Shadcn came out ahead on 67 of those nights.

The design system is Astryx, and it was number one on GitHub trending on Tuesday (3,111 stars that day) with "agent ready" in its one-line pitch. The scores sit on its own public wiki. I think that ledger is the most agent-ready thing in the repo, and it's the reason I trust the rest of it.

What "agent ready" looks like on disk

In a July HN thread, ildon asked the question I had: what makes this design system agent ready, compared to a traditional one? Their guess was the heavy commenting. So I installed it in a throwaway folder (version 0.7.12) and read the files.

Every component carries a typed doc file. Button.doc.mjs is 435 lines: props with types and defaults, do and don't rules, WCAG contrast targets for each state, a full example, and two overlays, one in Chinese and one called docsDense that says the same things in fewer tokens. The core npm package ships 262 of these files, about 2.4 MB, inside node_modules.

The CLI is what reads them. Nearly every command has a --json mode, and an error comes back with a stable code and a guess:

$ npx astryx component Buttn --json { "apiVersion": 1, "error": "No component named \"Buttn\"", "code": "ERR_UNKNOWN_COMPONENT", "suggestions": [ { "name": "Button", "reason": "similar name (distance 1)" }, { "name": "IconButton", "reason": "keyword \"button\" (distance 1)" } ] } 

The README promises those codes are append-only, so an agent can branch on them safely. astryx manifest --json describes all 17 commands (43 counting subcommands) in one 90 KB payload. A gap-report command even lets an agent draft an issue for the package that owns a missing feature; nothing gets filed without --confirm-public.

Then astryx init --features agents wrote two files into my project: a 29-line block between ASTRYX:START and ASTRYX:END markers in AGENTS.md, and a GEMINI.md that only imports it. (With a CLAUDE.md in the project, it writes there too, since Claude Code skips AGENTS.md in that case.) The block is a short workflow (start every page from a template), two setup imports, and six rules: no raw <div>, no hex or px values, and so on.

My favorite detail is a code comment in the generator, above the last rule. That rule is a "SELF-CHECK" telling the agent to re-read its file and swap raw markup for components. Per the comment, the other rules alone still left raw CSS in roughly 11 to 13% of complex runs, and the re-read pass cut that about four times. A line of prompt, with a measured effect written next to it.

The test that grades the agent

That measurement comes from internal/vibe-tests. The default battery holds 70 prompts, things like a delete confirmation dialog with a destructive button. Each prompt lists the components a good answer should use, and the agent never sees that list. Every prompt gets a fresh agent with no prior context, and every target gets the same task text; only the system's name and the project folder change. Scoring is a script, not a model: five dimensions averaged with equal weight, where correctness starts at 100 and loses 15 points per TypeScript error.

The wiki's score ledger has one row per nightly run. I tallied it with my own script:

126 nightly runs, 2026-03-20 to 2026-09-01 best Astryx setup ahead of shadcn 46 shadcn ahead of both Astryx setups 67 level 13 mean overall: Astryx 90.9, shadcn 94.6, plain HTML 81.3 Astryx ahead of plain HTML on 115 of 126 nights 

Astryx clearly beats raw HTML. Against shadcn it loses more often than it wins, and the losses are about correctness. Since late June the ledger also breaks Astryx's score into its five parts: on the 32 nights shadcn came out ahead, Astryx's correctness averaged 69, against 92 on the other 35. Its other four scores barely moved.

Docs can't outvote the training data

The failure lists are the part worth reading. The nightly debugger posts them to one rolling issue, #3164. Agents wrote onPress where Astryx wants onClick. They wrote gap="md" where it wants gap={2}. They wrote <Button>Save</Button> where Astryx requires a label prop, and justify="space-between" where the value is "between". On August 29, the spacing strings and onPress alone made 40 of the night's 44 type errors. The verdict in that entry, as in many others: the docs were correct.

So the agents weren't missing the docs. They were writing the API they had seen everywhere else. Shadcn never has to teach an agent anything, because the model already knows it.

A May issue, #2182, hit the same wall from the other side. A core contributor reported that agents who read the 12.5 KB dense theme doc scored 20% and 45% compliance on a separate benchmark, while one that skipped it and guessed scored 94%. The same issue noted that MUI and Ant Design score well there because their theme APIs are already in training data. Another contributor later couldn't reproduce the collapse (in his runs, no docs at all did worst), but a 3 KB version did the same job with about 80% fewer tokens. Today astryx docs theme --dense prints 2.6 KB, with the depth moved to separate pages.

Here's my claim. "Agent ready" can't just mean "we wrote docs for agents." Astryx's agent docs are more thorough than any I've read, and agents still type onPress. It has to mean you know the distance between your API and what models already expect, prop by prop, and you pick a side each time: bend toward the habit, or keep your API and let the type checker teach. On the prop I checked, Astryx keeps its own: label is still required on Button in 0.7.12. A scoreboard is the only honest way to see what that choice costs.

Where it gets shaky

The ledger's last row is September 1, and the newest nightly issue I found is from September 2. I can't tell from the repo whether the runs moved somewhere else or stopped.

The scoring also cuts against strictness. Astryx's date input takes a string type shaped like YYYY-MM-DD, so an agent's plain string state fails the compile. A library typed with plain strings would accept the same code. Some of the gap is Astryx catching mistakes the other targets let through, and a score built on type errors counts both the same way.

And the global install reaches into your agent setup. By its docs, it writes a skill file in the background on first run and links it into ~/.claude/skills and up to six other agents' folders; astryx init --user --remove-agents turns that off. I only ran the CLI from inside a project, with HOME pointed at a scratch folder, and no skill file showed up there.

Try it

npx @astryxdesign/cli search button npx @astryxdesign/cli component Button --dense 

Use the scoped name. The README warns that a bare npx astryx resolves to an unrelated npm package until the CLI is installed.

When an agent keeps writing onPress against your component, is that a docs bug or an API bug?