"This feels risky" is a sentence a reviewer can act on. It is not something a CI job can branch on. Getting a model to produce a number you can threshold usually means asking for JSON, then writing a parser, then writing a fallback for when the parser fails.

This post sets up the other path: a diff goes in, typed answers come out. You get a score, the option the model picked, and a confidence value from 0 to 1. You will see the config for Cursor, the question set worth copying, and a script that turns the answers into an exit code.

What typed output changes

A chat model asked to "rate this diff from 1 to 5" writes a sentence that contains a 4. A decision model answers the question as a distribution over the levels you defined, plus a confidence value that tells you whether the distribution was concentrated.

The difference matters at the edges. A prose answer of "around a 3, leaning low" leaves you to decide what that means for a merge. A typed answer of score: 2.4, confidence: 0.41 says the model split its probability between two adjacent levels and is not sure. That is a signal to send the change to a person, and your code can check it in one comparison.

None of this replaces review. Google's reviewer guide puts design, functionality, complexity and tests ahead of style, and a single score does not cover any of them well. The score is a triage signal: it decides which changes get a second pair of eyes first, not whether a change is correct.

Add the server to Cursor

Cursor reads MCP servers from ~/.cursor/mcp.json for every project, or from .cursor/mcp.json in a repository for that project only. The Cursor MCP documentation lists both, along with the Customize page in the sidebar.

Create a key under Settings › API Keys first. Then keep it in your environment, not in the file:

{ "mcpServers": { "jevstation": { "url": "https://jevstation.com/api/mcp", "headers": { "Authorization": "Bearer ${env:JEVSTATION_API_KEY}" } } } } 

Cursor resolves ${env:NAME} inside url and headers, so this file can be committed. Each developer sets JEVSTATION_API_KEY in their shell profile, then restarts Cursor; the Jev Cursor integration page notes that Cursor reads environment variables when it starts, and covers the same setup with the failure modes spelled out.

There is no command, args, or envFile here. The server is remote and speaks Streamable HTTP, one of the two transports defined in the MCP specification, so nothing runs on your machine.

After the restart, open Customize and confirm the toggle is on. The server exposes several tools, including jev_decide, jev_run_question_set and jev_guard_tool_call. If the server shows an error, open Output, then MCP Logs. A 401 there usually means Cursor started before the variable existed, or the key is missing or has been deleted.

One detail about the account behind the key: JevStation is an independent workspace that runs Jev, TypeSafe's System One model, and bills per evaluation. New accounts include 50 credits. A call costs one credit when the state is at most 8,000 characters and the set has at most five questions, and three credits above either limit.

Write the rubric before the prompt

The question set is the ceiling on the answer. "How risky is this change?" with levels low, medium, high returns a number you cannot defend, because every reviewer means something different by medium.

Concrete levels are the fix. This set asks three things about one diff:

id type what it asks
risk score how risky is this to ship without a second human review?
area choice which part of the system does the diff change most?
tests noul does the diff add or update a test that would fail if the behaviour changed?

The score levels, written as descriptions rather than adjectives:

  1. Mechanical change: rename, formatting, comment, or a dependency bump with no logic edit.
  2. Localized logic change covered by tests in the touched files.
  3. Logic change in a shared module, or the touched path has no test.
  4. Touches auth, payments, a data migration, or deletes data.

Level 4 is not "very risky". It names the conditions that make a change expensive to get wrong. A reviewer reading the level list can disagree with a specific level instead of arguing about a word.

Two rules keep the set useful. Ask one judgement per question, and combine the answers in code. If you want a single number, weight the three answers yourself; a weighted sum is easier to change later than a rewritten prompt. And keep the count at five or fewer, because a sixth question moves the call from one credit to three.

When you do get a score, read it with its distribution. A score of 2.4 across four levels usually means most of the probability sat on level 2 with some on level 3, not that the change is a little worse than level 2. The answer carries the probability of each level for exactly this reason.

Ask Cursor to score a diff

With the server connected, the agent can call jev_decide itself. Naming the tool and the questions removes most of the guesswork:

Run git diff --cached --stat and git diff --cached, then call jev_decide once. state: a JSON object with keys "files" (the stat output), "diff" (the diff, truncated to 8,000 characters if longer) and "message" (my last commit message). questions: risk: score, "How risky is this change to ship without a second human review?" levels: [mechanical change / localized logic change with tests / shared module or missing test / auth, payments, migration, deletion] area: choice, "Which part of the system does the diff change most?" options: api, data, ui, build tests: noul, "The diff adds or updates a test that would fail if this behaviour changed." Show me the score, the confidence, and the probability of each level. Do not summarise the diff. 

The answer comes back in one response, with an object per question id:

{ "answers": { "risk": { "score": 2.4, "confidence": 0.41 }, "area": { "choice": "api", "confidence": 0.86 }, "tests": { "noul": 0.12 } } } 

Two things are worth noticing. The tests answer is a probability of 0.12, not a boolean, because a yes/no question returns a number you can threshold and noul answers carry no separate confidence value. And the risk answer at 0.41 confidence is the model saying the diff sat between two levels. On a change that touches payments, that is a reason to stop, even though 2.4 looks unremarkable.

Cursor asks for approval before each MCP call by default, and you can expand the call to read the exact arguments. That review step is where you catch a diff that includes a token or a connection string, because the state is sent to the model for evaluation.

Put the same judgement in CI

The MCP route is for a person working in an editor. A pre-push hook or a PR job needs its own path, and the same questions work through the HTTP API. The thresholds below are placeholders; tune them on your own merged diffs. This script reads the staged diff and exits non-zero when the change needs a human:

// review-score.mjs — run: node review-score.mjs import { execSync } from "node:child_process"; const MAX_STATE = 8000; const diff = execSync("git diff --cached --unified=3", { encoding: "utf8" }); if (!diff.trim()) { console.error("No staged changes. Stage a diff first."); process.exit(2); } const state = { files: execSync("git diff --cached --name-status", { encoding: "utf8" }).trim(), diff: diff.length > MAX_STATE ? diff.slice(0, MAX_STATE) + "\n[diff truncated]" : diff, }; const res = await fetch("https://jevstation.com/api/v1/systemone", { method: "POST", headers: { Authorization: `Bearer ${process.env.JEVSTATION_API_KEY}`, "Content-Type": "application/json", }, body: JSON.stringify({ state, questions: { risk: { type: "score", instructions: "How risky is this change to ship without a second human review?", criteria: [ "Mechanical change with no logic edit", "Localized logic change covered by tests in the touched files", "Logic change in a shared module, or no test for the touched path", "Touches auth, payments, a data migration, or deletes data", ], }, tests: { type: "noul", instructions: "The diff adds or updates a test that would fail if this behaviour changed.", }, }, }), }); if (!res.ok) { console.error(`HTTP ${res.status}: ${await res.text()}`); process.exit(2); } const body = await res.json(); const data = body.data ?? body; // the API wraps the result in "data" const { risk, tests } = data.answers; console.log( `risk ${risk.score.toFixed(2)} (confidence ${risk.confidence.toFixed(2)}), ` + `tests ${tests.noul.toFixed(2)}` ); console.log(`credits: ${data.credits?.charged} charged, ${data.credits?.remaining} left`); const needsHuman = risk.score >= 2.5 || risk.confidence < 0.6 || tests.noul < 0.3; if (needsHuman) { console.error("Route to a human reviewer before merge."); process.exit(1); } 

Three choices in that script are deliberate. The threshold lives in one expression, so tuning it does not touch the questions. A truncated diff is marked with [diff truncated] rather than silently clipped, because a state that quietly drops half the hunks produces a confident answer about the wrong change. And a missing key fails with the HTTP status instead of a stack trace, which is what you want in CI logs.

Redact before you send. state goes to the model for evaluation, so strip tokens, passwords and connection strings from the diff first, or the script becomes a way to leak them.

Three failure modes

Confidence is not correctness. A confidently wrong answer is still wrong. Start with thresholds that escalate more often than you think necessary, log the answers next to the outcome, then tighten. A third-party pilot on 300 examples across three classification datasets found that calibration varied by dataset: Jev had a better Brier score on two and a worse one on the third. The author notes it is a pilot, not a leaderboard or a universal model ranking. That is another reason to treat confidence as a prior rather than a guarantee.

The score is not severity. A weighted average hides which level carried the probability. A 2.4 with most mass on level 3 and some on level 1 can describe a change that deletes rows in a migration. When the score lands near a threshold, print the distribution and treat the scalar as only half the answer.

Aliases move under you. jev-latest points at the current release, so the answers your thresholds were tuned against can shift without a commit on your side. Log the versioned model id that answered, and pin it once the bands are stable.

There is a fourth, quieter one: an MCP tool is a filter, not a security boundary. A guard tool that decides whether to run a command is useful in front of an approval prompt, and it does not replace the permissions underneath it or the allow-list in your harness.

Where this earns its keep

The setup fits a specific job: deciding which changes deserve the careful reviewer first, when the alternative is a queue sorted by arrival time. It also fits routing, triage and release notes, where a branch in code is more useful than a paragraph.

It does not fit style, formatting, or anything a linter already decides, and it cannot tell you a change is correct. The score is a second opinion with a number attached.

Start with the three questions above on your last twenty merged diffs. Compare the scores with the incidents that followed, or with who ended up reviewing each change. If the bands hold, move the threshold into CI. If they do not, the questions need another level description, not a new model.