Flutter 3.50 was announced on 11 November with official agent plugins, and one of their rules hot reloads the app through the MCP server every time the agent edits a file under lib/. That's good. But a reload only says the code compiled. It doesn't say the button works, or that the header still fits on a phone.

My loop used to be the same every time: hot reload, take a screenshot, drag the PNG into the chat, paste the widget source, and ask why the button does nothing. Today, I'll show you how I hand that part to the agent itself, with the commands and the real output from my runs.

TL;DR: Tapir runs inside your Flutter debug build and gives Claude Code, Cursor or Codex an MCP server: a Semantics snapshot with a ref for every control, real taps and typing, an effect block that says whether an action worked, and the errors the app raised. Resize the web view to iPhone width and the agent catches layout bugs a desktop window hides.

There is a catch, and I'll be honest about it at the end.

What you'll walk away with:

  • Tapir installed in a Flutter app and wired into your agent
  • What the agent actually sees: a short snapshot with a ref for every control
  • A real fill and tap, and how the agent knows the tap did something
  • A layout bug that a desktop browser hides and an iPhone-sized viewport shows

Let's start.

Install Tapir

Two commands:

flutter pub add tapir dowel dart run tapir tapir:install 

tapir:install patches lib/main.dart for you: it adds the import and calls TapirPlugin.install() behind !kReleaseMode. It also scaffolds ./bin/dowel, a fast CLI wrapper, and registers Tapir as a plugin of Dowel, the CLI and MCP server it rides on. My main.dart after the install:

The guard matters. Debug and profile builds carry Tapir, and a release build leaves it out. Without the guard, anyone who reaches the app's VM Service could drive your app.

Connect your agent

One more command writes the MCP entry:

dart run tapir mcp:install 

It writes this into .mcp.json at the project root:

{ "mcpServers": { "dowel": { "command": "./bin/dowel", "args": ["mcp:serve"], "cwd": "." } } } 

Claude Code doesn't reload .mcp.json by itself, so run /mcp reconnect dowel once. Cursor, VS Code, Windsurf and Codex read the same entry; the MCP setup page has a line for each. When I asked the server for its tool list, it answered with 40 tapir_* tools: tapir_snap, tapir_tap, tapir_fill, tapir_screenshot, tapir_exceptions and the rest.

Start the app through Tapir

The app has to be started by Tapir, not by a plain flutter run, because every command dials the session it records:

dart run tapir start --device=chrome --cdp-port=9333 

The CDP port lets Tapir take screenshots through Chrome and resize the viewport, which we need in a minute. On a native target you pass the device instead (--device=macos, an iOS simulator's id, an Android serial). I stayed on the web here, because the bug I want to show only needs a narrow screen.

My example is a small sign-in page: an email, a password, a "Sign in" button and a "Need help?" link next to the title. It has one bug I left in on purpose.

What the agent sees

The first thing an agent does is take a snapshot. Here is the same call from the terminal, so you can see what the agent gets:

./bin/dowel tapir:snap --interactiveOnly 
- button "Need help?" [ref=e1] - textbox "Email" [ref=e2] typeable: true - textbox "Password" [ref=e3] typeable: true - button "Sign in" [ref=e4] 

Think of a guest list at the door instead of a photo of the crowd. The agent doesn't guess pixel positions from a screenshot. It reads the app's Semantics tree, and every control gets a short ref (e1, e2, ...) it can act on. Five lines cost almost nothing in context, so the agent can take a new snapshot after every step.

Let it drive

Now the agent fills the form with bad values and taps the button:

./bin/dowel tapir:fill --ref=e2 --text="ada" ./bin/dowel tapir:fill --ref=e3 --text="short" ./bin/dowel tapir:tap --ref=e4 ./bin/dowel tapir:snap --grep="Enter|Use at" 
✓ Filled e2 ✓ Filled e3 ✓ Tapped e4 (target unchanged) - textbox "Email": "ada" [ref=e2] typeable: true - text "Enter a valid email." - textbox "Password": "•••••" [ref=e3] typeable: true - text "Use at least 8 characters." 

These are real pointer and key events going into the widget tree, so the validators run the way they run for a person.

The sign-in form at iPhone width, with "Enter a valid email." and "Use at least 8 characters." in red under the two fields
Tapir filled the form with bad values and tapped Sign in.

A green tick only says the event left Tapir. It doesn't say the widget took it. So the agent reads the effect block (--json prints it):

{"effect": {"kind": "text", "verified": true, "value": "ada@example.com"}} {"effect": {"kind": "treeChanged", "changed": false, "routeChanged": true}} 

The first is a fill that stuck. The second is the tap on "Sign in" with valid values: the button itself didn't change, but a new route opened (routeChanged, new in Tapir 1.1.0). The next snapshot shows text "Welcome back, ada@example.com".

What if the agent taps something it can't reach? Before a tap, Tapir checks that the widget is enabled, on screen, stable and not covered. I tapped the old "Sign in" ref again while the welcome page was on top:

Widget ref=e4 is not actionable: obscured by other widget (top=_RenderInkFeatures) 

The refusal names the reason (obscured by, not enabled, off-viewport, not stable), so the agent doesn't retry blindly or add a Future.delayed and hope.

The bug only an iPhone-sized screen shows

On my desktop Chrome window the page looks fine, and tapir:exceptions returns an empty list. Now let's give it iPhone 15 Pro's width:

./bin/dowel tapir:resize_viewport --width=393 --height=852 --dpr=2.0 --mobile --touch --hold 

--hold keeps the size while the agent works, so run it in the background (tapir:device_profile --preset=iphone-15-pro sets the same size for one command only). Then a new snapshot:

✗ ⚠ 1 render error(s) captured on this screen (run tapir:exceptions / tapir_exceptions for full detail): ✗ - overflow: A RenderFlex overflowed by 67 pixels on the right. - button "Need help?" [ref=e1] overflow: true 

The agent didn't need a screenshot to find this. The snapshot puts the error on top and marks the widget that is pushed off the screen. The title and the link sit in a Row, and the title has no room to wrap. The fix is one widget, Expanded around the title:

tapir:hot_reload_and_snap reloads the app and returns a fresh snapshot, the screen and the recent errors in one round trip. The overflow flag is gone from the snapshot.

Left: the title row at 393 pixels wide, with Flutter's yellow and black overflow stripe over the "Need help?" link. Right: the same row after the fix, the title wraps to two lines and the link fits
Before and after the Expanded fix, at iPhone 15 Pro width.

Note: tapir:exceptions keeps the whole history, so the old overflow is still in the list after the reload. Note the time before an action and pass --since=<time>; after a hot restart that call came back empty.

That's the loop I wanted: the agent changes the code, reloads, checks the screen at the size the user has, and only then says it is done.

The theme

The screenshots are dark because the example uses a dark theme in my blog's colours. If you want your copy to look the same, here is the file:

Where Tapir is the wrong tool

  • It is not your regression suite. Tapir drives a live app; it writes no test file. Keep integration_test or Patrol in CI for flows that must never break. The official Dart and Flutter MCP server has a Flutter Driver tool and a prompt that turns a user journey into a driver test, which is closer if a test file is what you want.
  • Debug and profile builds only. A release build has no Tapir in it, on purpose.
  • The web session trades one thing away. With --cdp-port, Flutter runs on the web-server device, which can't evaluate Dart expressions, so tapir_evaluate returns an error there. Start without the port when you need it.
  • A widget test can catch this overflow too, if you write one at that width. Tapir is for the screens nobody wrote a test for yet, while the agent is still building them.

Install

flutter pub add tapir dowel dart run tapir tapir:install dart run tapir mcp:install 

The docs, every command and the full MCP tool list are on the Tapir page. If your agent reads skills, npx skills add anilcancakir/tapir teaches it the rules above.

How do you check your agent's UI changes today: screenshots in the chat, a test suite, or by hand? I'd like to hear it in the comments.

That's all. Have a nice day.

Originally published on my blog.