We Renamed One Button and Broke 63 Tests: A Test Maintenance

Here's the diff that started it:

- <button id="submit-btn" class="btn btn-primary">Submit</button> + <button data-action="confirm" class="btn btn-brand">Confirm</button> 

A designer wanted consistent naming across the design system. A frontend developer made the change in five minutes. It looked harmless, and visually it was.

By the time the pipeline finished, the QA lead, Anjali, was staring at a wall of red:

FAILED checkout.spec.ts > places an order FAILED profile.spec.ts > updates address FAILED onboarding.spec.ts > completes wizard FAILED invoices.spec.ts > downloads invoice ... (59 more) 63 failed, 112 passed, 175 total 

Not one of those 63 failures was a bug. The product worked perfectly. The tests were broken.

This case study follows what happened next: the real cost of test maintenance, the decision framework the team used, and a practical plan to cut maintenance work without losing coverage. Reading time: roughly 6 minutes.

The Background

Anjali kept a simple spreadsheet tracking where her team's hours went. When she finally totaled it for the quarter, one line stood out:

Writing NEW tests: 22% Investigating real bugs: 27% Triaging false failures: 16% Fixing broken locators/waits: 35% 

More than a third of the QA team's capacity went to keeping existing tests alive rather than creating new value. That's the hidden tax of locator-based automation.

Why Locator-Based Tests Are Fragile

Traditional scripted tests find elements through selectors. A typical page object looks like this:

// pages/CheckoutPage.ts export class CheckoutPage { constructor(private page: Page) {} submitButton = '#submit-btn'; couponInput = 'input[name="coupon_code"]'; totalLabel = 'div.summary > span.total-amount'; async placeOrder() { await this.page.click(this.submitButton); await this.page.waitForSelector('.order-confirmation'); } } 

Every one of those strings is a contract with the DOM, and nobody told the DOM it signed one. Here's how the fragility scales:

1 UI change x N tests that touch that element = N broken tests 

If your "Submit" button appears in 63 tests (login, checkout, profile, onboarding), a single rename is 63 failures. Page objects reduce the edit effort to one file, but they don't eliminate the detection and triage effort: someone still has to open each failure and figure out whether it's real.

The Decision Framework

Anjali presented three options to the team. Here's the comparison she used:

Option B is genuinely good practice, and you should do it regardless. Stable test IDs make any suite more resilient:

// Stable, explicit hooks for tests <button data-testid="checkout-confirm" className="btn btn-brand"> Confirm </button> 

But the team noticed a problem with relying on it alone: it requires every developer to remember, on every change, forever. The suite's stability depended on human consistency.

So they went with a hybrid: adopt data-testid conventions and pilot a tool that doesn't depend on a single hard-coded locator in the first place.

The Pilot

They picked their 20 most maintenance-heavy tests and ran them through an AI-driven QA platform. This is where BotGauge's AI testing platform entered the picture. The pitch that made them try it: tests are written as plain-English scenarios, and the platform finds elements using multi-modal AI rather than a brittle chain of CSS selectors, with self-healing when the UI shifts. You can run a similar pilot on your own flakiest tests to judge it for yourself.

The same checkout scenario, expressed by intent:

Scenario: Customer places an order Given I am signed in as a customer with items in my cart When I go to checkout And I apply the coupon "WELCOME10" And I confirm the order Then I should see an order confirmation And the total should reflect a 10% discount 

Notice what's missing: no #submit-btn, no div.summary > span.total-amount, no explicit waits. The step "confirm the order" doesn't care whether the button says Submit or Confirm, or what its ID is.

The "rename test"

To validate the claim, the team did the most honest experiment possible. They reproduced the original incident on a branch:

git checkout -b experiment/rename-submit-button # apply the exact same diff that broke 63 tests last month git apply rename-button.patch npm run build && ./scripts/run-pilot-suite.sh 

Results on the 20 migrated tests (illustrative):

Legacy scripted suite (same 20 tests): 14 failed, 6 passed Intent-based suite (20 tests): 0 failed, 20 passed 

That experiment, more than any vendor deck, is what convinced the skeptical senior engineers. The lesson is to replay your own worst incident against any tool you evaluate.

Rolling It Out Safely

They avoided a big-bang rewrite. Their sequence:

Sprint 1 Instrument: tag each test with maintenance cost (edits per month). Sprint 2 Migrate the 20 costliest tests. Run old and new side by side. Sprint 3 Compare: false failures, runtime, and bugs caught by each. Sprint 4 Retire migrated legacy tests. Plan the next 40. Ongoing New tests are written intent-first. Legacy shrinks on its own. 

A tiny script helped them pick what to migrate first:

# maintenance_cost.py import subprocess, collections # Count how often each spec file was edited in the last 90 days log = subprocess.check_output( ["git", "log", "--since=90.days", "--name-only", "--pretty=format:", "tests/"], text=True, ) counts = collections.Counter(line for line in log.splitlines() if line.strip()) for path, edits in counts.most_common(10): print(f"{edits:>3} edits {path}") 

Files at the top of that list are your highest-ROI migration candidates: they're the ones costing you the most to keep alive.

The Results After One Quarter

That last row mattered more than the team expected. Engineers had been quietly avoiding small UI improvements because of the test fallout. Removing that friction improved the product, not just the test suite.

Takeaways

1. Track maintenance as a first-class metric. If you don't measure it, 35% of your QA capacity can vanish unnoticed.
2. Adopt data-testid regardless. It's cheap, it helps any tool, and it documents testable surfaces.
3. Don't depend on human discipline alone. Systems that need everyone to remember something forever will eventually fail.
4. Replay your worst incident. Any evaluation of a testing tool should include reproducing the change that hurt you most.
5. Migrate by pain, not by alphabet. Start with the tests that cost the most, not the ones that are easiest.

Try This Today

Run the maintenance_cost.py script on your own test folder. If the top five files account for most of your edits, you've found your pilot candidates. Then ask the question this team asked: "If a designer renamed our most-used button tomorrow, how many of our tests would fail?"

If the honest answer is "a lot," that's your sign.

What's the most absurd reason one of your tests has ever failed? Tell me in the comments.