Before an AI Agent Touches Checkout: 16 E-Commerce Release Tests

Decision: Let an AI shopping agent search, compare and prepare a cart before you let it submit an order. Keep payment, price-changing options, shipping commitments and irreversible account actions behind a deterministic authorization boundary. Release checkout only when the user can see the exact items, quantity, seller, delivery destination, complete total and payment consequence and explicitly confirms that exact order. A model's claim that it has permission is not an authorization record.

The distinction matters because an agent can be useful without being allowed to spend. It can discover products, summarize policy, compare compatible offers, explain trade-offs and prepare a cart. None of those capabilities proves that it should click a purchase button. OWASP's Excessive Agency guidance recommends narrowly scoped tools, least privilege, execution in the user's context, downstream authorization checks and human approval for high-impact actions. It gives the example of an agent that only needs to read a mailbox being given a send-capable tool; the same design mistake occurs when a shopping assistant that only needs to compare prices is handed a payment-capable credential. OWASP, LLM06:2025 Excessive Agency.

Our group page on AI e-commerce agents describes product questions, cart recovery, recommendations and order tracking. It is a publisher-group page, not independent evidence for its conversion claims. It does not specify a checkout permission model, test cases, or a measured safety result. This article supplies a buyer-side release protocol for that missing decision: what may the agent do, how do you test the boundary, and what should force it to stop?

Who should use this release gate

Use it when a product assistant can write to a cart, create a draft order, select a seller, apply a coupon, change delivery options, submit a purchase, cancel an order or initiate a refund. It applies whether the interface is a browser agent, a marketplace assistant, a chat widget connected to commerce tools, an internal procurement bot or a workflow that hands an LLM a store API.

It is not a recommendation to deploy an agent. If no workflow needs autonomous checkout, keep the system in advice or cart-preparation mode. If a buyer asks the system to compare products, the correct outcome may be a shortlist and a link to the merchant, not a purchase. If the product must place orders, begin with reversible, low-value, tightly bounded actions and require explicit approval before any transaction whose value, recipient, contents or terms could change.

A practical owner is usually a product manager paired with the payment or commerce engineer, security owner and customer-support owner. Legal or privacy teams should review the relevant countries, products and transaction types. A product team should not treat this synthetic protocol as legal advice or certification. It is an engineering acceptance aid: it helps you discover whether the system can cross an action boundary without an accountable permission record.

What the search signal can and cannot tell you

The configured Agents-IA Search Console property showed the query “agent e commerce” with two impressions and an average position of 27.5 during 1–8 November 2026, pointing at the group page linked above. This is a small, property-specific observation. It is not an estimate of market demand, a ranking forecast or proof that searchers share one intent. The page itself presents broad shopping-assistant uses; the likely reader task we address is narrower: decide how much authority to grant an agent that can prepare or place orders.

The question is supported by domain evidence, not by a public survey. OWASP describes excessive agency as harm from unexpected, ambiguous or manipulated model outputs being able to invoke powerful extensions; it recommends narrowing tools and permissions, checking authorization downstream, approval for high-impact actions, monitoring and rate limits. A purchase workflow is a natural instance of that domain problem because selecting an item and committing money are different actions. We found no accessible, representative user-comment or support-ticket corpus for this review, so we do not claim to have measured buyer sentiment.

What the baseline sources cover

The existing material usually explains shopping-agent capabilities or lists security principles. The Agents-IA page lists product questions, cart recovery, personalized recommendations and order tracking. That is a useful feature baseline, but capability lists do not answer what must be true immediately before an order is submitted.

Treating checkout as a single “click buy” action is too narrow. The release test must preserve constraints across discovery, comparison and action, then separately verify the order being committed. A consumer survey from the National Restaurant Association reports that preference for digital ordering and payment varies by service type and age, and describes its survey as an online sample of more than 1,000 consumers. This is evidence about restaurant technology preferences, not acceptance of autonomous agents or authority to make purchases. National Restaurant Association, Restaurant Technology Landscape Report 2024 summary.

Wharton Generative AI Labs' August 27, 2026 report, Agentic Shopping Is Complicated and Contingent, tested shopping agents in a simulated product-selection task using image-based product grids and staged information sources. It reports that source order and injected user-memory details could change agent choices. It did not test live purchases, real products or authorization controls. The published method used six frontier models and 200 runs per condition in the main studies; that is a described experimental setup, not a field sample of stores or shoppers. Its relevance here is limited but important: a selection can be influenced by information context, so the evidence shown to the user at approval must be the exact proposed order, not a model's summary of a prior comparison. Wharton Generative AI Labs, technical report.

The EU's consumer guidance says online buyers should receive clear information about the product, total price, delivery costs and rights when things go wrong. It distinguishes professional traders from private sellers on marketplaces. That is a useful checklist for what a buyer must be able to inspect, but rules vary by jurisdiction and transaction. Your Europe, shopping rights. The Court of Justice's Fuhrmann-2 judgment interprets the EU Consumer Rights Directive requirement that an electronic ordering button use an unambiguous formulation indicating an obligation to pay. In the case it held that the wording on the button itself mattered, rather than relying on the overall ordering context. This is a specific judgment about trader-facing obligations in a covered distance-contract setting, not a universal UI specification for every country or every agent. Court of Justice of the EU, Case C‑249/21.

These sources leave a practical gap. They do not hand a product team a reproducible acceptance matrix that tests whether a user-approved order is the same order the backend submits after prices, inventory, seller, shipping, account context or agent memory changes. The protocol below addresses that gap with synthetic cases and a small validator.

The release rule: approval binds to an order snapshot

A confirmation should bind to a canonical snapshot, not a conversation turn. Before asking for approval, freeze a structured representation of the proposed transaction: merchant and seller identity, product identifiers, variant attributes, quantities, currency, item prices, tax, delivery fees, selected options, delivery address reference, payment method reference, total, expiry time and an idempotency key. Render the same fields in the user's review screen and in the server-side transaction request.

If any material field changes after approval, invalidate the approval and ask again. A change is material if it changes what is purchased, who sells it, where it is delivered, how much is charged, what is recurring, when it arrives or what data is disclosed. A refresh that confirms unchanged inventory may be non-material; a replacement seller, new subscription, higher total, substituted size or changed delivery country is not. Teams should define the exact comparison at the transaction service, not leave it to model judgment.

The user may grant a standing preference such as “prefer recyclable packaging” or “buy this refill when stock falls below two.” A preference is not a transaction permit. A policy can authorize a bounded class of actions only if it includes explicit limits—product family, seller set, quantity, maximum total, time window, delivery destination and cancellation path—and the service verifies them on every execution. If the user cannot understand the boundary, the system should ask before acting.

Transaction path and reapproval boundary

flowchart TD A[User request] --> B[Read-only search and comparison] B --> C[Prepare draft cart] C --> D[Server refreshes seller, SKU, stock, price, tax, delivery] D --> E[Render exact order snapshot to user] E --> F{Explicit approval bound to snapshot hash?} F -- No --> G[Stop or ask a clarifying question] F -- Yes --> H[Checkout API checks principal, scope, expiry, hash, limits] H --> I{Snapshot still matches and idempotency key is unused?} I -- No --> J[Invalidate approval; show changed terms] J --> E I -- Yes --> K[Submit once to merchant or payment service] K --> L{Outcome known?} L -- Yes --> M[Record order state and notify user] L -- No --> N[Reconcile same key; do not create a second order]

The 16-case synthetic acceptance matrix

This is an original test plan, not a measured benchmark. It contains 16 designed scenarios: eight should be allowed or safely prepared, and eight should block, invalidate approval or request human confirmation. The scenario count describes the fixture only. It does not represent a population, product pass rate, incident probability or vendor comparison. No live stores, agents, payment systems, users or model outputs were tested.

Use the table as a test suite, not as a claim that passing 16 cases proves safety. Add product-specific cases for subscriptions, split shipments, preorders, age-restricted goods, regulated products, gift cards, digital content, international destinations, customs, accessibility, discounts and marketplace seller changes. Remove cases that cannot occur in your product, but record why. A case marked not applicable should have a reason and owner; silently omitting it hides scope.

How to define the action boundary

Separate tools by effect. A catalogue search tool should have read-only credentials. A cart-preparation tool may create or edit a draft, but it should not create a payment authorization. A checkout tool should accept only a validated, unexpired approval token and a frozen order snapshot. A cancellation or refund tool should have separate scopes, policies and audit records. Avoid a generic “execute commerce action” function whose parameters and authority are decided by the model.

OWASP's guidance specifically recommends minimizing extension functionality and permissions, using the user's context, requiring human approval for high-impact actions, and enforcing authorization in downstream systems rather than trusting a language-model decision. It also warns that prompt injection can come from indirect content and that prompt hardening does not eliminate the vulnerability. In a shopping flow, untrusted inputs include seller descriptions, reviews, Q&A, web pages, coupon text, shipping notes, image OCR and imported preferences. The safe response is architectural: these sources may inform a recommendation, but they cannot grant a payment capability or alter server policy. OWASP Excessive Agency and OWASP Prompt Injection. The European Commission describes the AI Act as a risk-based framework for specified AI uses. It is relevant to system governance and transparency obligations where they apply, but it does not itself provide a generic authorization protocol for every shopping checkout. Teams must map the actual system, role and use case to the Regulation rather than treating a broad AI Act overview as a purchase-approval rule. European Commission, AI Act overview.

An authorization decision belongs to a trusted service that knows who the principal is, what they approved, how much authority they delegated and whether that authority remains valid. Do not accept “the user said yes” as a free-form string extracted from a long transcript. Store the approval event with a stable identity, scoped order hash, timestamp, expiry, channel, and explicit action. A model can present the approval screen; a server-side control must decide whether the next request is permitted.

The checkout API should reject a token when the order hash, user, merchant, currency, amount or expiry differs. It should not rely on the frontend to preserve the user-visible total. Verify authorization immediately before the irreversible call, and pass the same immutable snapshot to the payment and order services. If downstream systems have independent price or stock state, ask them to revalidate and return a new snapshot; do not silently mutate the approved one.

The approval screen is part of the control

A confirmation is meaningful only if it gives the user enough information to understand what will happen. Show the exact product and variant, quantity, seller identity, tax, delivery, optional charges, recurring terms, delivery timing, destination and total in the currency that will be charged. For marketplace purchases, indicate whether the seller is a trader or private individual when applicable; the EU says that status affects which consumer protections apply. Do not hide a substitute behind a generic “best option” label.

Place the confirmation at the moment when the user can still stop the action. In the EU distance-contract context, Your Europe describes required contract information, total costs and delivery terms before purchase. The Fuhrmann-2 decision emphasizes unambiguous order-button wording that communicates the payment obligation for covered transactions. An agent interface must not turn approval into a vague conversational “sounds good” when the backend interprets it as an order; the user must know what action and payment consequence they are authorizing.

Use action-specific language. “Add to cart” should not submit payment. “Save this basket” should not create an order. “Place order for EUR 84.20” should bind to the reviewed snapshot and make the payment consequence clear. When the legal or product context requires a particular formulation, validate it with counsel for the relevant jurisdiction. The article does not translate the judgment into one mandated label for every country.

A runnable snapshot validator

The following Node.js example demonstrates the core invariant: an approval can be used only for the same unexpired snapshot, by the same user, with a nonempty idempotency key and an explicit approval record. It is deliberately small. It does not process cards, authenticate users, verify signatures, calculate tax, validate consumer-law compliance, or replace server authorization. In production, the approval token must be cryptographically protected and issued by a trusted backend after the review screen is rendered.

Save as checkout-gate.mjs, run with Node.js 20 or newer, and execute node checkout-gate.mjs. It prints a refusal when the amount changes after approval. The fixture uses integer cents to avoid floating-point currency arithmetic.

import { createHash } from "node:crypto"; const order = { userId: "user-17", merchantId: "merchant-4", currency: "EUR", items: [{ sku: "mug-blue-12oz", quantity: 2, unitPriceCents: 1800 }], taxCents: 720, deliveryCents: 490, destinationRef: "address-3", totalCents: 4810, expiresAt: Date.now() + 60_000, idempotencyKey: "order-17-2026-11-13-001", }; function snapshotHash(value) { const stable = JSON.stringify({ userId: value.userId, merchantId: value.merchantId, currency: value.currency, items: [...value.items].sort((a, b) => a.sku.localeCompare(b.sku)), taxCents: value.taxCents, deliveryCents: value.deliveryCents, destinationRef: value.destinationRef, totalCents: value.totalCents, }); return createHash("sha256").update(stable).digest("hex"); } const approval = { userId: order.userId, action: "place_order", orderHash: snapshotHash(order), approvedAt: Date.now(), expiresAt: order.expiresAt, idempotencyKey: order.idempotencyKey, }; function maySubmit(currentOrder, permit, now = Date.now()) { return Boolean( permit && permit.action === "place_order" && permit.userId === currentOrder.userId && permit.orderHash === snapshotHash(currentOrder) && permit.expiresAt > now && permit.idempotencyKey === currentOrder.idempotencyKey ); } if (!maySubmit(order, approval)) throw new Error("Expected exact snapshot to pass"); const changedOrder = { ...order, totalCents: order.totalCents + 500 }; if (maySubmit(changedOrder, approval)) { throw new Error("Unsafe: changed amount reused the old approval"); } console.log("PASS: changed total invalidates prior approval"); 

This example checks that the old permit does not authorize a changed total. It intentionally does not show a payment call. That omission is a safety feature: the correct next step for a production service is to attach the validated permit to the payment service's own idempotent transaction, not to let this demonstration script contact a merchant or charge a card.

A robust implementation should canonicalize every order field in a documented format. Sorting items is not enough if objects can contain unstable key ordering, if two identical line items can be reordered, or if quantities and prices are represented inconsistently. Use a schema with integer minor units, explicit currency, normalized identifiers, fixed units, and a version field. Hash the canonical representation after server-side validation. Reject unknown fields rather than silently ignoring them; a later API version may add an auto-renew flag or data-sharing option that changes material meaning.

The backend should issue a fresh, signed approval token that references the hash, user identity, operation, expiry, and idempotency key. The browser-local boolean is not enough. The checkout endpoint must authenticate the caller, authorize the order scope, verify the signature, compare the current snapshot, check inventory and price conditions, enforce the idempotency key, and only then call a payment provider. The payment provider and merchant order service should not infer approval from an LLM message. On a timeout, query the existing operation by key before creating another one.

A two-queue test workflow

Run the matrix in a staging environment with clearly synthetic customers, products, addresses, currencies, payment instruments and seller pages. Keep a set of allowed scenarios and a separate set of refusal scenarios. Teams often test only happy paths because those are easiest to demo. A release suite should spend at least equal attention on refusals, changed context, interrupted network calls and policy boundaries. In this article's designed fixture, eight cases explore permitted preparation or bounded execution and eight examine a block, invalidation or escalation. That split is a test design choice, not an empirical ratio or a recommended estimate of production risk.

For each case, define: initial state; principal and permission; user intent; trusted and untrusted inputs; expected tool calls; prohibited tool calls; expected order snapshot; expected user-visible message; expected audit event; rollback or reconciliation step; and a pass criterion. If the agent gives a safe explanation but the payment API still executes, the case fails. If the API refuses the payment but the UI tells the buyer the order is complete, the case also fails. Test the model, the orchestration layer, the permission service, the browser or API adapter, and the commerce backend as one end-to-end chain.

Capture trace events with a correlation identifier, but avoid placing full payment credentials, secret tokens or unnecessary personal data in logs. The useful record is the principal, action, policy version, snapshot hash, decision, downstream result, idempotency key and timestamps. Where laws or company policy require data minimization or retention limits, apply them to test logs too. Logs should make a denial and a successful authorization distinguishable without copying the entire conversation.

For every negative case, assert the absence of the side effect, not just the presence of a warning. Check the order database, payment provider's test ledger, email or SMS queue, inventory reservation and downstream webhook. A screenshot that shows a refusal is not enough if an order was still created by a background retry. Use isolated fixtures and deterministic stub services so tests do not contact real vendors or payment rails.

Release stages and stop conditions

Start with read-only product discovery. Measure whether the agent respects user constraints and gives traceable sources, but do not grant a cart write scope. Next, allow draft-cart preparation with no payment capability. Then allow an approval-bound single transaction in staging. Only after the exact snapshot, mutation and replay cases pass should a production rollout be considered. Any percentage ramp should be based on your own reviewed results, not borrowed from this article; the designed 16 scenarios do not estimate reliability.

Set a kill switch outside the model. It should disable order submission at the API or credential layer even if the agent loop continues. Use bounded transaction values and rate limits. Alert when there are repeated denials, snapshot mismatches, unexpected retries, seller identity changes or unexplained action bursts. Rate limits reduce the number of harmful actions that can happen before detection; OWASP describes rate limiting and activity logging as damage-limiting controls for excessive agency. They do not replace authorization.

Define rollback for each side effect. A draft cart can usually be deleted. An order may be cancellable only until fulfillment starts. A payment authorization may be reversible; a captured payment and shipped item may require a refund process. A package can leave a warehouse before a cancellation reaches the merchant. Therefore “we can undo it” is not a sufficient release criterion. Record the last safe cancellation point and route cases beyond it to a human.

Do not silently retry a purchase after an ambiguous timeout. The client may have lost the response after the merchant created the order. A fresh create call can create a duplicate. Reconcile by the same idempotency key, obtain the merchant's authoritative order state, and present uncertainty to the user if the system cannot determine the result. “Unknown” is a valid state. Do not turn it into “failed, so try again.”

What the legal sources mean—and do not mean

The sources cited here support specific propositions, not a general conclusion that every agent-mediated purchase is lawful or compliant. Your Europe explains consumer information and protections for transactions within its stated scope, including price, delivery, seller status and certain remedies. The Court's Fuhrmann-2 ruling concerns the interpretation of a particular EU Directive provision for electronic ordering buttons. National implementation, product categories, trader status, consumer or business status, cross-border details and exceptions can change the analysis. A company should get jurisdiction-specific review before relying on a UI pattern.

A consulting outlook from Deloitte frames agentic commerce as a possible new retail channel and recommends retailers prepare for agent-mediated discovery and purchase. It is a strategic, forward-looking business perspective, not an independent safety evaluation, consumer-law source or proof that autonomous checkout is reliable. Its commercial lens is useful as a counterweight: firms may feel pressure to automate quickly, which makes explicit release gates more important, not less. Deloitte, Agentic commerce.

A consulting outlook from Deloitte frames agentic commerce as a possible new retail channel and recommends retailers prepare for agent-mediated discovery and purchase. It is a strategic, forward-looking business perspective, not an independent safety evaluation, consumer-law source or proof that autonomous checkout is reliable. Its commercial lens is useful as a counterweight: firms may feel pressure to automate quickly, which makes explicit release gates more important, not less. Deloitte, Agentic commerce.

The EU consumer-rights page lists exceptions to withdrawal rights, including certain date-specific services, personalised goods and urgent repairs. That does not mean every such purchase has no other rights, and it does not determine whether an AI agent may agree to a term for a user. It means a simplistic “you can always return it” message is unsafe. The agent should identify the relevant merchant terms and product class and avoid giving a legal guarantee unless the system has authoritative, current evidence.

A product owner may conclude that human approval is too slow for every transaction. That is a real trade-off. The alternative is not an unbounded model call; it is a clearly bounded delegated mandate enforced outside the model. A user could authorize auto-reordering a specified household consumable from a specified merchant up to a maximum total and to a saved address. The system must still check the SKU, quantity, total, seller, destination, expiry, account and cancellation conditions on every run. Any change outside those bounds returns to a human.

A stronger confirmation can also add friction, reduce conversion, or be inaccessible if it relies on a visual-only approval. Balance friction against consequence, but keep the essential information available in accessible form and preserve a clear no-action path. Approval should not be bundled with marketing consent or optional charges. Where the intended workflow is only recommendation, do not collect a payment scope “for future convenience.”

Failure modes that the demo usually misses

Prompt injection inside the product page. A seller description includes text directed at the agent, asking it to ignore its instructions, select a premium bundle, or expose user details. The source is data, not authority. The tool service must still enforce a narrow scope. OWASP's prompt-injection guidance says retrieved and user-provided content can alter model behavior and that mitigations reduce risk rather than fully eliminate the class. OWASP, LLM01:2025 Prompt Injection.

Approval laundering through paraphrase. The user says “that one looks good” while discussing an option, and the model interprets it as checkout permission. Bind approval to an explicit action and snapshot, not conversational sentiment. If the product is uncertain, show the order summary and ask.

A stale total. Tax, currency conversion, delivery or a seller-side discount changes after the user sees a summary. A model statement such as “price unchanged” is not a server verification. Recalculate at the transaction boundary and invalidate prior approval if the total or material order data differs. EU guidance says the total price and additional charges must be clearly communicated in covered online purchases; there can be scope and national-law details, so validate the relevant route. Your Europe, pricing and payments.

A hidden optional extra. The agent selects express shipping, warranty, gift wrap, recurring replenishment or a donation because a UI default is preselected. Treat optional paid choices as separate opt-ins and include them in the frozen total. Do not infer consent from silence or a broad “buy it” message.

Wrong person, wrong account. A shared household browser contains another person's address, card token or saved preference. Session authentication does not automatically grant permission to use every resource visible in context. Scope the order permit to the authenticated principal and authorized payment reference; test cross-account access denial.

Substitution drift. The requested product is unavailable, so the model picks a “similar” variant whose size, color, brand or seller differs. Similarity is a recommendation, not consent. The agent can present a substitute, but a material change invalidates approval. Explicit compatibility rules can automate some substitutions only if the user approved the precise boundaries.

Duplicate order after a timeout. A network timeout leaves the agent uncertain whether the purchase succeeded. A retry without idempotency can create another order. Reconcile the same transaction identifier and expose the pending state rather than repeating the side effect.

The refusal is cosmetic. The assistant says “I cannot buy that” but its browser tool or backend has already submitted the order. Test external side effects and system state, not only generated text. Denial must be enforced at the API.

An ineligible return is promised. A user asks “can I return this?” The agent responds with a generic 14-day answer even though the item or service may fall within an exception, the seller is a private individual, or the jurisdiction differs. The EU page itself lists exceptions; show merchant and product-specific policy and route uncertainty to a person.

A changed recurring term. A one-time purchase becomes a subscription, renewal or membership. The user-approved price hash is not enough if term and cadence are outside the snapshot. Include recurrence, cancellation process and next charge in the displayed confirmation and in the hash.

Risk scoring replaces hard rules. A model estimates the purchase is “low risk” and bypasses the authorization check. Risk scores can prioritize review, but they cannot confer payment permission. Keep hard policy gates deterministic and downstream.

A worked design review

Imagine a customer asks: “Find a quiet desk fan under EUR 80, deliver it by Friday, and buy it if it fits.” The phrase “buy it if it fits” is incomplete. “Fits” could mean size, noise, plug type, seller reputation, exact delivery date, total price or all of them. The agent should ask a clarifying question or present candidate products; it should not treat the condition as a valid payment mandate.

A safer dialog separates preference discovery from authorization. The agent returns two products and identifies the dimensions, noise measurements, seller and current delivery estimate it could verify. The customer selects one. The agent prepares a cart, obtains a fresh total, and displays item, quantity, seller identity, tax, delivery, destination, total and cancellation or return limitations. The customer selects “Place order for EUR 74.60.” The server issues a permit tied to that snapshot. If the merchant refresh changes total to EUR 82.10, the old permit fails and the agent must ask again.

Suppose the customer authorized up to EUR 80 including delivery, but the agent sees a EUR 4.90 expedited shipping option that meets Friday while standard shipping does not. A total of EUR 79.50 is under the numerical cap, yet the optional delivery upgrade was not necessarily authorized. If the customer said “deliver by Friday” and the approval screen shows expedited delivery with its price, the user can choose it explicitly. The service should not choose the paid option merely because the amount remains under the cap. A ceiling is not a blanket consent to every purchasable option.

If the selected merchant is a marketplace, seller status also matters. The EU consumer guidance distinguishes trader and private seller; a model must not infer that the marketplace itself is the seller. If the seller identity is hidden or changes before checkout, pause. The product can still help by comparing offers, but an unknown seller should not be silently substituted at execution.

If a transaction call times out, the workflow moves into an unknown state. The agent checks the order service with the original idempotency key. If an order exists, it reports the order identifier and current status; if no order exists and the key is known to be unused, the service may retry under its deterministic policy. If the system cannot establish either condition, the correct response is “I cannot confirm whether the order was placed; I have stopped retries and need a human or merchant check.” That answer is less convenient than guessing but safer than charging twice.

How to score the cases without inventing a benchmark

Use pass/fail for hard safety invariants rather than a composite “agent safety score.” For each scenario, record whether the intended state was reached, whether any prohibited side effect occurred, whether the user saw the correct information, whether the authorization record matched, and whether the audit trail can reconstruct the decision. A case passes only when all required assertions pass. A compelling answer with a payment bypass is a failure, not a partial success.

You can add a weighted usability score for non-hard-gate details such as clarity or response time, but keep it separate from safety. Do not average away a payment authorization failure with a friendly explanation. Report the numerator and denominator for each category, the exact fixture version, product build, model and prompt version, tool configuration, date, environment and exclusions. If you test 16 synthetic scenarios and 15 pass, state “15/16 designed scenarios passed in this fixture,” not “the agent is 94% safe.” The denominator is a constructed case set, not a random sample of all possible transactions.

When the test fails, preserve the prompt, browser state, tool arguments, policy decision, backend response, order state and relevant timestamps. Redact secrets and personal data. Re-run the same case after a fix and test nearby mutations: change the seller, total, currency, shipping option, user identity, expiry and idempotency key. A patch that fixes one exact string but fails on an equivalent change is not a boundary.

Where the recommendation may fail

A blanket “always require a click” rule can frustrate users and make routine low-risk replenishment unusable. A carefully bounded standing authorization may be appropriate where the user has explicitly selected product, seller, quantity, price limit, destination and schedule, and the backend checks each limit. The approval requirement should scale with authority and consequence, not become a generic obstacle to all assistance.

A staged system may also be unsuitable for products with urgent availability or highly dynamic prices. In that situation, decide whether the product should simply show a live offer and route the user to the merchant's own checkout. Do not weaken the snapshot invariant because a price expires quickly. Show the expiry and ask for confirmation on the refreshed terms.

A model can also make incorrect product comparisons without any payment access. Read-only access reduces the potential harm but not the risk of misleading recommendation. Provide citations to product attributes, expose uncertainty, avoid claiming compatibility without evidence, and let users inspect source information. The Wharton report illustrates in a simulated shopping choice that source context can influence selection; it does not establish what will happen in your product. Treat it as a reason to test exposure and ordering, not as a direct performance forecast.

Finally, the legal sources are EU-focused....