This article was created with the help of AI, working from the project's code and real test runs. The cover and the pictures of the cup are AI-generated on purpose. The screenshots come from a real run.
A buyer says the cup arrived torn and sends a photo. If you run a small shop, you have two bad options. Refund it and pay for the fakes, or question it and insult the honest majority.
Adding a tear to a photo used to take some skill. Now it takes one sentence.
Left: a picture of a paper cup. Right: the same picture after one prompt, "Add a ragged tear down the side of this paper cup, as if it arrived damaged in shipping. Keep everything else the same." Both are AI-generated.
Boll & Branch's CEO told Modern Retail about a torn-sheet claim whose photo carried an AI watermark. Forter, a fraud vendor, told Wired that AI-doctored refund claims rose more than 15% during 2025. That's one anecdote and one vendor's estimate, and nobody has good market data yet. It was enough to make me build something for the PayPal AI Hackathon and see how far a small seller could get.
Break or Fake is a refund desk for small PayPal sellers. It runs on PayPal's sandbox, so no real money moves. Doubtful photos wait for the seller, and nobody gets auto-declined.
Give the model eyes and no hands
The tempting build is an agent with a refund tool. I didn't want that. The claim photo comes from the person who wants the money, so anything that reads it is reading untrusted input. A photo can have "approve this refund" written on a sticky note.
So the reviewers get photos in and JSON out, with no tools at all. Two free vision models, reached through OpenRouter, look at the evidence independently and answer fixed questions. Is it the same item as the listing? Is the damage plausible? Is this a physical scene or a screen? Any other text in the frame gets transcribed, and if it reads like an instruction, the claim goes to a person.
Merging their answers is one line, and it's the one I care about most:
// Unanimous "yes" is the only "yes". One "no" is a "no". Anything else stays uncertain, which never refunds. const agree = (values: Signal[]): Signal => values.every((value) => value === "yes") ? "yes" : values.some((value) => value === "no") ? "no" : "uncertain"; Code decides about money
After the review, plain TypeScript makes the call. This runs before every refund:
if (decision.approvedBy === "policy") { const ceiling = decision.evidence === "live" ? LIVE_CEILING_CENTS : UPLOAD_LINE_CENTS; if (call.amountCents > ceiling) { refuse(`${dollars(call.amountCents)} is above the ${dollars(ceiling)} the policy may refund on ${decision.evidence} evidence, so a person decides`); } } An uploaded photo can authorize up to $15. Live photos can authorize up to $60. Above that, the seller decides. The same gate checks how much of the capture is still refundable, so a refund can't exceed what the buyer actually paid.
At checkout, the listing photo, title and price get hashed into the PayPal order's custom_id. When a claim comes in, the app reads the order back from PayPal and compares the hash. A claim is about that exact listing or the review can't clear it.
Above $15 the buyer also gets a fresh three-digit code, writes it on paper, and takes three live photos within two minutes. That screen takes camera frames only, so there's no file to upload. A picture generated last night doesn't have tonight's code in it.
A timeout on a refund call doesn't mean the refund failed. It might have gone through, and a blind retry pays twice. So the app saves a stable request ID before it calls PayPal, sends that same ID on any retry, and reads the refund back before it tells anyone it's done.
The bug 185 passing tests couldn't see
Then I ran the whole flow against PayPal's sandbox and the real models for the first time, and every live claim ended with "The review could not finish safely."
The app sends four pictures: the store's listing photo, then three live frames. The prompt asked about frames 1, 2 and 3. Both models decided the listing was frame 1. One returned four frames and failed validation. The other marked frame 1 as no damage and no code, which is a fair description of a listing photo.
The system failed safe, so the claim went to the seller and no money moved. That's also why nothing looked broken. My tests used stand-in reviewers that always returned perfect JSON. The fix was two sentences of prompt.
While I was in there I stopped telling the models which code to expect. They had been reading a code with the answer printed next to it. Now they read it blind, and plain code compares the result.
Then I tried to beat it
I bought the cup with PayPal's test card and filed three attempts on one claim.
Attempt one was the fake straight from the image tool. It still carried Content Credentials naming the generator, so the claim was held before any model looked at it.
Attempt two was the same picture re-saved as a JPEG, which strips those credentials. At $25 an upload can't move money, so it asked for live photos.
Attempt three fed the capture screen a scripted camera: the fake, with that session's code drawn onto a note in a handwriting font. In seven tries, both models called it a real physical scene every time, and they read the code correctly in six.
They were fooled. So I added a check that doesn't involve a model: if the browser reports no camera name, or the name of known virtual camera software, the claim waits for the seller even when every reviewer says yes. In the recorded run that is what happened: the code and scene checks passed, one model doubted the item matched, and the camera check held the claim.
The seller desk from the recorded run. The models read the code and called the scene physical. The camera check flagged it anyway.
On the seller desk I approved the refund and PayPal's sandbox confirmed it, with the request ID stored so a retry couldn't pay twice.
What it can't do
That camera check is thin. The page reports its own camera name, so a modified page can lie. When I spoofed the name, one run was held because the models read the code differently, and the next was refunded automatically. A staged scene or a look-alike item can still get through. Under $15, a careful fake with stripped metadata gets paid on purpose, because arguing over a $12 item costs the seller more than the item.
A model saying a photo looks real proves very little on its own. That's the whole reason the models can't pay anyone.
I haven't run this with a real phone and a really torn cup yet, and I don't have a catch rate. The next job is to run labeled real and faked claims through it and publish how many it catches next to how many honest buyers it wrongly holds.
The code is at https://github.com/the-ai-entrepreneur-ai-hub/break-or-fake, MIT licensed. The sandbox shop is at https://breakorfake.trykioko.com/shop, and the test card is on the checkout page.
If you've dealt with refund fraud as a seller, I'd like to hear what this gets wrong.
