November 12, 2026 · Shawn Kelley, CEO
When I ran payments and risk at SeatGeek, we had a chargeback vendor who would always tell us, "Our win rates were in line with the industry norm" on all our quarterly reviews.
I kept asking for one thing and I got pretty blunt about it:
If we're at the norm, help us get better than that. Even just 2% better. Help me segment down and find something my team can focus on and improve. If the fix turned out to be a better template for one segment, fine, let's build that... but even pushing as hard as I did, I never got it.
Knowing what I know now, we weren't crushing it at all.
There was a lot of room to do better, and at that point all our responses were template-based. What I got instead was a benchmark, which ended the discussion.
A norm is an average-based comparison. It only tells you something if you know what's on the other side of it, and in my QBRs I never did.
Whose norm is it? A vendor's benchmark often comes from its own book of merchants, or from an industry survey instead. If the merchants behind it are mostly in a different vertical, with different products and different fraud, its average says very little about you. Unless the vendor shows you who's in it, you can't check.
Which win rate is it? There are several ways to calculate a win rate, and they give very different numbers for the same quarter. One counts only the disputes you fought and got a final answer on. Another counts every dispute that came in, including the ones nobody worked.
I went through how win rate gets defined in a separate post. If your number and the benchmark use different definitions, the comparison doesn't hold, however close they look.
Which reason codes, and which issuers? A blended rate averages your easy wins with your hardest losses. You can sit right at the norm overall while one reason code loses again and again, and the blended number will never show you which one.
How much data is behind it? Ask how many disputes sit behind both numbers, and over what period. On a low-volume reason code or a single issuer, being a point or two above or below the norm can be noise.
Does it count dollars? Most win rates count cases. Two merchants at the same rate can recover very different amounts, at very different cost, depending on which cases got fought.
And a norm is a snapshot. It won't tell you whether you're getting better quarter to quarter, or how many of these disputes you could have stopped before they came in.
A good review starts with where you're losing.
It shows you the segments that are losing, whether that's a reason code, a product line or a single issuer. It explains why each one is losing. And it tells you what to change upstream, wherever the disputes start, so fewer of those disputes come in next quarter.
What I wanted from that vendor was data to take back to my product team. My team was measured on our own metrics, and anything a vendor did to help us improve them mattered more to me than where we sat against an average.
You don't need a new vendor to start asking these. Bring them to the next review and see which ones get a real answer.
Which definition of win rate is in this report, and which one does the benchmark use? Ask for both to be written on the slide. If they don't match, the comparison can't tell you anything.
Which reason codes are we losing, and why? Ask for the losses broken out by reason code, with the vendor's explanation for each.
What did you change last quarter because of our losses? Not every loss comes with a reason. Processors don't always pass along the issuer's notes, and issuers don't always explain themselves. But across a quarter of losses, patterns show up: the same missing evidence, the same procedural miss. If nothing changed in how the responses were built, nobody is learning from them.
Which disputes did you choose not to fight, and how much were they worth? A strong win rate on a narrow set of cases can sit beside a lot of dollars that were never contested. Ask for the count and the dollar total, and the reasoning behind the cutoff.
Where are we weakest, and what would it take to improve that one segment? Even 1% better on one segment is worth asking for. You're asking for one segment, one change and a way to measure whether it worked.
What should our product team fix? Ask which product or checkout flow drives the most disputes. A vendor can only answer this if it sees more than the dispute file: your orders, fulfillment and support history. If yours doesn't, ask what it would take to connect them.
What should our support team do differently? Some disputes start as a customer who tried to reach you first. Ask which ones look like that, and what support could have done to settle them before they turned into chargebacks.
What did you commit to last quarter, and did it happen? Bring the list from the last review. If the same request keeps landing on a roadmap and never comes off it, that tells you as much as any win rate.
If the answers are thin, that's useful too. It tells you how much of the work your vendor is actually doing, and how much of it is reporting.
Responding to chargebacks is the easy part. The value is in understanding why they happen, so you can make improvements to your product and your customer support, or whatever is actually driving them.
That only happens if the data leaves the dispute team.
A cluster of losses in one reason code is a place to start looking. The cause might sit in product or support, or in fulfillment, fraud rules, billing or the refund policy, and the people who own those are rarely in the QBR. A short list of fixes gives them something to do before next quarter.
That's the work we built Disputed around: tying each dispute back to the order, the fulfillment and the support history behind it, so the review ends on a list of fixes.
At SeatGeek I would have taken 1% better and a list of things to fix. What I got every quarter was word that we were doing fine.