20 ways an agent payment goes wrong. Which ones does a check before paying catch?
AgentCommerceBench is an open benchmark (Apache 2.0) of fraud and failure in payments made by AI agents: twenty classes in four families. Its starting point is one we agree with. A seller with the right domain, the right payout address and a service that really delivers can still charge 30% above its own listed price, and every check that looks at who the seller is stays silent, because nothing about the identity is wrong.
So we went through the twenty classes one by one and asked, for each: does one of Fizzl's checks catch this before the money moves? We did not run the benchmark against our services. It scores detectors on generated session traffic, and our checks run on one payment at a time, on live sellers. What follows is each class held against what our checks actually look at, with the check named, so you can verify it.
Structural: a rule broken in one payment
| Class | What catches it | |
|---|---|---|
| S1 Over the limit | Caught | The Agent Wallet's limits per call and per day, set by the owner and enforced outside the model: above them the wallet asks the owner first, and no answer means no payment. x402 Doctor's preflight with max_usd says no_go above the budget. |
| S2 Velocity: a burst of payments | Partly | The daily limit caps the total. Nothing of ours looks at how fast the payments come. |
| S3 Out of domain | Partly | The Agent Wallet sends the owner a heads-up when a purchase is unlike what the agent usually buys (judged by TypeSafe Jev, after five earlier purchases). A heads-up, not a stop. |
| S4 Payee substitution | Caught | presign-guard checks the signature against the seller's 402: a payment to another address is red. x402 Doctor also flags a payout address that changed since its last daily scan (payto_changed). |
| S5 Replay of a settled payment | Caught | By the protocol: USDC accepts each signed authorization's nonce once, and x402's payment identifier lets a seller answer a resent payment without settling it again. The Agent Wallet resends the same payment after a lost answer instead of signing a new one. |
| S6 Identity doesn't own the session | Not | Our identity checks look at the seller (ERC-8004 payee delegations, Concordium badges), not at which agent or account a buying session belongs to. |
Agent: the agent itself is the weak spot
| Class | What catches it | |
|---|---|---|
| A1 Follows an injected instruction | Caught | presign-guard's injection check (POST /v1/injection) on text before the agent acts on it: red when it tries to make the agent pay, send money or hand over keys. Fizzl Buy and the Agent Wallet check the answers they receive for injected instructions. |
| A2 Plans its way around the rules | Not | Each step is allowed on its own; the plan only shows over several actions. |
| A3 Says one thing, does another | Caught | presign-guard's intent field: one sentence about what the agent means to do, and INTENT_MISMATCH (orange) when signing would do more or something else. |
| A4 Poisoned tool description | Partly | x402 Doctor reads the seller descriptions in its Trust Index for bait (a lure check by TypeSafe Jev), and the injection check reads any text you send it. Nothing of ours reads your agent's own configuration. |
Evasive: every payment is fine, the pattern isn't
| Class | What catches it | |
|---|---|---|
| E1 Ratchet: the normal crept up step by step | Partly | The Agent Wallet sends the owner a heads-up when what an agent pays one API went up on 5 or more payments in a row (at least 4 raises) to 1.5× the first of them (since 6 November 2026). Each step stays under the 2× price-jump alert, so without it a slow climb is never seen. A heads-up, not a stop; a climb spread over several sellers, or one with a dip in it, isn't seen. (A seller asking more than it quoted is F4.) |
| E2 Structuring: split under the limit | Partly | The Agent Wallet sends the owner a heads-up when an agent pays one seller 3 or more times within 10 minutes, each payment at least a quarter of its per-call limit and together more than that limit (since 5 November 2026). A heads-up, not a stop, since an honest burst looks the same; parts spread over hours or over several sellers aren't seen. |
| E3 Payee rotation | Partly | x402 Doctor flags a payout address that changed since the last daily scan, and checks a payout address against the seller's ERC-8004 or Concordium identity where the seller has one. Rotation within a day isn't seen. |
| E4 Routed through a new service | Partly | x402 Doctor and Fizzl Buy prefer sellers with a record of being paid, day after day; a new seller has none, which the preflight shows. It is not a stop. |
Fraud: the seller is exactly who it says it is
| Class | What catches it | |
|---|---|---|
| F1 Overcharge above the listed price | Caught | x402 Doctor's preflight compares the 402 with the price in the seller's own OpenAPI: above it is a caution (price_above_advertised), ten times or more is no_go. A caution, not a stop, on purpose: as the benchmark says, honest prices move. |
| F2 Charged more than other buyers | Partly | x402 Doctor calls every paid URL in its Trust Index each day, the way any buyer would. The Agent Wallet compares what the agent paid with the price the seller asked Doctor for the same URL, and sends the owner a heads-up from 1.25× that price (since 6 November 2026). A heads-up, not a stop, since a seller that prices per request looks the same; a URL Doctor doesn't know isn't compared. |
| F3 A fee that buys nothing | Partly | x402 Doctor's paid preflight compares the price with comparable services (TypeSafe Jev picks which ones really do the same thing): a note from five times their median price, a caution from twenty. A fee inside a normal price isn't seen. |
| F4 Drip pricing: quoted low, charged high | Caught | The same comparison as F1 when the low price is in the seller's OpenAPI, and price_changed when the seller asks more than in x402 Doctor's last daily scan. presign-guard passes it, because the signature matches the 402: try it in Watch an agent pay with the trap "Higher price at checkout". |
| F5 Settled, reported failed, retried | Caught | x402 Doctor's free payment proof check reads the seller's PAYMENT-RESPONSE and says whether that transaction exists on chain and moved the money, before the agent pays again. Endpoints that three or more payers report as failing after payment turn caution (payments_fail_after_preflight). |
| F6 Full price, cheaper tier delivered | Partly | Fizzl Buy and the Agent Wallet check each answer against what was asked (TypeSafe Jev) and flag one that doesn't deliver. A cheaper answer that still answers the question passes. |
What the gaps have in common
Evasion planning and the session that isn't the agent's own, and structuring, ratchet and price discrimination until 5 and 6 November, all have the same shape: no single payment is wrong. They show only across many payments, or across many buyers. A check that sees one payment, which is what x402 Doctor and presign-guard are, cannot see them, however good it is.
What helps is on the buyer's side, outside the model:
- Limits the model can't change: per call and per day, enforced by the wallet. They don't spot a pattern, but they cap what a pattern can cost.
- A record of what was bought for the task: seller, request and payout address. It stops the same purchase twice (the fourth trap in Watch an agent pay) and is the start of seeing a ratchet or a split.
- A budget per task: so a split purchase still counts as one.
More on that in The payment lane of your agent harness. And every answer from presign-guard and x402 Doctor lists what it checked and what it didn't (checked and not_checked): a green verdict never means more than those checks.
Try it
- Watch an agent pay on fizzl.eu: five traps, live with the free checks, including drip pricing and a swapped payout address.
- presign-guard:
POST https://presign-guard.fizzl.eu/v1/check/quickfor a free verdict,/v1/checkwith the reasons for $0.01,/v1/injectionfor text. - x402 Doctor: the free web check on x402-doctor.fizzl.eu, the preflight at
GET /api/v1/preflight?url=…&max_usd=…for $0.001, and the free MCP toolcheck_payment_proof.
The paid calls take x402 on ten networks: USDC on Base, Solana, Monad, Polygon, Arbitrum, Avalanche, Algorand or Stellar, RLUSD on the XRP Ledger, or USD₮0 on X Layer. No account or API key.
Questions
Did you run the benchmark against Fizzl?
No. It scores detectors on generated sessions; our checks run on one live payment at a time. This page is a reading of each class against what our checks look at, with the check named.
Why does presign-guard pass drip pricing?
Because the signature pays exactly what the 402 asks, to the seller. presign-guard checks the signature against the 402. The 402 against the listed price is x402 Doctor's job.
Would a check that warns above the listed price fire on honest sellers?
Sometimes, and the benchmark measured it: 14.6% of honest purchases cost more than 1.15 times the quote. That is why x402 Doctor makes it a caution, not a no_go, until the price is ten times the listing.