Method

The pre-registered method for run 001, rendered from the committed file.

It was written before there was a result to fit it to. The method was first committed at 931d7af on Sep 7, 2026, 01:46 UTC. The first mainnet payment of run 001 was made on Sep 15, 2026, 06:19 UTC — eight days later. A standard fixed after the numbers are in is not a standard, so the ordering is the point, and it is checkable rather than asserted.

Every amendment is in the history. It has been amended 9 times since, each commit prefixed amend:, and the file's history is the complete list. None of them is a silent edit — among them are one recording that the first attempt at the run was void rather than quietly re-running it, and one narrowing a coverage claim from five networks to four. Followability is the whole claim this project makes, so the diffs are the evidence for it and not a footnote to it.

Run 001 is complete. The calls were made on Sep 15, 2026 and the labeling is done. Read the Run 001 findings or inspect the full technical report. The method is here first, and dated, on purpose.

Written 2026-09-06

Written before any mainnet payment, so I can't adjust the standard to fit the result.

The question

Does an endpoint being reachable, compliant and well-graded predict that it is worth paying for?

Everything published about x402 endpoints today is measured from outside: uptime probes, 402-envelope compliance, price stability, on-chain settlement volume. Nothing measures whether the buyer got what they paid for. This run tests whether the outside measures predict the inside one.

Prediction, stated before the run: they will not fully predict it. Some share of endpoints that pass every external check will return responses a buyer cannot use. I expect that share to be non-trivial. If it turns out to be zero, that is the finding and I publish it as such.

Secondary prediction, about the shape of the failures: most of what I find will be mechanically detectable — empty, malformed, insufficient, no_response — and wrong will be rare, or hard to catch. Confidently incorrect answers are the failure I expect to be hardest to surface.

If that holds it is the more interesting result, because small and invisible is a worse problem than large and obvious: a wrong answer is the one that propagates into whatever the agent produces. And note the bias it runs against — my checkability rule deliberately selects for conditions where wrong can be caught, since I only buy where I already know the right answer. A real buyer usually doesn't. So if wrong stays rare even under conditions optimized to detect it, then in production it is effectively undetectable by a single buyer, and only corroboration across buyers could surface it.

Three axes, three batches, one report

I run these as separate batches so that a quality label is never a side effect of testing whether the plumbing works.

Axis 1 — Coverage

Can I verify a payment on the networks carrying the most endpoints?

I rank by endpoint count because it's the number I can cite. Solana leads on daily transactions, though, so this sample skews toward where endpoints are listed rather than where payments actually happen.

Target networks, by listed endpoints (source: x402.fuchss.app/network, read 2026-09-06): Base 38,541 · Solana 1,619 · Base Sepolia 453 · Avalanche 261 · Arbitrum 92 · BNB 14.

Circle publishes no native USDC on BNB Smart Chain. What trades there under that name is Binance-Peg USDC — a different issuer with different redemption. A BNB review is therefore denominated in a different asset from the other four networks, and I don't treat prices across the five as directly comparable. I verify BNB payments the same way regardless, since the asset is an address the submission names.

• Solana: new verifier.
• Avalanche, Arbitrum: EVM, so config only — USDC contract address and RPC per chain. BNB: RPC only, per the note above.
• Deliverable: one verified review on each network, with tx hash. Amended 2026-09-14: four networks, not five. See below — BNB has no endpoint this run can buy from.
• I also normalize the network-name aliases that show up in the wild (solana:mainnet, solana-mainnet, solana-mainnet-beta, base-mainnet, bare polygon / bsc) to CAIP-2 rather than rejecting a payment over spelling.

Amendment, 2026-09-14, at the sample freeze and before any payment: Axis 1 covers four networks. BNB is reported as unreachable under this frame rather than bought.

Freezing the sample turned up something the endpoint count alone didn't show. All 13 BNB Chain endpoints in the catalog belong to a single provider, every one grades B, and every one is DeFi yield, bridge quotes or intents — precisely the categories this run puts out of scope because grading them would be a guess rather than a judgment. There is no BNB endpoint that is both grade A and in a category I can check, and there is no BNB provider to spread across even if I dropped the grade filter.

I could buy one BNB call outside the frame purely to fill the axis. I don't, because of what it would and wouldn't show. It would prove that a chain I had already configured verifies a payment — which the test suite establishes more cheaply — from an endpoint whose category I excluded as ungradeable. The row would then need excluding from Axis 3 anyway, so the only thing bought is a tick in a table, at the cost of a paragraph explaining why the tick doesn't mean what a reader would take it to mean.

The honest version is shorter and says more: BNB's entire presence in this catalog is one provider, grade B, in categories out of scope. That is an observation about the ecosystem, not a gap in the run. A chain with 13 listed endpoints and one operator behind them is not a chain with a supply of things to buy, and a coverage axis that reported a green tick there would be describing my configuration rather than the ecosystem's.

What this does not change: the verifier still supports BNB, still refuses it the same way when an RPC is missing, and the note above about Binance-Peg USDC still holds for anyone who does submit a BNB review. The capability is built and tested. What is absent is something in scope to spend on, and that absence is the finding.

(Counts read 2026-09-06 put BNB at 14 listed endpoints; the 2026-09-14 freeze found 13. Neither number changes the conclusion.)

Axis 2 — Reach

Of the endpoints I select, how many can actually take a payment?

This isn't a re-run of the ecosystem probe study that already exists. It's a check that the same pattern holds on my sample, measured by payment rather than by probe.

• For every endpoint I attempt, I record: reachable, returned a well-formed 402, payment settled, response returned.
• Deliverable: a count. "Of N endpoints attempted, M returned a valid 402, K settled a payment, J returned a response."
• Reference point I cite rather than reproduce: of 72,395 listed endpoints, x402 Trust reports 51% unreachable, 23% of reachable ones non-compliant, and 67.8% of pay-to wallets having ever received a payment.

Axis 3 — Quality

Of the endpoints that are up, compliant and well-graded, how many returned something I could use?

This is the finding. Everything above exists to make it credible.

• Sample frame: x402 Trust's grade A endpoints, sampled across distinct providers, filtered to categories where I can check the answer. Sampling their top grade is deliberate: I want to test the best case, not a random sample.
• I do not use their top-25 leaderboard, even though it is the obvious frame. Every one of the visible top ten is a proxy route on a single host, so a sample drawn from it would measure one provider rather than whether a grade predicts quality.
• The obvious version of this question is already answered and I don't re-ask it. x402 Trust publishes the grade distribution — 36.4% of monitored endpoints grade F, 16.3% grade A — so "most endpoints are bad" is public knowledge. The open question is the narrow one: of the endpoints that grade A, how many deliver.
• Deliverable: labeled outcomes with tx hashes, published, re-verifiable by anyone.
• Exclusion: reviews where the payment could only be confirmed as recipient credited rather than payment traced are excluded from the quality findings, because the payer isn't attributable on that path. They still appear in the published dataset, marked, and they still count for Axis 1 and Axis 2. This path only occurs under node indexing lag, so I expect it to be rare; if it isn't, that itself goes in the report.

The labeling rule (written once, applied to every call)

There are two outcomes, not four:

1. The response gave me what I wrote down before I paid.
2. It didn't.

Everything else — whether I retried, went elsewhere, or gave up — is what happened after outcome 2, not a third outcome. Retrying and discarding are both recovery from the same failure and both cost money.

Before each call I record one line: what I'm asking for, and what a correct answer looks like. Then I ask one question of the response: did it give me that?

Cost per useful result is spend ÷ count of outcome 1. Everything spent reaching outcome 2 is waste, however I recovered.

The pre-written expected answer is what makes the dataset defensible. Without it the label is my mood.

Reasons, for outcome 2 only

Outcome 1 needs no reason — "it worked" is not a finding about anything. Outcome 2 gets a short code from a fixed set, so that reasons aggregate across calls rather than describing one:

Plus an optional free-text note, so the specific detail survives without polluting the codes.

The distinction between wrong, empty and malformed is the distinction between an endpoint that is broken and one that is lying. Those are different findings about a seller and I don't collapse them into a single failure rate.

Why the labeling is by hand

Not because automation isn't ready. The human correction is the signal the tool is built to capture. In production, the label arrives when whoever the agent serves says "that's wrong," and the agent relays it. Doing it by hand here is a faithful instance of the primary signal, not a stopgap — with the expected answer written first, so the correction is grounded rather than a reaction.

That is also why these labels are the reference set: any automatic labeler built later gets validated against them, and its agreement rate published alongside anything it computes. Validating a labeler against itself would prove nothing.

Category selection (the checkability rule)

I only buy from endpoints whose answers I can independently check. A guess in the dataset is worse than an absence.

In scope, anchor category first:

• Geocoding — venue names and addresses from The Blueprint, where I already have the coordinates. The cleanest category I have: the correct answer exists before I ask. Public venue names and addresses only. No user-account data from Blueprint's registrants, ever.

  Forward geocoding (name + address in, coordinates out), 8 Blueprint venues:

  • Outcome 1 if the returned point is within 100 meters of the coordinate I already hold for that venue. Outcome 2 beyond that, or if it returns a point in the wrong city.
  • 100 m absorbs the legitimate difference between a building centroid, a street entrance and a parcel center without letting a wrong-block answer through. My reference coordinates are not survey-grade either; they are what the platform used in production.

  Reverse geocoding (coordinates in, address out), 8 different Blueprint venues:

  • Outcome 1 if the street number and street name match the address I hold, ignoring formatting differences (abbreviations, casing, suite numbers, ZIP+4). Outcome 2 if either differs, or if it returns a neighboring address.
  • A returned business name is a bonus, not part of the pass condition.

• Web search — checkable in the strict sense first, in a judgmental sense second. Both standards run against the same search providers, which is how I spread across endpoints rather than across topics.

  Known-answer lookups:

  • Factual queries where the answer is fixed and I can verify it — World Cup goal scorers, the year a named venue opened, anything checkable against a public source in seconds.
  • Outcome 1 if the correct answer is present and correct in the results. Outcome 2 if it's absent (empty or insufficient), or if the results assert something wrong (wrong).
  • Queries spread across four difficulty tiers, two each, because a failure means something different in each: easy and famous (baseline coverage), specific but public (index depth), recent (index freshness), precise numeric (returns the value, not pages that might contain it).
  • A query only belongs here if I wouldn't have to argue about whether a returned answer is right. If "famous" or "best" appears, or the subject is ambiguous (two people with the same name), it moves to relevance or gets dropped.
  • I verify each expected answer against a source at grading time, not from memory.

  Web search relevance:

  • Open queries where there's no single right answer, graded on whether the results are on-topic and current. Outcome 1 if on-topic and current enough to act on; outcome 2 if off-topic (wrong), out of date (stale), or empty (empty). This is the softer standard and I label it as such.

• Weather — a location whose conditions I can observe directly. Outcome 1 if the value is correct at the time of the call, within the endpoint's own advertised resolution. Outcome 2 if wrong (wrong), for the wrong location (wrong), or materially out of date (stale).

• Price / market lookups — verifiable against a public source in seconds. Outcome 1 if within 1% of spot at call time, outcome 2 (stale or wrong) otherwise. Crypto rather than equities, because it trades 24/7 and there's no market-hours ambiguity.

Out of scope for run 001: trading signals, DeFi yield analysis, whale movements, "insights" — anything where grading the answer needs domain expertise I don't have. I exclude them because my label would be a guess, not because the endpoints are bad.

Sample shape

Roughly eight calls per category, spread across at least three or four distinct endpoints per category where the frozen leaderboard supports it. Eight calls to one geocoder tests that geocoder; eight calls across four tests whether the grade predicts quality, which is the actual question. If a category has fewer graded endpoints than that, I shrink the category rather than padding it with repeat calls to the same place, and I report the endpoint count alongside the call count.

Trial calls

Some catalogs offer a free trial call. Where one exists I use it once before paying, only to confirm the endpoint is alive and returns parseable data — never to preview the answer I am about to grade. The paid call is the one that gets reviewed. I record which endpoints were trial-checked first.

Protocol

1. Freeze the sample. Pull the grade A endpoints across distinct providers and record them — endpoint, grade, score, price, network — before I pay anyone. The list doesn't change once the run starts. Done 2026-09-14: 34 calls across 11 distinct providers, in x402-spend/examples/run-001.json, with the selection and what the frame could not supply written up in x402-spend/examples/run-001-sample.md.
2. Write the question and the expected answer for each call, in advance, one line each, and commit that file before any payment. The expected answers exist in git before the run, so nobody has to take my word for when they were written. They go in the same file as step 1, which the run harness refuses to start without.
3. Pay. One call per endpoint, same dedicated wallet throughout.
4. Label immediately, against what I wrote down: outcome 1 or 2, plus a reason code and an optional note when it's 2. No re-labeling later to tidy the dataset; if a label does change, I record that it changed and why.
5. Publish every row, including the ones where the endpoint did fine.

Attempt 1, void — recorded rather than quietly re-run

Amendment, 2026-09-15. Run 001 was executed once, at 05:59–06:00 UTC on 2026-09-15, and all 34 rows are void as quality data. The harness sent every request with no query parameters at all. Attempt 2 supersedes it. This section stays in the document because a run that is re-done silently is a run whose published numbers nobody can reconcile against its own history.

The defect. The plan's endpoint column was filled from the frozen catalog, which lists bare endpoint URLs, while the harness treated that column as the complete URL to request. The query column — the pre-registered question — was prose meant for a reader and was never encoded onto the wire. So GET https://2s.io/api/geocode/address went out with no q, and GET https://2s.io/api/weather/current with no lat or lon. The endpoints behaved correctly; 22 of them answered 400.

What it cost. 8 rows settled, $0.041 total. 22 returned 400 after the payment leg but before settlement, so they cost nothing. 2 returned 404 and 2 401, still undiagnosed.

The eight that returned 200 are the interesting ones, and are not yet labeled. Three calls to the same search endpoint — carrying, nominally, a World Cup question, a question about a college soccer coach, and an open question about football managers — came back byte-identical at 386 bytes each. Two calls to a geocoder came back identical at 473. Those endpoints returned a default payload to a request that asked nothing, and charged for it.

Read one way that is outcome 2, reason wrong: a well-formed, plausible, incorrect response, which is precisely the failure this method predicted would be hardest to catch — and it very nearly passed as a success because the status code was 200. Read the other way, an endpoint given no parameter cannot be said to have ignored one. Both readings are open and neither is labeled, because attempt 1 cannot distinguish them: the request was malformed, so the endpoint's behaviour toward a well-formed request is simply unobserved. Attempt 2 sends real parameters and can tell the two apart. That comparison is now a stated expectation of attempt 2 rather than a finding of attempt 1.

No row from attempt 1 is labeled, published, or counted — not for Axis 1, not for Axis 2, not for Axis 3. The results log and receipts are kept, and the defect, its cost and this reasoning are published alongside attempt 2.

What changed before attempt 2, so the same class of defect fails loudly instead of silently:

• Rows carry a params object, separate from endpoint. Keeping endpoint identical to the frozen catalog entry preserves the provenance chain from the sample freeze.
• The harness refuses to pay a row whose parameter names are not in the 402's own declared input, where the 402 declares one. A request the seller has already said it cannot answer is not worth paying for.
• The harness records response bodies. Attempt 1 discarded every one of them, including the 22 400 bodies that would have named the missing parameter on the spot. A research harness that throws away evidence is not one.

The pre-registered query and expected columns are unchanged and were not rewritten in light of any response. params is derived from query, which is a change of encoding rather than of expectation.

Wallet and spend

A dedicated run wallet per network family, funded with a few dollars and nothing more. Key in the password manager first, .env second, deleted after. Never in a handoff, a chat, or a repo. maxAmountPerPayment set as a hard ceiling. At typical prices, around a cent or two a call, the whole run comes in under $5.

What would falsify my prediction

If every endpoint on the leaderboard returns a usable answer, the finding is that external grades do predict quality, and x402-spend is narrower than I've been assuming — closer to spend accounting than to a quality signal. I publish that in the same words as any other result. Saying so in advance is what makes this an experiment instead of a marketing exercise.

Exploratory findings

Everything above is confirmatory: it tests what I declared before paying. The run will also surface things nobody declared in advance, and those go in their own section of the report, labeled as exploratory.

The distinction is not cosmetic. A confirmatory finding answers a question asked before the data existed. An exploratory one is something I noticed while looking, so I can't rule out that I noticed it because it was there to be noticed. Exploratory findings are hypothesis-generating; they don't carry the pre-registered claim and the report says so where they appear.

Rules I hold myself to for that section:

• They never change a label. Grading against something not pre-registered would break the one property this document exists to protect. Observations go in the report, not into the dataset's outcome column.
• Mechanical before interpretive. "These two responses agree to eight decimal places and both name the same upstream" is an observation. "These endpoints are resellers" is an inference from it. Both can appear; which is which gets stated.
• They are not presented as though they were tested. No claim that an exploratory pattern generalizes beyond the 34 rows it was seen in.

Announcing the section in advance is itself part of the pre-registration: it means exploratory findings were planned for as a category, rather than a heading invented after the fact to house whatever turned up.

Known limits

• Single reviewer, single judgment. One rater, no inter-rater check.
• Small n per category.
• The sample is the best-graded endpoints, so nothing here generalizes to the whole directory.
• Latency measured from one location.
• I chose categories for checkability, and checkability correlates with simplicity. The hardest endpoints to grade are also where quality problems are most likely to hide.
• My own recall is part of the instrument. I grade against a source at grading time rather than from memory, but a mis-specified expected answer is still possible, and I flag any call where I had to correct one.
• I built the tool this finding argues for, so my bias runs toward finding a gap. The pre-registered prediction and the falsification clause above are the only guards against that, plus publishing every row so anyone can re-check my work.

Outputs

• The dataset, public, CC BY 4.0, at x402-spend-reviews — every row re-verifiable against the chain by tx hash.
• One report covering all three axes. The headline is Axis 3; Axes 1 and 2 are what make it credible.
• A message to x402 Trust with the finding, framed as complementary: they measure whether an endpoint is alive, this measures whether it delivered.
• A message to Orthogonal with the same finding, since routing on price and availability has the same blind spot.