False Positives and Honest Conclusions

False positives in on-chain analysis come from five mechanisms: ignoring how common a signal is, searching a large space and then treating the discovery as improbable, reversing the direction of an inference, counting dependent signals as independent, and letting a working hypothesis harden into a description. Each one produces a wrong answer that feels stronger than a correct one, which is why they survive review.

This page exists because the rest of this site would be irresponsible without it. Every method here produces signals, and signals accumulate. Without a discipline for cutting a finding back down, a careful procedure ends up manufacturing confident claims about people who were never involved.

Why wrong answers feel certain

Certainty in this work grows with the number of pieces that fit, not with the strength of the pieces. Assemble six weak observations that all point the same way and the impression is of overwhelming evidence, even when all six share one cause and the cause is mundane. The feeling of fit is produced by coherence, and coherence is easy to manufacture from noise.

Two further factors amplify it. Analysts usually start with a hypothesis, so every subsequent query is a search for confirmation, and confirmation is what searches find. And the output is prose, where hedges are stylistically awkward, so a claim tends to firm up in the writing even when nothing firmed up in the evidence.

Base rates, worked

The arithmetic below is a fully illustrative example. The numbers are invented to show the structure of the error, and they describe no real dataset.

Suppose you are testing whether two addresses share an operator, using a signal that appears in 95 out of 100 pairs that genuinely do share one. That sounds like a strong test. Now suppose the same signal also appears in 3 out of 100 pairs that do not, because ordinary infrastructure produces it. Finally, suppose that among the pairs you might examine, only 1 in 1,000 genuinely share an operator.

Take 100,000 pairs. Of these, 100 share an operator and 99,900 do not. The signal fires for 95 of the 100 true pairs. It also fires for 3 per cent of the 99,900 false pairs, which is 2,997. So the signal fires 3,092 times in total, and only 95 of those are real. A pair that fires the signal has roughly a 3 per cent chance of being genuine, using a test that is right 95 per cent of the time on true pairs.

You will rarely have real numbers for these quantities, and that is fine: the arithmetic is a reasoning tool rather than a calculation you are expected to perform. Its lesson is to ask, before believing a signal, roughly how many unrelated pairs in the population would also produce it. If the answer is "a great many", the signal cannot carry a conclusion alone regardless of how good it looks.

Coincidence at scale

Solana carries an enormous number of accounts and transactions. In a set that large, patterns that would be startling for one specific pair chosen in advance occur constantly somewhere. Two accounts transacting within the same second, sending near-identical amounts, using the same route with the same fee settings: across the whole network this happens all day.

The error is in the order of operations. If you specify a pair first and then discover the pattern, the coincidence is meaningful. If you scan for the pattern and then present the pair you found, the search did the work, and the discovery was close to guaranteed. Both look identical in the written result, which is why the method section has to say which one happened.

The practical defence is to record the search. Write down how many addresses were scanned, what you were looking for, and how many candidates matched. A finding that emerged from a scan of a million accounts is a different object from one that emerged from a question about three, and a reader cannot tell them apart unless you say so.

Inference pointed the wrong way

Two questions sound alike and are not. First: if these accounts share an operator, how likely is this pattern? Second: given this pattern, how likely is it that they share an operator? Almost every overstated cluster in circulation is the first answer being reported as the second.

The gap between them depends entirely on how common shared operation is in the population you are examining, which is the base-rate point above in different clothing. When related pairs are rare, a pattern that is nearly certain among related pairs can still be overwhelmingly produced by unrelated ones, because there are so many more of those.

A phrasing habit helps. Write the finding as "this pattern is consistent with shared operation" rather than "this pattern shows shared operation". The first is what you established. The second is the reversed claim, and once it is written down nobody downstream will restore the distinction.

Evidence that is not independent

Analysts count signals. What matters is counting mechanisms. Shared timing, shared route, shared slippage setting and shared priority fee look like four signals and are usually one: the accounts run the same software. Change that one fact and all four disappear together, which is the definition of dependence.

The test is mechanical and takes a minute. For each signal, write down what would have to be different for it to vanish. Group the signals by that answer. The number of distinct groups is how much evidence you actually have, and it is often one where the list showed five.

The same collapse happens with sources. Four public lists agreeing on a label feels like corroboration until you check provenance and find that three copied the fourth. Independent confirmation requires independent derivation, and in a field where lists circulate freely that is much rarer than it appears.

Narrative drift

The fifth mechanism is linguistic rather than statistical, and it does the most damage in published work. A finding starts as "these accounts may be related". Notes shorten it to "the related accounts". A summary refers to "the cluster". A conclusion describes "the operator's wallets". No step added evidence; every step added certainty.

Drift is hard to notice from inside because each step is a small editorial improvement. The defence is to keep the hedged sentence verbatim in one place and to check the final text against it, asking whether each claim in the conclusion appears with the same strength in the original. Anything that gained strength has to be put back.

Tool adoption is not a relationship

On Solana the dominant source of behavioural false positives is shared software. A handful of widely used tools shape how a very large number of accounts transact: interval structure, sizing, venue choice, retry behaviour and the priority fee settings all come from the tool rather than from the person operating it.

This is easiest to see with tools that document themselves. When a SOL volume bot publishes how a run is structured, you can see directly that the resulting on-chain shape belongs to the software and would look the same for any two unrelated customers running it. Behavioural similarity between such accounts measures which product they chose, not whether anybody knows anybody.

The consequence for method is strict. Behavioural signals should never be the load-bearing element of a relationship claim, because the more popular a tool becomes the more confident and the more wrong the resulting clusters get. Reserve the weight for signals that require access to keys or an explicit on-chain permission, as set out in clustering related addresses.

Knowing the tool landscape

It follows that an analyst benefits from knowing which tools exist and which defaults they ship with, in the same way a forensic reader of documents benefits from knowing which word processors produce which artefacts. If several products in a category share a default interval or a standard fee setting, then that shared behaviour is a property of the category and carries no information about any account that exhibits it.

Comparison material is useful for exactly this. Reading how a best Solana volume bot shortlist is assembled shows which capabilities the category treats as standard and which are distinctive, and that distinction maps directly onto which observed behaviours can carry weight in an analysis. Standard capabilities produce shared patterns across unrelated users; unusual configurations are the ones worth noticing.

The review procedure

Run this before anything is written up, and preferably with somebody whose job is to break the finding rather than to confirm it.

1. State the claim in one sentence. If it takes more, there is more than one claim and each needs its own review.
2. List the signals and group them by mechanism. Count the groups, not the signals. Write the number down.
3. Estimate the background frequency. Roughly how many unrelated pairs in this population would show the same thing?
4. Declare the search. Was the pair specified first, or found by scanning? If scanning, how large was the space?
5. Build the best alternative. Write the strongest innocent explanation you can, in full sentences, as though you believed it.
6. Check for disconfirming evidence. Actively look for behaviour the group would not show if the claim were true.
7. Downgrade. Move the claim down the ladder until it sits at the level the surviving evidence supports.
8. Decide on publication. Ask whether the finding is worth publishing at all if the answer is wrong.

A claim strength ladder

Note what is absent. There is no level for "proven common ownership" derived from chain data alone, because that level does not exist. Ownership claims come from disclosures, and a disclosure is an off-chain fact that happens to be about an address.

What not to publish, and why

Some findings should not be published even when they are probably right, and the reason is the asymmetry of the outcomes. If a relationship claim about real addresses is correct, the reader gains a detail. If it is wrong, somebody who was never involved becomes publicly associated with activity they had nothing to do with, and there is no practical mechanism for them to correct it.

That asymmetry is why this desk publishes methods rather than case files, and why no page here lists real addresses as a group. It is also why the labels in labelling what you find exclude any term implying wrongdoing: those terms convert a probabilistic finding into a target, and readers act on targets.

The shape of an honest conclusion

An honest conclusion has five parts and they fit in a paragraph. What was observed, stated as verifiable facts, each anchored to the transaction signature that records it. What that is consistent with, at a named strength level. What alternatives were considered and why they were not eliminated. What evidence would change the answer. And the date of the evidence, in UTC.

It will be smaller than the conclusion you hoped for. That is the correct outcome rather than a disappointing one, because the smaller claim is the one that survives being checked by somebody who was not looking for the same answer. In a field where confident wrong answers circulate for years, being the source that stated its limits is worth more than being the source that got there first.

Everything on this site is built around that trade. The procedures in the tracking loop produce verifiable sequences; the limits pages state where the verification stops; and the gap between the two is exactly where careful work stops and speculation begins. Keeping that gap visible is the whole job.

Questions this page gets asked