What Ranking Does to a Correlation
Eleven colonic mucosa methylomes, one number that means almost nothing, and what changes when you throw the values away
The conclusion first
Every figure on this page is built from GEO series GSE81211 (Kang, Bae, Han, Kim, Kim, and Yi, 2016): Illumina 450K methylation on colonic mucosa from three people without inflammatory bowel disease and eight with ulcerative colitis, plus one colon cancer cell line. 478,440 CpGs survive with no missing values.
- The correlation is useless before you do anything to it. Across all 478,440 CpGs, every pair of the eleven mucosa samples correlates between \(0.957\) and \(0.990\). Two people with ulcerative colitis average \(0.983\). A person with colitis and a person without average \(0.975\). That gap — 0.008 — is the entire visible effect of the disease.
- Ranking does not fix it. Swap every value for its rank and the same comparison gives \(0.981\) against \(0.973\). The gap is still 0.008. Reaching for Spearman because Pearson looked suspicious buys you nothing here.
- What ranking buys is immunity to the scale you chose. Converting beta values to M-values — a step in almost every methylation pipeline — moves Pearson’s \(r\) by \(0.002\) on average across the mucosa pairs, and by \(0.031\) for the cancer line. It moves Spearman’s \(\rho\) by \(2 \times 10^{-11}\), which is to say it cannot move it at all.
- Ranking starts to earn its keep once you choose the CpGs. On the 10% most variable probes, the gap between same-group and cross-group pairs is \(0.100\) under Pearson and \(0.150\) under Spearman. Narrow to the top 1% and both gaps go negative.
Everything correlates with everything
Take two colonic biopsies, measure methylation at half a million CpGs, and ask how well they agree. The answer is \(r = 0.98\). Take two biopsies from people with different disease status and ask again: \(r = 0.98\). Take the two most dissimilar mucosa samples in the whole series and you still get \(r = 0.96\).
These are the samples from the panel study that reported a set of newly hypermethylated genes in ulcerative colitis (Kang, Bae, Han, Kim, Kim, and Yi, 2016), so there is a real signal in them somewhere. It is simply not what a whole-array correlation reports.
The matrix is not lying. Those samples genuinely are that similar: they are all human colon, and the methylome of human colon is largely fixed regardless of who is inflamed. The problem is that \(0.98\) is being read as evidence of similarity when it is mostly a statement about the shape of the numbers.
The values are stacked at the ends
A methylation beta value is the fraction of molecules methylated at one CpG: 0 means none, 1 means all. Biology does not spread those fractions evenly.
methylation fraction (beta)
Most CpGs are switched firmly off or firmly on. That produces two dense clouds a long way apart on both axes, and two distant clouds produce a high correlation by construction — before any biology is involved. The coefficient is mostly reporting that both samples agree about which CpGs are the “off” ones, which is a fact about being human colon, not about being inflamed.
Ranking does not rescue it
The standard reflex when a correlation looks untrustworthy is to rank the data. Sort the CpGs from least to most methylated, replace each value with its position in that order, and correlate the positions instead. That is all Spearman’s \(\rho\) is: Pearson’s \(r\), computed after ranking.
It changes almost nothing here. The mucosa pairs go from a mean of \(0.980\) to \(0.977\), and the disease gap stays at 0.008. Ranking removes the influence of how far apart the two clouds are, but the ordering was never in dispute — both samples put the same CpGs at the bottom and the same CpGs at the top. The clouds were an accurate picture of genuine agreement about ordering. Ranking cannot subtract a problem that was not there.
This is worth stating plainly, because the reflex is so common: a rank correlation is not a more conservative version of a correlation. It answers a different question, and on this data the two questions happen to have nearly the same answer.
What ranking does guarantee
Here is what ranking is actually for.
Nobody tests for differential methylation on beta values. Beta is bounded between 0 and 1 and its variance collapses at both ends, so the standard move is the M-value: the log-odds of being methylated. It stretches the crowded ends of the scale out and leaves the middle roughly where it was. It measures the same molecules; it just puts them on a different ruler.
Change the ruler and Pearson’s \(r\) changes. Across the mucosa pairs it shifts by \(0.002\) on average, and for the cancer line it moves from \(0.715\) to \(0.746\) — a shift of \(0.031\), more than ten times larger. Nothing was remeasured. The only thing that changed was which axis the same data was plotted against.
Spearman’s \(\rho\) does not move at all — by \(2 \times 10^{-11}\), which is floating-point noise rather than a real difference. The M-value transform is monotone: it can stretch the scale but it cannot reorder anything, and rank correlation sees only the order.
That is the honest division of labour. Pearson’s \(r\) is partly a claim about your data and partly a claim about the units you happened to store it in. Spearman’s \(\rho\) is only ever a claim about ordering. If your analysis would give a different answer depending on whether the pipeline handed you beta or M, that is a fact worth knowing before you publish.
The M-value is undefined at beta = 0 and beta = 1, which a 450K array does reach. The values here are clipped to \([0.001, 0.999]\) first, which is the usual guard. Ranks are always computed from the beta values, so the clip cannot quietly reorder anything.
Watch it happen
Two samples, 3,000 CpGs drawn at random from the 478,440. Pearson’s \(r\) is computed on whatever is currently plotted — so pressing Ranks makes the number travel, in real time, from Pearson to Spearman, because that is the whole of the difference between them.
The beta / M-value switch is the other half of the argument: in the value view it moves \(r\) and leaves \(\rho\) alone, and in the rank view it does nothing whatsoever, because the picture is already made of orderings.
The widget recomputes on its own 3,000-CpG sample, so its numbers land near — not exactly on — the whole-array figures quoted in the prose.
Where ranking starts to pay
The disease signal was never going to survive being averaged over half a million CpGs, most of which have nothing to do with inflammation. Throw the uninformative ones away and the picture changes. Below, gap is how much more two samples from the same group agree than two samples from different groups — the thing a correlation would need to see for it to be worth computing.
| CpGs kept | Number | Pearson gap | Spearman gap |
|---|---|---|---|
| all | 478,440 | +0.008 | +0.008 |
| most variable 50% | 239,220 | +0.031 | +0.036 |
| most variable 25% | 119,610 | +0.065 | +0.086 |
| most variable 10% | 47,844 | +0.100 | +0.150 |
| most variable 5% | 23,922 | +0.080 | +0.115 |
| most variable 2% | 9,569 | −0.007 | −0.016 |
| most variable 1% | 4,785 | −0.077 | −0.079 |
Two things happen at once. The gap grows as the CpG set narrows, and it grows faster under ranking — at the top 10% the rank-based gap is half again as large as the value-based one. This is where Spearman earns its place: once the constant, uninformative CpGs are gone, what is left is a handful of probes with large swings, and Pearson spends its attention on the size of those swings while Spearman only asks whether the samples agree about direction.
Then it reverses. Somewhere between the top 5% and the top 2% the gap turns negative — samples from the same group now correlate less than samples from different ones. Nothing biological changed at that threshold. The set simply got small enough that a few probes dominate, and the coefficient starts describing those probes rather than the samples. The filter that made the signal visible will, pushed a little further, manufacture the opposite one.
The widget shows one pair at a time rather than the group average, but the effect is visible in it. At the top 10%, Normal 1 against Normal 2 reads \(\rho = 0.84\) while Normal 1 against UC 1 reads \(0.60\). Push the slider to the top 1% and roughly thirty CpGs remain, at which point the readout is describing those thirty probes.
What ranking throws away
Ranking costs you something, and the cancer cell line shows what.
HCT116 is the one sample here that is genuinely, structurally different from the rest. Its Pearson correlation with the mucosa samples is \(0.715\); its Spearman correlation is \(0.676\). Under ranks it looks further from the mucosa samples than it does under values, because a rank correlation cannot see that its disagreements are concentrated in a comparatively small number of CpGs with very large differences. A rank knows that one sample is higher than another. It does not know by how much.
That matters because “do these two measurements agree?” and “do these two measurements rank things the same way?” are different questions, and only the first is a question about concordance. A tissue assay and a plasma assay could rank every participant identically and still disagree about every value — one reading systematically half the other. Spearman would report a perfect result. The assays would not be interchangeable.
The same caution applies in the other direction, to grouping. Every number in this post is computed within a defined comparison: normal against normal, colitis against colitis, normal against colitis. Pool them and the disease difference dissolves into an average. Pooling groups is the fastest way to manufacture — or erase — a correlation, and no choice of coefficient protects against it. That is a design decision, not a statistical one.
For this study
- Report a correlation with the CpG set attached. “\(\rho = 0.98\) across all probes” and “\(\rho = 0.80\) across the 10% most variable” describe the same samples and mean opposite things.
- Fix the probe filter before looking at the outcome. The table above shows how far a threshold can be pushed until it produces whatever answer is wanted.
- For tissue–plasma agreement, a rank correlation answers whether the two assays order participants the same way. It does not answer whether they can be substituted for one another. If the claim is substitution, the magnitudes have to be compared directly.
- State whether coefficients were computed on beta or M-values. It changes Pearson and it cannot change Spearman, so saying which one was used is enough to make the number reproducible.
prepare-data.py in this post’s directory downloads the GSE81211 series matrix from GEO, computes every number quoted above on the full 478,440 CpGs, and writes the committed methylation-subset.json that the figures and widget read. It is not run when the site builds; this page executes no code. Re-run it by hand to regenerate the data:
source .venv/bin/activate
python posts/beyond-correlation-association-vs-concordance/prepare-data.pyThe random subset behind the widget is drawn with a fixed seed, so it is identical on every run.