What is a Biological Signature?

A fingerprint is a few features that stand in for a whole person. Finding one in a dataset is feature selection.

A fingerprint identifies a person from a patch of skin. A biological signature does it with molecules — and pulling one out of data takes four steps, of which the last is the one people get wrong.

Thousands of molecular measurements narrowing into a small biological signature.

NoteCross-post

This article was originally published on the Project Delphi ML Blog.

A signature is a short list of measured molecular features (genes, proteins, or metabolites) that distinguishes one condition from another.

It is feature selection on wide, small-\(n\) data: tens of thousands of genes, tens of patients. A model given all features will fit noise.

Measure wide

screen · penalize · resample

Keep the stable signal

Gene 04

CpG 17

Protein 09

Gene 31

Metabolite 06

Feature selection is compression with a biological purpose: reduce thousands of measurements to a small panel without throwing away the distinction that matters.

1 Procedure

1. Measure wide. Every feature on every sample.
2. Find the groups. Use known labels, or let clustering propose groups.
3. Select. Score features by group separation:
   • \(t\)-test: distance between group means relative to within-group scatter.
   • Mutual information: how much the feature tells you about group membership.
   • Penalized models (lasso, elastic net): drop features by charging a cost for each one used.
4. Validate on held-out data. Redo selection inside each cross-validation fold.

Selecting on all samples, then testing on a “held-out” subset, leaks the selection. Ambroise and McLachlan (2002) showed that this bias drives reported error toward zero; nested selection raises it again.

Leaky validation

All samples

→

Select features

→

Cross-validate

Too optimistic: the held-out samples already influenced which features were chosen.

Honest validation

Training fold

→

Select features

→

Held-out fold

Defensible: every fold repeats selection before the untouched samples are evaluated.

The held-out fold must remain invisible to every outcome-informed step—not only model fitting, but also feature screening, tuning, scaling, and imputation.

2 Canonical example: Golub leukemia

ALL and AML are two acute leukemias that look similar under a microscope and take opposite chemotherapies. Diagnosis used to rest on four specialist lab tests.

Golub et al. (1999) measured 6,817 genes in 38 acute-leukemia samples.

• Unsupervised clustering, given no disease labels, nearly recovered the split (one group 24 ALL + 1 AML; the other 10 AML + 3 ALL).
• A 50-gene predictor was then tested on 34 independent samples from other banks.

Fifty features out of 6,817 is a signature.

6,817genes measured in the discovery experiment

50genes retained in the predictive signature

34fresh samples used for independent testing

The same four steps apply to environmental DNA (a few marker sequences implying a species was present) and to exoplanet spectra (gases inconsistent with abiotic chemistry).

3 References

• Ambroise, C. & McLachlan, G. J. (2002). Selection bias in gene extraction on the basis of microarray gene-expression data. PNAS.
• Golub, T. R. et al. (1999). Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286(5439):531–537.
• Guyon, I. & Elisseeff, A. (2003). An introduction to variable and feature selection. JMLR.
• Meinshausen, N. & Bühlmann, P. (2010). Stability selection. JRSS-B.