19  Experimental Design, Replication, and Power

No amount of sophisticated modelling rescues a confounded design. The most important statistics in a mass spectrometry study are decided before a single sample enters the instrument.

Every analysis chapter so far assumed the data already existed. This chapter steps back to the decision that determines whether an experiment can answer its question at all: how it is designed. Get the design right — enough replication, no confounding, randomised run order — and the modelling in the chapters that follow is straightforward. Get it wrong, and no normalization, imputation, or empirical-Bayes moderation can recover a signal that was never separable from the noise.

This chapter is deliberately placed at the start of the statistical-modelling part: it is the bridge between how MS data is generated (Parts I–IV) and how it is analysed (Chapters 20–23).

WarningThe One Mistake to Avoid

Confounding your condition with batch or run order. If every case runs on Monday and every control on Tuesday, no statistical method can separate biology from day. Randomize and block before you acquire a single sample.

19.1 Learning Objectives

By the end of this chapter you will be able to:

  • Distinguish biological, technical, and batch sources of variation in MS data
  • Choose the right kind and number of replicates for your question
  • Randomise and block sample acquisition to protect against run-order and batch effects
  • Recognise and avoid confounding between the factor of interest and technical factors
  • Estimate statistical power and required sample size for MS experiments, both analytically and by simulation
  • Account for MS-specific realities — missingness, dynamic range, and dropout — when planning

19.2 Sources of Variation

Every measured intensity is a sum of signal and several kinds of noise. Good design maximises the first relative to the rest.

Source Example in MS Design lever
Biological Genuine between-subject differences This is your signal — sample enough subjects
Technical Sample prep, digestion, injection variability Technical replicates, internal standards
Batch Reagent lots, columns, instrument recalibration, acquisition day Randomisation, blocking, QC pools
Run-order / drift Sensitivity decline across a long sequence Randomise injection order; interleave QCs

The central design question is not “how do I remove noise?” but “how do I keep noise from aligning with my factor of interest?” Noise that is random can be modelled; noise that is confounded with the biology is fatal.

19.3 Replication

19.3.1 Biological vs. technical replicates

  • Biological replicates are independent experimental units (different subjects, animals, cultures). They capture the variability you want to generalise over, and they are what determines statistical power.
  • Technical replicates are repeated measurements of the same biological sample (re-injections, duplicate digestions). They quantify measurement precision but do not increase power for biological inference — averaging them gives one value per biological unit.
Warning

Treating technical replicates as if they were biological replicates is pseudoreplication — it inflates the apparent sample size and produces anti-conservative p-values. In limma, model technical replicates with duplicateCorrelation() and block on the biological unit (Chapter 21); never let them count as independent samples.

19.3.2 How many biological replicates?

The honest answer is it depends on the effect size and variance — which is exactly what the power analysis below quantifies. As rough field guidance for discovery MS:

Study type Typical minimum per group Notes
Cell-line proteomics (low variance) 3–4 Controlled system; small biological variability
Tissue / animal proteomics 5–8 Higher biological variance
Human clinical cohorts 20+ per group Large inter-individual variability; often imbalanced
Untargeted metabolomics (human) 30+ per group High variance + many features to correct over

These are starting points, not substitutes for a power calculation with your own pilot variance.

19.4 Randomisation and Blocking

Two samples measured back-to-back are more alike than two measured a day apart. If all your cases run on Monday and all controls on Tuesday, day and condition are confounded and no analysis can separate them.

  • Randomise injection order across the whole sequence so condition is not correlated with run position or acquisition day.
  • Block deliberately when a nuisance factor is unavoidable: if a study spans two plates, put a balanced mix of conditions on each plate rather than one condition per plate, then include plate as a covariate in the model (Chapter 21).
  • Interleave QC samples (a pooled aliquot of all samples) at regular intervals to monitor drift and enable signal correction (Chapter 6, Chapter 17).
  • For isobaric labelling (TMT/iTRAQ), balance conditions across channels and include a common reference channel in every plex for cross-batch normalisation (Chapter 14).
Code
# Randomise injection order while keeping QC pools at fixed positions
set.seed(2024)
samples   <- paste0("S", 1:40)
run_order <- sample(samples)                 # randomised biological samples
# Insert a QC pool every 10 injections
schedule  <- append(run_order,
                    values = rep("QC_pool", 4),
                    after  = c(0, 10, 20, 30))

19.4.1 The confounding check

Before acquisition, tabulate your factor of interest against every technical factor. Any strong association is a red flag.

Code
# Is condition balanced across batch? (want roughly uniform counts)
table(metadata$condition, metadata$batch)
# Is condition correlated with run order? (want ~0)
cor(as.integer(factor(metadata$condition)), metadata$run_order)

19.5 Power and Sample Size

Statistical power is the probability of detecting a true effect of a given size. Planning for adequate power is what separates a study that can succeed from one that is underpowered before it begins. Power depends on four quantities; fix any three and the fourth follows:

  1. Effect size — the difference you care to detect, in MS usually a log₂ fold-change
  2. Variability — the residual standard deviation (estimate from pilot data or literature)
  3. Sample size — biological replicates per group
  4. Significance threshold — after multiple-testing correction (Chapter 20)

19.5.1 Analytical power for a two-group comparison

For a single feature, a two-sample t-test power calculation gives a first estimate. The effect size (Cohen’s d) is the log₂ fold-change divided by the standard deviation.

Code
library(pwr)

log2fc <- 1.0            # detect a two-fold change
sd_est <- 0.8            # residual SD on the log2 scale (from pilot data)
d      <- log2fc / sd_est

pwr::pwr.t.test(d = d, sig.level = 0.05, power = 0.80,
                type = "two.sample")   # -> required n per group

19.5.2 Multiple testing changes everything

A single-feature calculation is optimistic: an MS experiment tests thousands of features and controls the false discovery rate, so the effective per-feature threshold is far stricter than 0.05. A practical adjustment is to target the Bonferroni-style level for the number of features you expect to test.

Code
n_features   <- 5000
target_fdr   <- 0.05
approx_alpha <- target_fdr / n_features        # conservative planning threshold

pwr::pwr.t.test(d = 1.0 / 0.8, sig.level = approx_alpha,
                power = 0.80, type = "two.sample")

19.5.4 MS-specific caveats

  • Missingness reduces effective sample size. A feature present in only half your samples has far less power than the nominal n suggests. Inflate planned n to cover expected dropout (Chapter 18).
  • Dynamic range and dropout interact. Low-abundance features are both more variable and more often missing, so they are systematically underpowered — power is not uniform across the feature table.
  • Imbalanced clinical cohorts lose power fast; the smaller group governs. Plan the case group, not the total.
  • Pilot data beats guesses. A small pilot to estimate the residual SD is the single most valuable input to a credible power calculation.

19.6 Design Checklist

Before acquisition:

19.7 Summary

  • The design decisions made before acquisition set the ceiling on what any analysis can achieve; confounding is unrecoverable downstream.
  • Biological replicates drive power; technical replicates quantify precision and must not be pseudoreplicated.
  • Randomise run order and block unavoidable nuisance factors so technical variation stays uncorrelated with biology.
  • Estimate power from effect size, variance, sample size, and a multiple-testing-aware threshold — and prefer simulation through your real limma pipeline over analytical shortcuts.
  • Budget for MS realities: missingness, dynamic range, and imbalance all erode power.

The next chapter takes a well-designed dataset and builds the statistical model that extracts its differentially abundant features.

19.8 Exercises

  1. Confounding audit. You are given a 24-sample study where samples 1–12 (all controls) ran on day 1 and 13–24 (all cases) on day 2. Explain why no normalization can fix this, and propose a re-acquisition schedule.
  2. Replicate accounting. A study has 4 subjects per group, each injected in triplicate. How many independent units does the differential test actually have, and how would you model the triplicates in limma (Chapter 21)?
  3. Analytical power. Using pwr, find the per-group sample size needed to detect a log₂FC of 1.5 with residual SD 0.9 at 80% power, first at α = 0.05 and then at a Bonferroni threshold for 4,000 features. Compare.
  4. Simulation power curve. Adapt the simulate_power() function to include 30% random missingness (set values to NA before fitting) and re-draw the power curve. How much does missingness raise the required n?
  5. Design a TMT experiment. You have 3 conditions × 6 replicates and TMT-10 plexes. Sketch a channel/plex layout that balances conditions across plexes and includes a reference channel.

19.9 Session Information

Code
sessionInfo()