Hypothesis testing

ANOVA and Nonparametric Multi-Group Tests

Choose among one-way, repeated-measures, factorial, rank-based, and follow-up procedures from the design and target comparison.

Direct answer

ANOVA partitions modeled variation to compare group means, while Kruskal–Wallis and Friedman use ranks for independent-group and repeated-block designs; the appropriate method follows the sampling structure and estimand.

Visual explanation

Between-group and within-group variation

betweenwithinrepeated
Group structure, repeated units, factors, and ranks determine the appropriate multi-group method.

What this calculation tells you

Multi-group methods control one planned global comparison rather than running many unadjusted pairwise tests. Their statistics summarize between-condition evidence relative to residual or rank variation.

Identify independent groups, blocks, repeated units, factors, interactions, outcome scale, and the parameter or distributional effect of interest before selecting a method.

Where it is used

Experiments

Compare several planned conditions in one coherent model.

Quality

Compare process groups while separating residual variation.

Agriculture and field trials

Represent blocks and factors without discarding design structure.

Education

Connect sums of squares, ranks, and study layouts visually.

Common situations

  • Comparing three or more independent groups.
  • Analyzing repeated conditions on the same units.
  • Testing two factors and interaction.
  • Choosing a rank-based multi-group method.

Start with the statistical question

Identify independent groups, blocks, repeated units, factors, interactions, outcome scale, and the parameter or distributional effect of interest before selecting a method.

Three group clouds feed into between- and within-group variation. Parallel panels show independent groups, repeated connected profiles, and a two-factor grid with an interaction crossing.

Worked example

A one-way ANOVA can reject equality among four means without saying which pair differs. Planned contrasts or multiplicity-aware follow-up comparisons must answer those narrower questions.

Assumptions that carry the result

Classical ANOVA relies on a suitable linear model and residual assumptions. Kruskal–Wallis assumes independent groups and interprets rank shifts; Friedman assumes matched blocks or repeated units.

Interpret the result without overreaching

An omnibus p-value does not measure effect magnitude, validate causal design, establish every pairwise difference, or justify ignoring interaction and repeated structure.

  • Running all pairwise t tests without multiplicity planning.
  • Using one-way ANOVA for repeated observations.
  • Calling a rank-test result a mean difference automatically.

Choose the right tool

Practical questions

Frequently asked questions

Does significant ANOVA mean every group differs?

No. It rejects the specified global equality model; follow-up comparisons require their own estimands and multiplicity control.

Is Kruskal–Wallis a median test?

Not automatically. It tests rank-distribution differences; a location-shift interpretation requires comparable shapes and other assumptions.

What is an interaction?

It means the modeled effect of one factor differs across levels of another, so main effects alone may be incomplete.

Further reading

Authoritative sources

Use these primary and professional resources to check definitions, conventions, or requirements that may extend beyond this guide.