Direct answer
Choose a statistical test only after defining the target effect, outcome type, study design, number of groups, independence or pairing, model assumptions, and whether estimation or testing is the primary goal.
Visual explanation
Design first, method second
What this calculation tells you
A statistical test is one component of an analysis plan. Different tests answer different null questions even when they accept similarly shaped input tables.
Start with the estimand: mean difference, proportion difference, association, distributional shift, variance, rate, or another target. Then map the actual sampling unit and dependence structure.
Where it is used
Research planning
Translate a question and design into an auditable candidate analysis.
Quality
Separate paired process changes from independent group comparisons.
Education
Organize method selection around estimands and sampling units.
Review
Identify mismatches between study design and reported test.
When this guide helps
- Comparing two independent groups.
- Analyzing before-and-after measurements.
- Testing association between categories.
- Comparing several groups or repeated conditions.
Start with the statistical question
Start with the estimand: mean difference, proportion difference, association, distributional shift, variance, rate, or another target. Then map the actual sampling unit and dependence structure.
A branching decision map moves from numerical or categorical outcome through group count and pairing to candidate families, with assumption checkpoints and an estimation-first exit.
Worked example
Two measurements from the same participants form paired differences, so an independent two-sample test discards the pairing. Two separate groups do not become paired merely because their sizes match.
Assumptions that carry the result
Every candidate has design and distributional conditions. Nonparametric does not mean assumption-free, and a normality pretest is not a universal automatic switch.
Interpret the result without overreaching
The selector cannot judge data collection, causal identification, multiplicity, missingness, measurement validity, protocol deviations, or whether the target question matters.
- Choosing from variable names rather than design.
- Treating repeated measurements as independent rows.
- Running several tests and reporting only the smallest p-value.
Worked case: paired measurements
The same 20 participants have before and after values.
Analyze within-person differences with a paired method rather than treating 40 observations as independent.
Pairing preserves the design and often removes between-person variability.
A missing partner needs an explicit missing-data rule.
Reproduce this worked caseOpen Statistical Test Selector
Worked case: independent groups
Twenty people receive A and a different twenty receive B.
Use an independent-group method appropriate to outcome and variance assumptions.
There is no natural within-person difference.
Applying a paired test would invent links that do not exist.
Reproduce this worked caseOpen Statistical Test Selector
statistical-test selection: compare assumptions, not just answers
A selector narrows candidates; it cannot validate sampling, causal identification or data quality. Write the estimand first.
| Case | Calculation focus | Interpretation |
|---|---|---|
| Same people twice | Dependent observations | Paired method |
| Different people | Independent groups | Two-sample method |
| Categorical counts | Contingency design | Chi-square/exact family |
statistical-test selection: calculation checklist
- Outcome type named
- Pairing/clusters retained
- Number of groups stated
- Assumptions checked
- Multiplicity considered
Practical questions
Frequently asked questions
Can a calculator choose the test automatically?
It can organize candidates from entered design features, but it cannot verify the scientific question, collection process, assumptions, or consequences.
Are nonparametric tests assumption-free?
No. They still require appropriate sampling, independence or pairing, outcome ordering, and an interpretation matched to their statistic.
Should I test normality first?
A mechanical pretest-and-switch workflow can distort error properties; inspect design, residuals, robustness, and model purpose together.
Further reading
Authoritative sources
Use these primary and professional resources to check definitions, conventions, or requirements that may extend beyond this guide.