ABENAN
The most common mistake is for a researcher to start from the software: open SPSS and look for a test that seems suitable. The right order is exactly the opposite: we first formulate the question and the hypothesis, then determine what the data look like, and only then choose the statistical tool that answers the question.
1. What is the purpose of the research question?
Most quantitative questions fall under one of the following purposes:
Description: What is the average satisfaction? What percentage agree?
Comparison: Does satisfaction differ between men and women?
Relationship: Is there a relationship between satisfaction and commitment?
Prediction: Which variables predict performance?
Change over time: Did the result change before and after the program?
If the question is purely descriptive, you may not need a hypothesis test at all, only frequencies, percentages, the mean, and the standard deviation. Do not use an inferential test just because a study “must contain tests”.
2. Identify the type of each variable
The type of variable narrows the options quickly:
Nominal: such as gender, region, or type of organization.
Ordinal: such as level of satisfaction: low, medium, high.
Quantitative: such as age, income, total score, or completion time.
Also identify the dependent variable you want to explain or compare, and the independent or grouping variable you expect to influence it. Many wrong choices come from not distinguishing between a variable’s role and its type.
3. Ask: how many groups do I have?
If your aim is comparison, the number of groups matters:
One group compared with a standard value: the One-Sample t test, when its assumptions are met.
Two independent groups: the Independent-Samples t test, or its non-parametric alternative, Mann–Whitney.
Two related groups, such as the same people before and after: the Paired-Samples t test, or Wilcoxon.
Three or more independent groups: One-Way ANOVA, or Kruskal–Wallis.
Three or more related measurements: Repeated-Measures ANOVA, or Friedman.
4. Are the samples independent or related?
This question changes the test even when the number of groups is the same. Comparing the scores of two different groups of employees means two independent samples. Measuring the same employees before and after training is a related measurement. Using a test for independent data on related data, or the reverse, can produce errors in estimation and in conclusions.
5. A simple map of common tests
| Research question | Shape of the data | Common test |
| Does the mean score differ between two independent groups? | Quantitative outcome + two independent groups | Independent t-test |
| Did the mean change before and after the program? | Quantitative outcome + two related measurements | Paired t-test |
| Do the means differ across three groups? | Quantitative outcome + 3 independent groups | One-Way ANOVA |
| Is there a relationship between two quantitative variables? | Two quantitative variables | Pearson or Spearman |
| Is there a relationship between two categorical variables? | Two nominal / categorical variables | Chi-square or Fisher |
| Which variables predict a quantitative score? | Quantitative outcome + predictors | Linear Regression |
| Which variables predict a binary outcome? | Binary outcome | Binary Logistic Regression |
This table is a starting point, not a substitute for examining the design and the assumptions. A test can be the right kind and still be unsuitable because of the sample size, the shape of the distribution, or dependence between observations.
6. Check the assumptions of the test
Parametric tests such as the t test and ANOVA rely on assumptions that differ by design. The most important are independence of observations, the shape of the distribution, and equality of variances. In the chi-square test, pay attention to the expected frequencies in the cells. Materials from Penn State and UCLA emphasize that choosing a method cannot be separated from the type of variables, the independence of the groups, and the assumptions about the data.
Do not rely on a single normality test and then decide automatically. Look at the plots, the sample size, the outliers, and the shape of the variable. Some tests also tolerate moderate departures better than others.
7. When do we use non-parametric alternatives?
Non-parametric tests are not a “lower-quality solution”, but they sometimes answer a slightly different question. We turn to them when the data are ordinal, when the distribution is very far from the assumptions and the sample is small, or when outliers have a strong influence. But a normality test value below 0.05 is not, on its own, enough reason to switch automatically to a non-parametric test.
8. Do not forget the effect size
A statistically significant difference can appear in a large sample and still be small in practical terms. So plan from the start to report the effect size, such as Cohen’s d, Eta Squared, or the correlation coefficient, alongside the confidence interval and the p value.
A worked example
Suppose the research question is: “Does job satisfaction differ by type of employer?”
Dependent variable: the satisfaction score, a quantitative variable.
Independent variable: type of employer, with three independent categories.
Aim: to compare the means of three groups.
Initial choice: One-Way ANOVA.
Before applying it: check independence, the distribution, equality of variances, and outliers.
If an overall difference appears: use appropriate post hoc comparisons to find where the difference lies.
Report the means and standard deviations, the F value, the degrees of freedom, the p value, and the effect size.
Common mistakes in choosing a test
Choosing the test based only on the name of the variable.
Confusing independent and related samples.
Running many tests in search of a significant result.
Turning a quantitative variable into categories without a scientific reason.
Ignoring the assumptions of the test or the outliers.
Reporting only the p value without the effect size and the confidence interval.
Using correlation to prove causation.
A decision checklist before the analysis
Write the research question clearly.
Identify the dependent and independent variables.
Classify the type of each variable.
Determine the number of groups.
Determine whether the groups are independent or related.
Review the sample size and the assumptions.
Choose the test and the effect size before looking at the results.
Document the reason for your choice in the study’s methodology.
ABENAN’s survey data analysis with SPSS service can help you build an analysis plan tied to your research questions instead of running scattered tests.
Frequently asked questions
Is it enough to know that my data are normal to choose the test?
No. Normality is only one factor. You also need to know the type of variable, the number of groups, whether the measurements are independent, the aim of the question, and the other assumptions. The data may be close to normal while the test is still unsuitable because the variable is categorical or the measurements are related.
Should I use a different test for each question in the questionnaire?
Not necessarily. Many items are combined into dimensions after checking the structure and reliability of the scale, and then the total score or the mean is analyzed according to the research question. Running a separate test for every item can increase the chance of random findings and makes interpretation harder.
When do I need statistical advice before collecting data?
It is best when the study involves multivariable models, repeated measurements, small samples, or questions that require estimating a sample size. Planning before data collection avoids discovering too late that the instrument or the sample cannot answer the hypothesis you need to test.
Loading comments…