Skill: regression-anova

Correlation, Regression & ANOVA Basics

Interpret correlation, fit and use least-squares regression lines, and read one-way ANOVA output.

1 Correlation: direction and strength

The correlation r (between −1 and 1) measures how tightly points follow a straight line. Sign = direction; size = strength. It is not causation.

Reading r

|r| ≈ 1: strong | |r| ≈ 0.5: moderate | |r| ≈ 0: weak/none

r = 0.9: strong positive. r = −0.7: moderate negative. r = 0.1: essentially no linear relationship. r² is the fraction of y-variation explained by x.

Example r = 0.8

Positive, strong. r² = 0.64: about 64% of the variation is explained by the linear model.
It does not prove x causes y — a lurking variable could drive both.

2 The least-squares regression line

The line ŷ = a + bx minimizes the sum of squared vertical residuals. It always passes through (x̄, ŷ).

Regression formulas

b = r · (s_y/s_x) | a = ŷ − b·x̄

Slope b: “ŷ changes by b for each 1-unit increase in x.” Never extrapolate far beyond the data.

Example Study hours vs score

r = 0.7, s_x = 2 hrs, s_y = 14 pts, x̄ = 5, ŷ = 75.
b = 0.7 × (14/2) = 4.9 pts per hour.
a = 75 − 4.9 × 5 = 50.5. Model: ŷ = 50.5 + 4.9x.
Predict for 8 hours: 50.5 + 4.9(8) = 89.7.

3 One-way ANOVA: comparing several means

ANOVA tests whether several group means differ, by comparing variation between groups to variation within groups. Big F → the groups differ.

ANOVA F-statistic

F = MSB / MSW | MSB = SSB/df_B, MSW = SSW/df_W

MSB measures how far group means spread apart; MSW measures noise inside groups. F near 1: no signal.

Example Reading output

SSB = 120, df_B = 2 → MSB = 60. SSW = 200, df_W = 20 → MSW = 10.
F = 60/10 = 6.0 — between-group variation is 6× the within-group noise.

4 Common mistakes

Correlation proves causation. r measures association only. Experiments (or strong theory) are needed for causal claims — always ask about lurking variables.

Wild extrapolation. The line is only trustworthy near the observed x-range. Predicting far outside it assumes the trend continues, which you cannot verify.

Reading a as “when x = 0” literally. The intercept is the prediction at x = 0, which is often outside the data (and sometimes nonsensical, like negative height).

5 Key vocabulary

Correlation r: −1 to 1; direction and strength of linear association.
r-squared: Fraction of y-variation explained by the linear model.
Least-squares line: ŷ = a + bx; minimizes squared residuals; passes through (x̄, ŷ).
Residual: y − ŷ: vertical miss of the line at each point.
Extrapolation: Predicting outside the observed x-range — risky.
ANOVA F: MSB/MSW; large F signals real differences among group means.

6 Quick checks

1. r = −0.85. Describe it in two words.

Strong negative. Sign = direction, |r| = strength.

2. b = 2.5 in ŷ = 10 + 2.5x (x = hours, y = points). Interpret b.

Each extra hour predicts 2.5 more points. Slope = rate of change.

3. F = 1.1 in an ANOVA. What does it suggest?

Little signal. Between-group variation barely exceeds within-group noise.

Practice Regression fitter Scatterplot lab