Study guides / CFA L2: CFA Program Level II / Multiple regression
Multiple regression
CFA L2: CFA Program Level II · quick review
Building and interpreting a multiple linear regression: what each coefficient means, the assumptions behind valid inference, and how to judge fit with the ANOVA table, adjusted R², AIC and BIC and F-tests before using the model to predict.
The rules
- Uses and process: multiple regression explains how several factors drive a variable, tests theories (does a factor earn a premium?) and forecasts. Choose variables from economic reasoning; use linear regression for a continuous dependent variable and logistic regression for a binary one. Then estimate, examine the residuals against the assumptions, assess fit and significance, and revise or use the model.
- Partial slope coefficients: each slope bj is the expected change in Y for a one-unit change in Xj, holding the other independent variables constant. The intercept is the expected Y when every X is zero. Coefficients depend on which other variables are in the model, so a model that drops a variable must be re-estimated.
- Assumptions: linearity between Y and the Xs; homoskedasticity (the same residual variance for all observations); independence of errors (residuals uncorrelated across observations); normality of the residuals; and independence of the independent variables (they aren't random and no exact linear relation exists among them). Check them before trusting any test.
- Residual plots: plot residuals against predicted values and against each X. A patternless cloud is what you want. A curve suggests the relation isn't linear; a fan or funnel suggests heteroskedasticity; long runs of same-sign residuals in time order suggest serial correlation; a normal Q-Q plot whose ends bend away from the line shows fat tails. Isolated extreme points flag outliers.
- ANOVA and goodness of fit: total variation (SST) splits into explained (SSR) and unexplained (SSE). R² is the share explained, but it never falls when a variable is added. Adjusted R² penalizes extra variables, can be negative, and rises with a new variable only if that variable's t-statistic exceeds 1 in absolute value. The standard error of estimate (√MSE) measures the typical residual. None of these shows whether coefficients are significant or unbiased.
- Information criteria: lower is better. AIC is preferred when the model's purpose is prediction; BIC penalizes extra parameters more heavily (whenever ln n exceeds 2) and is preferred when the goal is the best goodness of fit. Compare only models of the same dependent variable on the same data.
- t-test on one coefficient: t = (estimate − hypothesized value) ÷ standard error, with n − k − 1 degrees of freedom. Rejecting H0: bj = 0 means Xj adds explanatory power given the other variables. Reject when |t| exceeds the critical value, or when the p-value is below the significance level.
- F-test of the whole model: H0 says all slope coefficients are zero. F = MSR ÷ MSE with k and n − k − 1 degrees of freedom; it's one-tailed, so reject if F exceeds the upper critical value. A significant F with no significant t-statistics is a sign of multicollinearity.
- Joint test of some coefficients: to test whether q variables together add nothing, compare a restricted model (without them) with the unrestricted model (with them). A large F means they're jointly significant, even if each t-statistic looks weak. The denominator degrees of freedom use the unrestricted model's k.
- Prediction: plug the assumed X values into the estimated equation using every estimated coefficient, including insignificant ones, unless the model is re-estimated without them. The forecast carries model error and sampling error in the coefficients, plus more uncertainty when the X values are themselves forecasts.
Formulas
- Multiple regression model: Y = b0 + b1X1 + b2X2 + … + bkXk + ε
- Predicted value: Predicted Y = b0 + b1X1 + … + bkXk (estimated coefficients, assumed Xs)
- Sum of squares: SST = SSR + SSE
- R²: SSR ÷ SST = 1 − SSE ÷ SST
- Adjusted R²: 1 − [(n − 1) ÷ (n − k − 1)] × (1 − R²)
- Mean squares: MSR = SSR ÷ k; MSE = SSE ÷ (n − k − 1)
- Standard error of estimate: √MSE
- t-statistic: (estimated bj − hypothesized bj) ÷ standard error of bj, df = n − k − 1
- F-statistic (all slopes zero): MSR ÷ MSE, df = k and n − k − 1
- Joint F-test (q restrictions): [(SSE restricted − SSE unrestricted) ÷ q] ÷ [SSE unrestricted ÷ (n − k − 1)], df = q and n − k − 1
- AIC: n × ln(SSE ÷ n) + 2 × (k + 1)
- BIC: n × ln(SSE ÷ n) + ln(n) × (k + 1)
Common traps
- Degrees of freedom are n − k − 1, not n − 2; k counts the slope coefficients, not the intercept.
- A higher R² after adding variables doesn't mean a better model; compare adjusted R², AIC or BIC.
- Dropping an insignificant variable and reusing the other coefficients without re-estimating gives wrong predictions.
- The F-test is one-tailed, and a significant F doesn't mean every coefficient is significant.
How it's tested
Item sets give regression output (coefficients, standard errors, an ANOVA table) and ask you to interpret a coefficient, compute R², adjusted R², F or a predicted value, or choose a model on adjusted R², AIC, BIC or a joint F-test.
Practise it
The full lesson, worked examples and practice questions are in Tickmark, where every answer choice is explained.
Open the full lesson