Study guides / CFA L2: CFA Program Level II / Regression diagnostics & extensions
Regression diagnostics & extensions
CFA L2: CFA Program Level II · quick review
What goes wrong in regressions (misspecification, heteroskedasticity, serial correlation, multicollinearity), how to detect and fix each, and the extensions: influential observations, dummy variables and logistic regression.
The rules
- Model specification: a good model rests on economic reasoning, is parsimonious, performs well out of sample, has the right functional form and satisfies the regression assumptions. Common failures: omitting a relevant variable (if it's correlated with included variables, their coefficients become biased and inconsistent), the wrong functional form (for example, levels where the relation is in logs or ratios), inappropriate scaling, and pooling data from different regimes. These often show up as heteroskedasticity, serial correlation or multicollinearity.
- Heteroskedasticity: unconditional heteroskedasticity (variance changes but isn't related to the Xs) causes no major problems. Conditional heteroskedasticity (variance related to the Xs) leaves coefficient estimates consistent but makes standard errors unreliable, usually too small, so t-statistics are inflated and you find significance that isn't there (Type I errors); the F-test is unreliable too.
- Breusch–Pagan test: regress the squared residuals on the independent variables; n × R² from that regression is chi-square with k degrees of freedom, one-tailed. If it's significant, use robust (White-corrected) standard errors, which change the standard errors but not the coefficients, or generalized least squares.
- Serial correlation: residuals correlated across observations, common in time series. Positive serial correlation makes standard errors too small and inflates t- and F-statistics (Type I errors). Coefficients stay consistent unless a lagged dependent variable is a regressor; then they become inconsistent. Durbin–Watson tests only first-order correlation and fails with a lagged dependent variable; the Breusch–Godfrey test handles p lags. Fix with serial-correlation-consistent (Newey–West) standard errors, which also correct for heteroskedasticity.
- Multicollinearity: two or more Xs are highly (not perfectly) correlated. Coefficients stay consistent but become imprecise: standard errors inflate, t-statistics shrink and real effects are missed (Type II errors). The classic sign is a high R² and a significant F with few or no significant t-statistics. A variance inflation factor above 5 warrants investigation; above 10 signals serious multicollinearity. Fix by dropping or replacing a variable or by adding data.
- Influential observations: a high-leverage point has an extreme X value; an outlier has an extreme Y given its Xs. Either can swing the estimates. Flag leverage hii (between 0 and 1) above 3 × (k + 1) ÷ n, and a studentized deleted residual beyond the critical t with n − k − 2 degrees of freedom. Cook's distance combines both: above 0.5 the point may be influential, above 1 it likely is. Investigate before deleting anything.
- Dummy variables: a qualitative factor with n categories needs n − 1 dummies; using n with an intercept creates perfect multicollinearity. An intercept dummy shifts the intercept: its coefficient is the difference in expected Y from the omitted base category, other Xs held constant. A slope dummy (D × X) changes the slope for that category. Test each with an ordinary t-test.
- Logistic regression: for a binary dependent variable (default or not, acquired or not), model the log odds, ln[P ÷ (1 − P)], as a linear function of the Xs, estimated by maximum likelihood. A coefficient is the change in log odds for a one-unit change in X, so e^b is the odds ratio; the change in probability depends on the starting point. Predicted probabilities always fall between 0 and 1.
- Logistic fit: the likelihood ratio test compares a restricted model with an unrestricted one, like a joint F-test; a large LR rejects the restriction. A pseudo R² can compare models only for the same dependent variable and data; it isn't a share of variance explained.
Formulas
- Breusch–Pagan: BP = n × R² from regressing squared residuals on the Xs; chi-square, df = k
- Breusch–Godfrey: F-test on p lagged residuals added to the regression, df = p and n − p − k − 1
- Variance inflation factor: VIFj = 1 ÷ (1 − R²j), where R²j comes from regressing Xj on the other Xs
- Leverage flag: hii > 3 × (k + 1) ÷ n
- Studentized deleted residual: ti = ei ÷ √[MSE(i) × (1 − hii)] = ei × √[(n − k − 2) ÷ (SSE × (1 − hii) − ei²)]; MSE(i) excludes observation i; compare with critical t, df = n − k − 2
- Cook's distance: Di = [ei² ÷ ((k + 1) × MSE)] × [hii ÷ (1 − hii)²]
- Logistic model: ln[P ÷ (1 − P)] = b0 + b1X1 + … + bkXk
- Probability from a logistic model: P = 1 ÷ [1 + e^−(b0 + b1X1 + … + bkXk)]
- Odds and probability: Odds = P ÷ (1 − P); P = odds ÷ (1 + odds)
- Likelihood ratio test: LR = −2 × (log likelihood restricted − log likelihood unrestricted); chi-square, df = q
Common traps
- Conditional heteroskedasticity and positive serial correlation inflate t-statistics (too many rejections); multicollinearity deflates them (too few).
- Robust standard errors fix inference, not the coefficients; the estimates don't change.
- Durbin–Watson is invalid with a lagged dependent variable, and in that case serial correlation also makes the coefficients inconsistent.
- Four quarters need three dummies, not four; and a logistic coefficient moves the log odds, not the probability, by b.
How it's tested
Vignettes show output with residual tests, VIFs or influence statistics and ask which problem is present, its effect on coefficients and t-statistics, and the right fix; others ask you to interpret dummy or logistic coefficients or compute a probability.
Practise it
The full lesson, worked examples and practice questions are in Tickmark, where every answer choice is explained.
Open the full lesson