Study guides / CFA L2: CFA Program Level II / Machine learning & big data
Machine learning & big data
CFA L2: CFA Program Level II · quick review
The main machine learning approaches and algorithms and when each fits, how to control overfitting, and the steps of a big data project for structured and text data, from cleansing and feature engineering to training and evaluating a model.
The rules
- Types: supervised learning uses labeled data to predict a target (regression for a continuous target, classification for categories). Unsupervised learning has no labels and finds structure (dimension reduction, clustering). Neural networks pass inputs through hidden layers of nodes, each applying a summation and an activation function; with many hidden layers they become deep learning nets, used for complex nonlinear tasks such as images and speech. Reinforcement learning has an agent learn by trial and error to maximize rewards.
- Overfitting: a model that fits training data too closely learns noise and generalizes badly. Total error splits into bias error (underfitting: poor fit even in sample), variance error (overfitting: results that change a lot out of sample) and base error (noise). Data is split into training, validation and test samples, and out-of-sample error is what counts. The best complexity minimizes total error.
- Fixing overfitting: reduce complexity (fewer features, penalties, limits on tree growth) and use cross-validation. In k-fold cross-validation the data is shuffled into k parts; train on k − 1 parts, validate on the remaining one, rotate k times and average the validation errors.
- Penalized regression (LASSO): minimizes the sum of squared errors plus a penalty of λ times the sum of the absolute coefficients. It shrinks some coefficients to exactly zero, so it selects features; it suits data with many, possibly correlated, features. λ is a hyperparameter chosen by the researcher; λ = 0 is ordinary least squares.
- SVM and KNN: a support vector machine is a linear classifier that finds the boundary with the widest margin between two classes; the observations on the margin are the support vectors. Soft-margin classification tolerates some errors when the data isn't perfectly separable, and kernels allow nonlinear boundaries; it suits complex, high-dimensional classification. k-nearest neighbor classifies a new observation by majority vote of its k closest neighbors; the choice of k (too small fits noise, too large blurs classes) and of the distance measure drive results.
- Trees and ensembles: classification and regression trees (CART) split the data repeatedly on the feature and cutoff that best separate it, giving an easy-to-explain model that overfits unless limited (maximum depth, minimum observations per node, pruning). Ensemble learning combines models: a voting classifier takes the majority prediction, and bagging trains models on bootstrap resamples. A random forest bags many trees, each split drawing on a random subset of features; it reduces overfitting but is harder to interpret.
- Unsupervised algorithms: principal components analysis replaces many correlated features with a few uncorrelated composites (eigenvectors) ranked by the share of variance they explain (eigenvalues); keep enough to explain most of the variance (a scree plot helps), at the cost of interpretability. K-means splits data into a preset number k of clusters by repeatedly assigning points to the nearest centroid and recomputing centroids; results depend on k and the starting points. Hierarchical clustering needs no preset k: agglomerative merges the closest clusters bottom-up, divisive splits top-down, and a dendrogram shows the result.
- Big data project steps: conceptualize the task (the problem, the output and how it will be used), collect data, prepare and wrangle it, explore it (exploratory analysis, feature selection, feature engineering), then train the model. Text projects follow the same path: text problem formulation, data curation, text preparation and wrangling, text exploration, model training. Big data is characterized by volume, variety, velocity and veracity.
- Preparing structured data: cleansing fixes incompleteness, invalidity, inaccuracy, inconsistency, non-uniformity and duplication. Wrangling includes extraction, aggregation, filtration (removing rows), selection (removing columns) and conversion. Outliers are handled by trimming (removing extremes) or winsorization (replacing them with cutoff values). Scale with normalization (to a 0–1 range; sensitive to outliers) or standardization (mean 0, standard deviation 1; less sensitive to outliers but works best when data is roughly normal).
- Text data: cleansing strips HTML tags, most punctuation, numbers and extra white space, often replacing them with annotations. Normalization lowercases, removes stop words, and applies stemming (cutting words to a base stem) or lemmatization (mapping to the dictionary root; more costly). Tokens build a bag of words and a document term matrix; n-grams keep short word sequences. Feature selection uses document frequency, chi-square tests and mutual information (near 0, a token says nothing about the class); feature engineering adds n-grams, named entity recognition and parts of speech. TF-IDF weights tokens that are frequent in a document but rare across documents.
- Training and evaluation: pick the method by task (labeled or not), data type and size; fix class imbalance by undersampling the majority class or oversampling the minority. A confusion matrix gives true and false positives and negatives: stress precision when false positives (Type I errors) are costly and recall when false negatives (Type II errors) are costly; the F1 score balances them and beats accuracy when classes are unequal. The ROC curve plots the true positive rate against the false positive rate; a larger AUC is better and 0.5 is no better than chance. Tune hyperparameters (for example, by grid search) to trade bias error against variance error.
Formulas
- Normalization: (X − minimum) ÷ (maximum − minimum)
- Standardization: (X − mean) ÷ standard deviation
- Precision: TP ÷ (TP + FP)
- Recall (true positive rate): TP ÷ (TP + FN)
- Accuracy: (TP + TN) ÷ (TP + TN + FP + FN)
- F1 score: (2 × precision × recall) ÷ (precision + recall)
- False positive rate: FP ÷ (FP + TN)
- RMSE: √[Σ(predicted − actual)² ÷ n]
- Term frequency (TF): Times a token appears in a document ÷ total tokens in that document
- Document frequency (DF): Documents containing the token ÷ total documents
- TF-IDF: TF × IDF, where IDF = log(1 ÷ DF)
Common traps
- Precision divides by predicted positives (TP + FP); recall divides by actual positives (TP + FN). Swapping them is the most common slip.
- Accuracy misleads when classes are imbalanced; use F1 or the ROC curve.
- K-means needs k set in advance; hierarchical clustering doesn't. PCA components are hard to interpret because they mix the original features.
- Trimming deletes outliers while winsorizing replaces them; normalization is the scaling method that is sensitive to outliers.
How it's tested
Vignettes describe a data project and ask which algorithm suits the problem, which step or technique a described action is, how to address overfitting or class imbalance, or to compute precision, recall, accuracy or F1 from a confusion matrix.
Practise it
The full lesson, worked examples and practice questions are in Tickmark, where every answer choice is explained.
Open the full lesson