Which technique reduces variance by averaging predictions from multiple models trained on bootstrap samples?

Which technique reduces variance by averaging predictions from multiple models trained on bootstrap samples?

• A. Bagging
• B. Boosting
• C. Stacking
• D. Pruning

Answer: (A)

Reducing variance in predictions is achieved by creating multiple models on different samples and averaging their outputs. This idea is realized by bagging: generate bootstrap samples from the training data, train a model on each sample, and then average their predictions (or use majority voting for classification). The bootstrap process introduces diversity among the models, so their errors aren’t perfectly the same. When you average the predictions, these independent-ish errors tend to cancel out, producing a more stable and often more accurate predictor on new data. Bagging is especially helpful with high-variance, low-bias models like decision trees. In contrast, boosting trains models sequentially to fix previous mistakes and often aims at reducing bias (and can increase variance), stacking combines models with a meta-model, and pruning reduces complexity of a single model rather than aggregating many; these are different approaches with different effects on bias and variance.

Reducing variance in predictions is achieved by creating multiple models on different samples and averaging their outputs. This idea is realized by bagging: generate bootstrap samples from the training data, train a model on each sample, and then average their predictions (or use majority voting for classification). The bootstrap process introduces diversity among the models, so their errors aren’t perfectly the same. When you average the predictions, these independent-ish errors tend to cancel out, producing a more stable and often more accurate predictor on new data. Bagging is especially helpful with high-variance, low-bias models like decision trees. In contrast, boosting trains models sequentially to fix previous mistakes and often aims at reducing bias (and can increase variance), stacking combines models with a meta-model, and pruning reduces complexity of a single model rather than aggregating many; these are different approaches with different effects on bias and variance.