Which ensemble approach is built by repeatedly sampling data and training on those samples to improve stability?

Which ensemble approach is built by repeatedly sampling data and training on those samples to improve stability?

• A. Random Forests
• B. Boosting
• C. Gradient Boosting
• D. AdaBoost

Answer: (A)

Repeatedly sampling data and training on those samples to improve stability is the bagging idea, bootstrap aggregating. In bagging, you create multiple bootstrap samples from the training data (sampling with replacement), train a separate model on each sample, and then combine their predictions. This process lowers variance because the individual models are trained on different portions of the data and, when you average or vote their outputs, the fluctuations of any single model tend to cancel out. Random Forests implement this approach by growing many decision trees on different bootstrap samples and then averaging their predictions (or using majority vote for classification). They also introduce randomness in feature selection at each split, which further decorrelates the trees and boosts stability and generalization. This differs from boosting methods, which train models sequentially to correct errors of earlier models and do not rely on training on multiple bootstrap samples. The description given matches bagging as applied in Random Forests, making that the best answer.

Repeatedly sampling data and training on those samples to improve stability is the bagging idea, bootstrap aggregating. In bagging, you create multiple bootstrap samples from the training data (sampling with replacement), train a separate model on each sample, and then combine their predictions. This process lowers variance because the individual models are trained on different portions of the data and, when you average or vote their outputs, the fluctuations of any single model tend to cancel out.

Random Forests implement this approach by growing many decision trees on different bootstrap samples and then averaging their predictions (or using majority vote for classification). They also introduce randomness in feature selection at each split, which further decorrelates the trees and boosts stability and generalization.

This differs from boosting methods, which train models sequentially to correct errors of earlier models and do not rely on training on multiple bootstrap samples. The description given matches bagging as applied in Random Forests, making that the best answer.