Let the tree grow fully, then remove weak parts

Let the tree grow fully, then remove weak parts

• A. Pre-Pruning
• B. Cost complexity pruning
• C. Post-Pruning
• D. Reduced error pruning

Answer: (C)

Growing the tree to full depth captures as much structure in the data as possible, including noise. Post-pruning uses that fully grown tree and then trims away branches that don’t improve performance on unseen data, yielding a simpler model that generalizes better. This approach often relies on a separate pruning or validation set to decide which parts to remove, so the final tree maintains accuracy while reducing overfitting. Pre-pruning would stop growing early, which can underfit because the tree never explores deeper splits. Cost complexity pruning is a specific post-pruning method that adds a penalty for tree size and prunes subtrees accordingly; it’s still about pruning after growth, but with a formal criterion. Reduced error pruning is another post-pruning technique that uses a pruning set to determine whether replacing a subtree with a leaf lowers validation error. The description given aligns with the general idea of post-pruning.

Growing the tree to full depth captures as much structure in the data as possible, including noise. Post-pruning uses that fully grown tree and then trims away branches that don’t improve performance on unseen data, yielding a simpler model that generalizes better. This approach often relies on a separate pruning or validation set to decide which parts to remove, so the final tree maintains accuracy while reducing overfitting.

Pre-pruning would stop growing early, which can underfit because the tree never explores deeper splits. Cost complexity pruning is a specific post-pruning method that adds a penalty for tree size and prunes subtrees accordingly; it’s still about pruning after growth, but with a formal criterion. Reduced error pruning is another post-pruning technique that uses a pruning set to determine whether replacing a subtree with a leaf lowers validation error. The description given aligns with the general idea of post-pruning.