Which technique adjusts the learning rate over time to prevent overshooting?

Which technique adjusts the learning rate over time to prevent overshooting?

• A. Stochastic Gradient Descent
• B. Mini-Batch Gradient Descent
• C. Batch Gradient Descent
• D. Dynamic Learning Rate

Answer: (D)

The idea being tested is how the step size in optimization is changed as training progresses. A dynamic learning rate intentionally varies the learning rate over time, often starting larger to explore and then reducing the step size as you approach a minimum. This gradual decrease helps prevent overshooting—where updates are too large and push the parameters past the optimal point—by making smaller, more precise moves later in training. In contrast, the other approaches describe how much data you use to compute each gradient (single sample, small batch, or the full dataset) but don’t inherently adjust the update size over time. So, the technique that adjusts the learning rate during training to prevent overshooting is the dynamic learning rate.

The idea being tested is how the step size in optimization is changed as training progresses. A dynamic learning rate intentionally varies the learning rate over time, often starting larger to explore and then reducing the step size as you approach a minimum. This gradual decrease helps prevent overshooting—where updates are too large and push the parameters past the optimal point—by making smaller, more precise moves later in training. In contrast, the other approaches describe how much data you use to compute each gradient (single sample, small batch, or the full dataset) but don’t inherently adjust the update size over time. So, the technique that adjusts the learning rate during training to prevent overshooting is the dynamic learning rate.