Instead of all data (batch) or one item (stochastic), you use small chunks.

Instead of all data (batch) or one item (stochastic), you use small chunks.

• A. Stochastic Gradient Descent
• B. Mini-Batch Gradient Descent
• C. Batch Gradient Descent
• D. Dynamic Learning Rate

Answer: (B)

Using small chunks of data to estimate the gradient is the approach here. It blends the stability of using the full dataset with the speed of updating from a single example. With mini-batches, you compute the gradient over a small subset of the training data, update the parameters, and repeat. This reduces the computational burden compared with using the entire dataset, while the gradient estimate is less noisy than using just one example, leading to faster and more frequent updates with smoother progress. It also fits well with vectorized hardware like GPUs because you can perform the operations on a batch all at once. Common batch sizes are in the tens to hundreds, such as 32, 64, or 128. So, when you don’t use all data at once or a single item, you’re using mini-batch gradient descent.

Using small chunks of data to estimate the gradient is the approach here. It blends the stability of using the full dataset with the speed of updating from a single example. With mini-batches, you compute the gradient over a small subset of the training data, update the parameters, and repeat. This reduces the computational burden compared with using the entire dataset, while the gradient estimate is less noisy than using just one example, leading to faster and more frequent updates with smoother progress. It also fits well with vectorized hardware like GPUs because you can perform the operations on a batch all at once. Common batch sizes are in the tens to hundreds, such as 32, 64, or 128. So, when you don’t use all data at once or a single item, you’re using mini-batch gradient descent.