In reinforcement learning, which problem involves choosing among multiple arms with uncertain rewards to maximize cumulative gain?

In reinforcement learning, which problem involves choosing among multiple arms with uncertain rewards to maximize cumulative gain?

• A. Policy-based approach
• B. Monte Carlo method
• C. Temporal Difference (TD) method
• D. Multi-Arm Bandit Problem

Answer: (D)

The main idea here is making a sequence of choices when each option (arm) gives a reward with unknown value, with the goal of maximizing the total reward over time. This setup is the multi-armed bandit problem. Each arm has a stochastic reward distribution that you don’t know upfront, so you must learn which arms are better while you’re pulling them. The challenge is balancing exploration (trying different arms to learn their rewards) with exploitation (pulling the arm that currently seems best) to maximize cumulative gain. This fits exactly because there are no state transitions or complex environment dynamics to worry about—just a series of arm pulls and their rewards. The other items are categories of methods used to solve broader reinforcement learning tasks: policy-based approaches focus on learning a mapping from states to actions, Monte Carlo methods estimate returns by sampling complete episodes, and Temporal Difference methods update value estimates using bootstrapping. They’re techniques within RL, not the specific problem of choosing among uncertain actions to maximize cumulative reward.

The main idea here is making a sequence of choices when each option (arm) gives a reward with unknown value, with the goal of maximizing the total reward over time. This setup is the multi-armed bandit problem. Each arm has a stochastic reward distribution that you don’t know upfront, so you must learn which arms are better while you’re pulling them. The challenge is balancing exploration (trying different arms to learn their rewards) with exploitation (pulling the arm that currently seems best) to maximize cumulative gain.

This fits exactly because there are no state transitions or complex environment dynamics to worry about—just a series of arm pulls and their rewards. The other items are categories of methods used to solve broader reinforcement learning tasks: policy-based approaches focus on learning a mapping from states to actions, Monte Carlo methods estimate returns by sampling complete episodes, and Temporal Difference methods update value estimates using bootstrapping. They’re techniques within RL, not the specific problem of choosing among uncertain actions to maximize cumulative reward.