Which approach is typically used to determine actions directly by optimizing a policy over actions in a given state?

Which approach is typically used to determine actions directly by optimizing a policy over actions in a given state?

• A. Deep Q-learning
• B. Policy Gradient approach
• C. Ordinary Least Squares (OLS)
• D. Stochastic Gradient Descent

Answer: (B)

Directly optimizing the actions taken in each state is a hallmark of policy gradient methods. Here, the policy π(a|s; θ) is parameterized and the parameters θ are adjusted to maximize the expected return. The learning signal comes from gradients of the performance objective with respect to θ, so the policy itself is updated to assign higher probability to better actions in each state. This lets you continuously improve the chosen actions by tweaking the policy itself, which is especially powerful when actions are continuous or when you want a stochastic policy. Value-based approaches, like Deep Q-learning, learn a value function Q(s,a) and select actions by maximizing this value, rather than directly adjusting the action-selection mechanism. Ordinary Least Squares is simply a linear regression method and doesn’t address how to optimize actions in reinforcement learning. Stochastic Gradient Descent is a general optimization algorithm used in many contexts, but it’s not, by itself, the method that directly optimizes a policy over actions.

Directly optimizing the actions taken in each state is a hallmark of policy gradient methods. Here, the policy π(a|s; θ) is parameterized and the parameters θ are adjusted to maximize the expected return. The learning signal comes from gradients of the performance objective with respect to θ, so the policy itself is updated to assign higher probability to better actions in each state. This lets you continuously improve the chosen actions by tweaking the policy itself, which is especially powerful when actions are continuous or when you want a stochastic policy.

Value-based approaches, like Deep Q-learning, learn a value function Q(s,a) and select actions by maximizing this value, rather than directly adjusting the action-selection mechanism. Ordinary Least Squares is simply a linear regression method and doesn’t address how to optimize actions in reinforcement learning. Stochastic Gradient Descent is a general optimization algorithm used in many contexts, but it’s not, by itself, the method that directly optimizes a policy over actions.