Which technique maps words to their base dictionary form using linguistic rules, often considering the word's part of speech?

Which technique maps words to their base dictionary form using linguistic rules, often considering the word's part of speech?

• A. Stemming
• B. Lemmatization
• C. Binary Bag of Words
• D. Feature Extraction

Answer: (B)

Lemmatization maps words to their base dictionary form using linguistic rules and often relies on the word’s part of speech. By analyzing morphology and consulting a lexicon, it selects the correct lemma for a given word in its context. For example, running as a verb becomes run, and better as an adjective can map toward good, depending on the POS. This POS-aware approach ensures you return a valid dictionary form, which helps normalize text for tasks like search or analysis. Stemming, in contrast, uses rough rule-based chopping of affixes and doesn’t guarantee real dictionary forms, so it can yield nonwords or overly aggressive reductions. Binary Bag of Words and Feature Extraction describe broader ways to represent text as features rather than the specific normalization to a canonical form.

Lemmatization maps words to their base dictionary form using linguistic rules and often relies on the word’s part of speech. By analyzing morphology and consulting a lexicon, it selects the correct lemma for a given word in its context. For example, running as a verb becomes run, and better as an adjective can map toward good, depending on the POS. This POS-aware approach ensures you return a valid dictionary form, which helps normalize text for tasks like search or analysis.

Stemming, in contrast, uses rough rule-based chopping of affixes and doesn’t guarantee real dictionary forms, so it can yield nonwords or overly aggressive reductions. Binary Bag of Words and Feature Extraction describe broader ways to represent text as features rather than the specific normalization to a canonical form.