edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer

Class RuleBasedLemmatizer

java.lang.Object

edu.northwestern.at.utils.IsCloseableObject

edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer.AbstractLemmatizer

edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer.RuleBasedLemmatizer

All Implemented Interfaces:

Lemmatizer, UsesLogger

Direct Known Subclasses:

EnglishLemmatizer

public class RuleBasedLemmatizer
extends AbstractLemmatizer
implements Lemmatizer

A rule-based lemmatizer.

Field Summary

Fields

Modifier and Type	Field and Description
protected Map2D<java.lang.String,java.lang.String,java.lang.String>	irregularForms Irregular forms.
protected java.util.Set<java.lang.String>	irregularFormsWordClasses Word classes of irregular forms.
protected java.util.Map<java.lang.String,java.util.List<LemmatizerRule>>	rules Lemmatizing rules.
protected java.util.Set<java.lang.String>	rulesWordClasses Word classes covered by rules.

Fields inherited from class edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer.AbstractLemmatizer

dictionary, lemmaSeparator, lemmaSeparatorString, lexicon, logger

Constructor Summary

Constructors

Constructor and Description
RuleBasedLemmatizer() Create a rule-based lemmatizer.

Method Summary

Methods

Modifier and Type	Method and Description
java.lang.String	cleanUpLemma(java.lang.String lemma) Clean up lemma.
java.lang.String	lemmatize(java.lang.String spelling) Returns a lemma given a spelling.
java.lang.String	lemmatize(java.lang.String spelling, java.lang.String wordClass) Returns a lemma given a word and a word class.
void	loadIrregularForms(java.net.URL url, java.lang.String encoding) Loads irregular forms from a URL.
void	loadRules(java.net.URL url, java.lang.String encoding) Loads lemmatization rules from a URL.

Methods inherited from class edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer.AbstractLemmatizer

cantLemmatize, countLemmata, getLemmaSeparator, getLogger, isCompoundLemma, joinLemmata, joinLemmata, setDictionary, setLexicon, setLogger, splitLemma

Methods inherited from class edu.northwestern.at.utils.IsCloseableObject

close

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.northwestern.at.morphadorner.corpuslinguistics.lemmatizer.Lemmatizer

cantLemmatize, countLemmata, getLemmaSeparator, isCompoundLemma, joinLemmata, joinLemmata, setDictionary, setLexicon, splitLemma

Field Detail

irregularForms

protected Map2D<java.lang.String,java.lang.String,java.lang.String> irregularForms

Irregular forms.

Irregular forms are stored in a HashMap2D. The compound key consists of the word class and irregular word form, and the value is the lemma.

irregularFormsWordClasses

protected java.util.Set<java.lang.String> irregularFormsWordClasses

Word classes of irregular forms.

rules

protected java.util.Map<java.lang.String,java.util.List<LemmatizerRule>> rules

Lemmatizing rules.

The rules are stored in a map with the word class as a key and a list of LemmatizerRule entries as the value.

rulesWordClasses

protected java.util.Set<java.lang.String> rulesWordClasses

Word classes covered by rules.

Constructor Detail

RuleBasedLemmatizer

public RuleBasedLemmatizer()
                    throws java.lang.Exception

Create a rule-based lemmatizer.

Throws:

java.lang.Exception

Method Detail

loadRules

public void loadRules(java.net.URL url,
             java.lang.String encoding)
               throws java.io.IOException

Loads lemmatization rules from a URL.

Parameters:

url - URL containing lemmatization rules.

encoding - Character set encoding for rules.

Throws:

java.io.IOException

loadIrregularForms

public void loadIrregularForms(java.net.URL url,
                      java.lang.String encoding)
                        throws java.io.IOException

Loads irregular forms from a URL.

Parameters:

url - URL containing irregular forms.

encoding - Character set encoding for irregular forms.

Throws:

java.io.IOException

lemmatize

public java.lang.String lemmatize(java.lang.String spelling,
                         java.lang.String wordClass)

Returns a lemma given a word and a word class.

Specified by:

lemmatize in interface Lemmatizer

Specified by:

lemmatize in class AbstractLemmatizer

Parameters:

spelling - The spelling.

wordClass - The word class. Ignored if null or empty. May contain more than one word class separated by commas, in which case the lemma rules for each class are applied in order.

Returns:

The lemma. The spelling is returned when a lemma cannot be found.

cleanUpLemma

public java.lang.String cleanUpLemma(java.lang.String lemma)

Clean up lemma.

Parameters:

lemma - The lemma to clean.

Returns:

The clean lemma.

A lemma may contain extraneous "!" characters added to ensure a specific ending is retained. The "!" marks are removed here.

lemmatize

public java.lang.String lemmatize(java.lang.String spelling)

Returns a lemma given a spelling.

Specified by:

lemmatize in interface Lemmatizer

Specified by:

lemmatize in class AbstractLemmatizer

Parameters:

spelling - The spelling.

Returns:

The lemma. The spelling is returned if a lemma cannot be found.