AbstractPartOfSpeechGuesser (MorphAdorner)

java.lang.Object
- edu.northwestern.at.utils.IsCloseableObject
- - edu.northwestern.at.morphadorner.corpuslinguistics.postagger.guesser.AbstractPartOfSpeechGuesser

All Implemented Interfaces:

PartOfSpeechGuesser, UsesLogger

Direct Known Subclasses:

DefaultPartOfSpeechGuesser
```
public abstract class AbstractPartOfSpeechGuesser
extends IsCloseableObject
implements PartOfSpeechGuesser, UsesLogger
```
Abstract PartOfSpeechGuesser.
A part of speech guesser "guesses" the probable part(s) of speech for a word which does not appear in the main lexicon. Alternate spellings, lexical rules based upon word prefixes or suffixes, and many other approaches may be used to find potential part of speech. This AbstractPartOfSpeechGuesser holds the references to a word lexicon, suffix lexicon, and spelling standardizer. Subclasses must override the abstract method guessPartsOfSpeech.

Some of the heuristics here only work reliably for English language text.

Field Summary

Fields
Modifier and Type	Field and Description
`protected Abbreviations`	`abbreviations` Abbreviations.
`protected SpellingStandardizer`	`auxiliarySpellingStandardizer` The auxiliary spelling standardizer.
`protected java.util.List<TaggedStrings>`	`auxiliaryWordLists` Auxiliary word lists.
`protected java.util.Map<java.lang.String,Lexicon>`	`cachedLexicons` Cache lexicon for unknown words.
`protected Cache<java.lang.String,java.util.Map<java.lang.String,MutableInteger>>`	`cachedWords` Cache parts of speech for unknown words.
`protected boolean`	`checkPossessives` Check for possessives of known nouns when guessing parts of speech.
`protected boolean`	`debug` True to enable debugging output.
`protected Logger`	`logger` Logger used for output.
`protected Names`	`names` Proper names.
`protected SpellingStandardizer`	`spellingStandardizer` The principal spelling standardizer.
`protected Lexicon`	`suffixLexicon` The affix/suffix lexicon.
`protected boolean`	`tryStandardSpellings` Try standardized spellings when guessing parts of speech.
`protected Lexicon`	`wordLexicon` The word lexicon.

Constructor Summary

Constructors
Constructor and Description

AbstractPartOfSpeechGuesser()

Constructors
Constructor and Description
`AbstractPartOfSpeechGuesser()`

Method Summary

Methods
Modifier and Type	Method and Description
`void`	`addAuxiliaryWordList(TaggedStrings wordList)` Add an auxiliary word list.
`protected void`	`addCachedWord(java.lang.String word, java.util.Map<java.lang.String,MutableInteger> tagMap)` Add word to cache.
`java.util.Map<java.lang.String,MutableInteger>`	`checkAbbreviation(java.lang.String word)` Check if word is abbreviation.
`java.util.Map<java.lang.String,MutableInteger>`	`checkAllUpperCase(java.lang.String word)` See if word is all uppercase.
`java.util.Map<java.lang.String,MutableInteger>`	`checkAuxiliaryWordLists(java.lang.String word)` See if word is defined in an auxiliary word list.
`java.util.Map<java.lang.String,MutableInteger>`	`checkCachedWord(java.lang.String word)` See if we have part of speech for a cached word.
`java.util.Map<java.lang.String,MutableInteger>`	`checkCurrency(java.lang.String word)` See if word is currency value.
`java.util.Map<java.lang.String,MutableInteger>`	`checkHyphenatedWord(java.lang.String word)` Check if word contains a hyphen.
`java.util.Map<java.lang.String,MutableInteger>`	`checkName(java.lang.String word)` See if word is a name.
`java.util.Map<java.lang.String,MutableInteger>`	`checkNumber(java.lang.String word)` See if word is a number.
`java.util.Map<java.lang.String,MutableInteger>`	`checkPossessiveNoun(java.lang.String word)` See if word is a possessive noun.
`java.util.Map<java.lang.String,MutableInteger>`	`checkPunctuation(java.lang.String word)` Check if word is punctuation.
`java.util.Map<java.lang.String,MutableInteger>`	`checkRomanNumeral(java.lang.String word)` See if word is Roman numeral.
`protected java.util.Map<java.lang.String,MutableInteger>`	`checkStandardSpellings(java.lang.String word, java.lang.String[] standardSpellings)` Try to get tags using standardized spellings.
`java.util.Map<java.lang.String,MutableInteger>`	`checkSuffixes(java.lang.String word)` Try to get tags using suffix analysis.
`java.util.Map<java.lang.String,MutableInteger>`	`checkSuffixes(java.lang.String word, java.lang.String[] standardSpellings)` Try to get tags using suffix analysis.
`java.util.Map<java.lang.String,MutableInteger>`	`checkSymbol(java.lang.String word)` Check if word is symbol.
`protected java.util.Map<java.lang.String,MutableInteger>`	`clonePosTagMap(java.util.Map<java.lang.String,MutableInteger> posTagMap)` Clone pos tag map.
`java.util.List<TaggedStrings>`	`getAuxiliaryWordLists()` Get auxiliary word lists.
`Lexicon`	`getCachedLexiconForWord(java.lang.String word)` Get cached lexicon for a word.
`Logger`	`getLogger()` Get the logger.
`java.util.Map<java.lang.String,MutableInteger>`	`getNoun(java.lang.String word)` Get tags for a noun.
`SpellingStandardizer`	`getSpellingStandardizer()` Get spelling standardizer.
`protected java.lang.String[]`	`getStandardizedSpellings(java.lang.String word)` Get standardized spellings for a word.
`Lexicon`	`getSuffixLexicon()` Get the suffix lexicon.
`Lexicon`	`getWordLexicon()` Get the word lexicon.
`abstract java.util.Map<java.lang.String,MutableInteger>`	`guessPartsOfSpeech(java.lang.String word)` Guesses part of speech for a word.
`java.util.Map<java.lang.String,MutableInteger>`	`guessPartsOfSpeech(java.lang.String word, boolean isFirstWord)` Guesses part of speech for a word.
`protected java.util.Map<java.lang.String,MutableInteger>`	`posTagsToMap(java.lang.String[] posTags)` Create map from array of part of speech tags.
`protected java.util.Map<java.lang.String,MutableInteger>`	`posTagToMap(java.lang.String posTag)` Create map with one (pos, count) entry.
`void`	`removeCompoundTags(java.util.Map<java.lang.String,MutableInteger> posTagsMap)` Remove compound part of speech tags from tag map.
`void`	`removeProperNounTags(java.util.Map<java.lang.String,MutableInteger> posTagsMap)` Remove proper noun and proper adjective tags from tag map.
`void`	`setAbbreviations(Abbreviations abbreviations)` Set abbreviations.
`void`	`setCheckPossessives(boolean checkPossessives)` Check for possessives of known nouns when guessing parts of speech.
`void`	`setLogger(Logger logger)` Set the logger.
`void`	`setSpellingStandardizer(SpellingStandardizer spellingStandardizer)` Set spelling standardizer.
`void`	`setSuffixLexicon(Lexicon suffixLexicon)` Set the suffix lexicon.
`void`	`setTryStandardSpellings(boolean tryStandardSpellings)` Try using standardized spellings when guessing parts of speech.
`void`	`setWordLexicon(Lexicon wordLexicon)` Set the word lexicon.

Methods inherited from class edu.northwestern.at.utils.IsCloseableObject
close

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface edu.northwestern.at.morphadorner.corpuslinguistics.postagger.guesser.PartOfSpeechGuesser
guessPartsOfSpeech

- Field Detail
  - debug
```
protected boolean debug
```
    True to enable debugging output.
  - logger
```
protected Logger logger
```
    Logger used for output.
  - abbreviations
```
protected Abbreviations abbreviations
```
    Abbreviations.
  - cachedWords
```
protected Cache<java.lang.String,java.util.Map<java.lang.String,MutableInteger>> cachedWords
```
    Cache parts of speech for unknown words.
    The key is the word spelling, the value is a map of parts of speech and associated counts for the spelling.
  - cachedLexicons
```
protected java.util.Map<java.lang.String,Lexicon> cachedLexicons
```
    Cache lexicon for unknown words.
    The key is the word spelling, the value is the lexicon to use to retrieve counts for probability calculations. Normally a cache entry is only created when the lexicon is the suffix lexicon. The word lexicon is assumed by default otherwise.
  - wordLexicon
```
protected Lexicon wordLexicon
```
    The word lexicon.
  - suffixLexicon
```
protected Lexicon suffixLexicon
```
    The affix/suffix lexicon.
  - spellingStandardizer
```
protected SpellingStandardizer spellingStandardizer
```
    The principal spelling standardizer.
  - auxiliarySpellingStandardizer
```
protected SpellingStandardizer auxiliarySpellingStandardizer
```
    The auxiliary spelling standardizer.
  - auxiliaryWordLists
```
protected java.util.List<TaggedStrings> auxiliaryWordLists
```
    Auxiliary word lists.
  - names
```
protected Names names
```
    Proper names.
  - tryStandardSpellings
```
protected boolean tryStandardSpellings
```
    Try standardized spellings when guessing parts of speech.
  - checkPossessives
```
protected boolean checkPossessives
```
    Check for possessives of known nouns when guessing parts of speech.
- Constructor Detail
  - AbstractPartOfSpeechGuesser
```
public AbstractPartOfSpeechGuesser()
```
- Method Detail
  - getLogger
```
public Logger getLogger()
```
    Get the logger.
    
    Specified by:
    
    getLogger in interface UsesLogger
    
    Returns:
    The logger.
  - setLogger
```
public void setLogger(Logger logger)
```
    Set the logger.
    
    Specified by:
    
    setLogger in interface UsesLogger
    
    Parameters:
    logger - The logger.
  - getSpellingStandardizer
```
public SpellingStandardizer getSpellingStandardizer()
```
    Get spelling standardizer.
    
    Specified by:
    
    getSpellingStandardizer in interface PartOfSpeechGuesser
    
    Returns:
    The spelling standardizer.
  - setSpellingStandardizer
```
public void setSpellingStandardizer(SpellingStandardizer spellingStandardizer)
```
    Set spelling standardizer.
    
    Specified by:
    
    setSpellingStandardizer in interface PartOfSpeechGuesser
    
    Parameters:
    spellingStandardizer - The spelling standardizer.
  - setAbbreviations
```
public void setAbbreviations(Abbreviations abbreviations)
```
    Set abbreviations.
    
    Specified by:
    
    setAbbreviations in interface PartOfSpeechGuesser
    
    Parameters:
    abbreviations - Abbreviations.
  - getWordLexicon
```
public Lexicon getWordLexicon()
```
    Get the word lexicon.
    
    Specified by:
    
    getWordLexicon in interface PartOfSpeechGuesser
    
    Returns:
    The word lexicon.
  - setWordLexicon
```
public void setWordLexicon(Lexicon wordLexicon)
```
    Set the word lexicon.
    
    Specified by:
    
    setWordLexicon in interface PartOfSpeechGuesser
    
    Parameters:
    wordLexicon - The word lexicon.
  - getSuffixLexicon
```
public Lexicon getSuffixLexicon()
```
    Get the suffix lexicon.
    
    Specified by:
    
    getSuffixLexicon in interface PartOfSpeechGuesser
    
    Returns:
    The suffix lexicon.
  - setSuffixLexicon
```
public void setSuffixLexicon(Lexicon suffixLexicon)
```
    Set the suffix lexicon.
    
    Specified by:
    
    setSuffixLexicon in interface PartOfSpeechGuesser
    
    Parameters:
    suffixLexicon - The suffix lexicon.
  - addCachedWord
```
protected void addCachedWord(java.lang.String word,
                 java.util.Map<java.lang.String,MutableInteger> tagMap)
```
    Add word to cache.
    
    Parameters:
    word - The word.
    tagMap - Tag map for the word.
  - posTagToMap
```
protected java.util.Map<java.lang.String,MutableInteger> posTagToMap(java.lang.String posTag)
```
    Create map with one (pos, count) entry.
    
    Parameters:
    posTag - The part of speech tag.
    
    Returns:
    The map.
    The count associated with the part of speech tag is the count of that tag in the word lexicon.
  - posTagsToMap
```
protected java.util.Map<java.lang.String,MutableInteger> posTagsToMap(java.lang.String[] posTags)
```
    Create map from array of part of speech tags.
    
    Parameters:
    posTags - The part of speech tags.
    
    Returns:
    The map with tags as keys and counts as values.
    The count associated with the part of speech tags is the count for each tag in the word lexicon. This is probably not the best choice but absent any other information, it is at least consistent.
  - clonePosTagMap
```
protected java.util.Map<java.lang.String,MutableInteger> clonePosTagMap(java.util.Map<java.lang.String,MutableInteger> posTagMap)
```
    Clone pos tag map.
    
    Parameters:
    posTagMap - The pos tag map to clone.
    
    Returns:
    The cloned map.
  - addAuxiliaryWordList
```
public void addAuxiliaryWordList(TaggedStrings wordList)
```
    Add an auxiliary word list.
    
    Specified by:
    
    addAuxiliaryWordList in interface PartOfSpeechGuesser
  - getAuxiliaryWordLists
```
public java.util.List<TaggedStrings> getAuxiliaryWordLists()
```
    Get auxiliary word lists.
    
    Specified by:
    
    getAuxiliaryWordLists in interface PartOfSpeechGuesser
  - getCachedLexiconForWord
```
public Lexicon getCachedLexiconForWord(java.lang.String word)
```
    Get cached lexicon for a word.
    
    Specified by:
    
    getCachedLexiconForWord in interface PartOfSpeechGuesser
    
    Parameters:
    word - The word.
    
    Returns:
    The lexicon.
    Most words do not have an associated cached lexicon, so the word lexicon is returned. Words whose category counts result from a suffix analysis will have a cached entry pointing to the suffix lexicon.
  - checkCachedWord
```
public java.util.Map<java.lang.String,MutableInteger> checkCachedWord(java.lang.String word)
```
    See if we have part of speech for a cached word.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech tags if word found in cache, else null.
  - checkName
```
public java.util.Map<java.lang.String,MutableInteger> checkName(java.lang.String word)
```
    See if word is a name.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech if word is a name, else null.
    Note: Only capitalized versions of names are considered.
  - checkPossessiveNoun
```
public java.util.Map<java.lang.String,MutableInteger> checkPossessiveNoun(java.lang.String word)
```
    See if word is a possessive noun.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech if word is a possessive noun, else null.
  - checkAllUpperCase
```
public java.util.Map<java.lang.String,MutableInteger> checkAllUpperCase(java.lang.String word)
```
    See if word is all uppercase.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech if word is all uppercase, else null.
  - checkNumber
```
public java.util.Map<java.lang.String,MutableInteger> checkNumber(java.lang.String word)
```
    See if word is a number.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech if word is a number, else null.
  - checkCurrency
```
public java.util.Map<java.lang.String,MutableInteger> checkCurrency(java.lang.String word)
```
    See if word is currency value.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech if word is currency value, else null.
  - checkRomanNumeral
```
public java.util.Map<java.lang.String,MutableInteger> checkRomanNumeral(java.lang.String word)
```
    See if word is Roman numeral.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech tags if word is a Roman Numeral, else null.
  - checkAuxiliaryWordLists
```
public java.util.Map<java.lang.String,MutableInteger> checkAuxiliaryWordLists(java.lang.String word)
```
    See if word is defined in an auxiliary word list.
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech as an array of tags else null.
  - checkPunctuation
```
public java.util.Map<java.lang.String,MutableInteger> checkPunctuation(java.lang.String word)
```
    Check if word is punctuation.
    
    Parameters:
    word - The word.
    
    Returns:
    Punctuation part of speech tag or null if word is not punctuation.
  - checkSymbol
```
public java.util.Map<java.lang.String,MutableInteger> checkSymbol(java.lang.String word)
```
    Check if word is symbol.
    
    Parameters:
    word - The word.
    
    Returns:
    Symbol part of speech tag or null if word is not symbols.
  - checkAbbreviation
```
public java.util.Map<java.lang.String,MutableInteger> checkAbbreviation(java.lang.String word)
```
    Check if word is abbreviation.
    
    Parameters:
    word - The word.
    
    Returns:
    Noun or proper noun part of speech if word appears to be abbreviation.
    A proper noun tag is emitted when the abbreviation begins with a capital letter.
  - checkHyphenatedWord
```
public java.util.Map<java.lang.String,MutableInteger> checkHyphenatedWord(java.lang.String word)
```
    Check if word contains a hyphen.
    Parameters:
    word - The word.
    
    Returns:
    Part of speech.
    If word contains a dash, extract the part after the last dash. If that is a word in the lexicon, use its part of speech. Otherwise return with no part of speech assign and let the subsequent suffix analysis deal with the word.
    
    The following cases are treated specially.
    
    a letter followed by ---'s is considered a possessive noun.
    
    ---'s or ---'S is considered a possessive noun.
    
    a letter followed by --- is considered a proper or common noun, or an exclamation.
  - getStandardizedSpellings
```
protected java.lang.String[] getStandardizedSpellings(java.lang.String word)
```
    Get standardized spellings for a word.
    
    Parameters:
    word - The word.
    
    Returns:
    The standardized spellings.
  - checkStandardSpellings
```
protected java.util.Map<java.lang.String,MutableInteger> checkStandardSpellings(java.lang.String word,
                                                                    java.lang.String[] standardSpellings)
```
    Try to get tags using standardized spellings.
    
    Parameters:
    word - The word.
    standardSpellings - Standard spellings on output, or null if none.
    
    Returns:
    Parts of speech for standardized spellings, or null if not found.
  - removeProperNounTags
```
public void removeProperNounTags(java.util.Map<java.lang.String,MutableInteger> posTagsMap)
```
    Remove proper noun and proper adjective tags from tag map.
    
    Parameters:
    posTagsMap - Map of potential tags.
    Removes proper noun or proper adjective tags from tag map. Map is set to null if it becomes empty.
  - removeCompoundTags
```
public void removeCompoundTags(java.util.Map<java.lang.String,MutableInteger> posTagsMap)
```
    Remove compound part of speech tags from tag map.
    
    Parameters:
    posTagsMap - Map of potential tags.
    Removes compound part of speech tags from tag map. Map is set to null if it becomes empty.
  - checkSuffixes
```
public java.util.Map<java.lang.String,MutableInteger> checkSuffixes(java.lang.String word)
```
    Try to get tags using suffix analysis.
    
    Parameters:
    word - The word.
    
    Returns:
    Parts of speech from suffix analysis or null if not found.
  - checkSuffixes
```
public java.util.Map<java.lang.String,MutableInteger> checkSuffixes(java.lang.String word,
                                                           java.lang.String[] standardSpellings)
```
    Try to get tags using suffix analysis.
    
    Parameters:
    word - The word.
    standardSpellings - List of standard spellings, or null if none.
    
    Returns:
    Parts of speech from suffix analysis or null if not found.
  - getNoun
```
public java.util.Map<java.lang.String,MutableInteger> getNoun(java.lang.String word)
```
    Get tags for a noun.
    
    Parameters:
    word - The word.
    
    Returns:
    A noun type tag.
    A proper noun tag is emitted when the abbreviation begins with a capital letter. A plural noun tag is emitted when the word ends with an "s".
  - guessPartsOfSpeech
```
public java.util.Map<java.lang.String,MutableInteger> guessPartsOfSpeech(java.lang.String word,
                                                                boolean isFirstWord)
```
    Guesses part of speech for a word.
    
    Specified by:
    
    guessPartsOfSpeech in interface PartOfSpeechGuesser
    
    Parameters:
    word - The word.
    isFirstWord - If word is first word in a sentence.
    
    Returns:
    Guessed parts of speech as an array of tags.
  - setTryStandardSpellings
```
public void setTryStandardSpellings(boolean tryStandardSpellings)
```
    Try using standardized spellings when guessing parts of speech.
    
    Specified by:
    
    setTryStandardSpellings in interface PartOfSpeechGuesser
  - setCheckPossessives
```
public void setCheckPossessives(boolean checkPossessives)
```
    Check for possessives of known nouns when guessing parts of speech.
    
    Specified by:
    
    setCheckPossessives in interface PartOfSpeechGuesser
  - guessPartsOfSpeech
```
public abstract java.util.Map<java.lang.String,MutableInteger> guessPartsOfSpeech(java.lang.String word)
```
    Guesses part of speech for a word.
    
    Specified by:
    
    guessPartsOfSpeech in interface PartOfSpeechGuesser
    
    Parameters:
    word - The word.
    
    Returns:
    Guessed parts of speech.

Class AbstractPartOfSpeechGuesser

Field Summary

Constructor Summary

Method Summary

Methods inherited from class edu.northwestern.at.utils.IsCloseableObject

Methods inherited from class java.lang.Object

Methods inherited from interface edu.northwestern.at.morphadorner.corpuslinguistics.postagger.guesser.PartOfSpeechGuesser

Field Detail

debug

logger

abbreviations

cachedWords

cachedLexicons

wordLexicon

suffixLexicon

spellingStandardizer

auxiliarySpellingStandardizer

auxiliaryWordLists

names

tryStandardSpellings

checkPossessives

Constructor Detail

AbstractPartOfSpeechGuesser

Method Detail

getLogger

setLogger

getSpellingStandardizer

setSpellingStandardizer

setAbbreviations

getWordLexicon

setWordLexicon

getSuffixLexicon

setSuffixLexicon

addCachedWord

posTagToMap

posTagsToMap

clonePosTagMap

addAuxiliaryWordList

getAuxiliaryWordLists

getCachedLexiconForWord

checkCachedWord

checkName

checkPossessiveNoun

checkAllUpperCase

checkNumber

checkCurrency

checkRomanNumeral

checkAuxiliaryWordLists

checkPunctuation

checkSymbol

checkAbbreviation

checkHyphenatedWord

getStandardizedSpellings

checkStandardSpellings

removeProperNounTags

removeCompoundTags

checkSuffixes

checkSuffixes

getNoun

guessPartsOfSpeech

setTryStandardSpellings

setCheckPossessives

guessPartsOfSpeech