edu.northwestern.at.morphadorner

Class MorphAdornerUtils

java.lang.Object

edu.northwestern.at.morphadorner.MorphAdornerUtils

public class MorphAdornerUtils
extends java.lang.Object

Morphological Adorner Utilities.

Static utility methods used by MorphAdorner.

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.lang.Runtime	runTime Runtime system.
protected static java.util.regex.Matcher	underlineCapCapMatcher
protected static java.util.regex.Pattern	underlineCapCapPattern Pattern to match _CapCap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	MorphAdornerUtils() Allow overrides but not instantiation.

Method Summary

Methods

Modifier and Type	Method and Description
static int	countPageBreaks(org.w3c.dom.Document document) Count page breaks in a document.
static NameStandardizer	createNameStandardizer(Lexicon wordLexicon, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Create proper name standardizer.
static SpellingMapper	createSpellingMapper(UTF8Properties properties) Create spelling mapper.
static SpellingStandardizer	createSpellingStandardizer(Lexicon wordLexicon, Names names, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Create spelling standardizer.
static java.lang.String	durationString(MorphAdornerSettings adornerSettings, long startTime) Get duration value for display.
static void	fixEmptySoftTags(XGOptions xgOptions, org.w3c.dom.Document document) Fix empty soft tags.
static java.lang.String	fixSpelling(java.lang.String spelling) Fix spelling.
static void	fixSupTags(org.w3c.dom.Document document) Fix sup tags.
static TaggedStrings	getExtraWordsList(java.lang.String wordFileName, java.lang.String posTag, java.lang.String loadedMessage, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Get extra words list.
static java.lang.String[]	getKWIC(java.util.List<AdornedWord> sentence, int wordIndex, int KWICWidth) Generate a KWIC line for a word in a sentence.
static java.lang.String	getLemma(MorphAdorner adorner, java.lang.String spelling, java.lang.String partOfSpeech) Get lemma (possibly compound) for a spelling.
protected static java.lang.String	getStandardizedSpelling(MorphAdorner adorner, java.lang.String correctedSpelling, java.lang.String standardizedSpelling, java.lang.String partOfSpeech) Get standardized spelling.
static int[]	getWordAndSentenceCounts(java.util.List<java.util.List<java.lang.String>> sentences) Get actual word and sentence count.
static int	getWordCount(java.util.List<java.util.List<java.lang.String>> sentences) Get count of words in a list of sentences.
static TaggedStrings	getWordList(java.lang.String wordFileName, java.lang.String posTag, java.lang.String loadedMessage, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Get word list.
static boolean	isAdorned(java.lang.String xmlFileName, int maxLinesToCheck) Check if file is already adorned.
static Lexicon	loadSuffixLexicon(MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Loads the suffix lexicon.
static void	loadTaggerRules(PartOfSpeechTagger tagger, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Loads part of speech tagger rules.
static TransitionMatrix	loadTransitionMatrix(PartOfSpeechTagger tagger, MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Loads the transition matrix.
static Lexicon	loadWordLexicon(MorphAdornerSettings adornerSettings, MorphAdornerLogger adornerLogger) Loads the word lexicon.
static void	logMemoryUsage(MorphAdornerLogger adornerLogger, java.lang.String label) Log current memory usage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

underlineCapCapPattern

protected static java.util.regex.Pattern underlineCapCapPattern

Pattern to match _CapCap

underlineCapCapMatcher

protected static final java.util.regex.Matcher underlineCapCapMatcher

runTime

protected static java.lang.Runtime runTime

Runtime system.

Constructor Detail

MorphAdornerUtils

protected MorphAdornerUtils()

Allow overrides but not instantiation.

Method Detail

countPageBreaks

public static int countPageBreaks(org.w3c.dom.Document document)

Count page breaks in a document.

Parameters:

document - The DOM document.

Returns:

The number of page break elements.

createSpellingMapper

public static SpellingMapper createSpellingMapper(UTF8Properties properties)
                                           throws java.io.IOException

Create spelling mapper.

Parameters:

properties - MorphAdorner properties.

Returns:

The spelling mapper.

Throws:

java.io.IOException

createNameStandardizer

public static NameStandardizer createNameStandardizer(Lexicon wordLexicon,
                                      MorphAdornerSettings adornerSettings,
                                      MorphAdornerLogger adornerLogger)
                                               throws java.io.IOException

Create proper name standardizer.

Parameters:

wordLexicon - The word lexicon containing names.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

The name standardizer.

Throws:

java.io.IOException

loadWordLexicon

public static Lexicon loadWordLexicon(MorphAdornerSettings adornerSettings,
                      MorphAdornerLogger adornerLogger)
                               throws java.io.IOException

Loads the word lexicon.

Parameters:

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

The word lexicon.

Throws:

java.io.IOException

loadSuffixLexicon

public static Lexicon loadSuffixLexicon(MorphAdornerSettings adornerSettings,
                        MorphAdornerLogger adornerLogger)
                                 throws java.io.IOException

Loads the suffix lexicon.

Parameters:

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

The suffix lexicon.

Throws:

java.io.IOException

loadTransitionMatrix

public static TransitionMatrix loadTransitionMatrix(PartOfSpeechTagger tagger,
                                    MorphAdornerSettings adornerSettings,
                                    MorphAdornerLogger adornerLogger)
                                             throws java.io.IOException

Loads the transition matrix.

Parameters:

tagger - Part of speech tagger.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

The transition matrix.

Throws:

java.io.IOException

loadTaggerRules

public static void loadTaggerRules(PartOfSpeechTagger tagger,
                   MorphAdornerSettings adornerSettings,
                   MorphAdornerLogger adornerLogger)
                            throws InvalidRuleException,
                                   java.io.IOException

Loads part of speech tagger rules.

Parameters:

tagger - Part of speech tagger.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Throws:

InvalidRuleException

java.io.IOException

createSpellingStandardizer

public static SpellingStandardizer createSpellingStandardizer(Lexicon wordLexicon,
                                              Names names,
                                              MorphAdornerSettings adornerSettings,
                                              MorphAdornerLogger adornerLogger)
                                                       throws java.io.IOException

Create spelling standardizer.

Parameters:

wordLexicon - The word lexicon.

names - The names list.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

The spelling standardizer.

Throws:

java.io.IOException

durationString

public static java.lang.String durationString(MorphAdornerSettings adornerSettings,
                              long startTime)

Get duration value for display.

Parameters:

adornerSettings - Adorner settings.

startTime - Start time.

Returns:

Duration value for display.

fixEmptySoftTags

public static void fixEmptySoftTags(XGOptions xgOptions,
                    org.w3c.dom.Document document)

Fix empty soft tags.

Parameters:

xgOptions - XML parsing options.

document - The DOM document.

On exit, the DOM document has empty soft tags expanded with a single blank as text, except tags. Empty tags get the special end of text section character, which prevents formation of bogus split words over gaps.

fixSpelling

public static java.lang.String fixSpelling(java.lang.String spelling)

Fix spelling.

Parameters:

spelling - Original spelling.

Returns:

The fixed spelling.

fixSupTags

public static void fixSupTags(org.w3c.dom.Document document)

Fix sup tags.

Parameters:

document - The DOM document.

Prepends a special marker character to the start of the text enclosed in ^{tags to allow disambiguation of old printer's abbreviations from other types of abbreviations. The special marker character is removed before the adorned XML text is written out.}

^E the y^T that y^c the y^e the y^en then y^ere there y^f if yⁱ thy y^m them yⁿ than y^o the y^t that y^u thou y that w^ch which w^t with

getLemma

public static java.lang.String getLemma(MorphAdorner adorner,
                        java.lang.String spelling,
                        java.lang.String partOfSpeech)

Get lemma (possibly compound) for a spelling.

Parameters:

adorner - The adorner.

spelling - The spelling.

partOfSpeech - The part of speech tag.

Returns:

Lemma for spelling. May contain compound spelling in form "lemma1:lemma2:...".

getStandardizedSpelling

protected static java.lang.String getStandardizedSpelling(MorphAdorner adorner,
                                       java.lang.String correctedSpelling,
                                       java.lang.String standardizedSpelling,
                                       java.lang.String partOfSpeech)

Get standardized spelling.

Parameters:

adorner - Adorner.

correctedSpelling - The spelling.

standardizedSpelling - The initial standardized spelling.

partOfSpeech - The part of speech tag.

Returns:

Standardized spelling.

getKWIC

public static java.lang.String[] getKWIC(java.util.List<AdornedWord> sentence,
                         int wordIndex,
                         int KWICWidth)

Generate a KWIC line for a word in a sentence.

Parameters:

sentence - The sentence as an array list.

wordIndex - The index of the word for which to generate a KWIC.

KWICWidth - Maximum width (in characters) of KWIC text.

Returns:

The KWIC sections as a String array. [0] = left KWIC text [1] = word [2] = right KWIC text

getWordCount

public static int getWordCount(java.util.List<java.util.List<java.lang.String>> sentences)

Get count of words in a list of sentences.

Parameters:

sentences - List of sentences each containing list of words.

Returns:

Total number of words in sentences.

getWordAndSentenceCounts

public static int[] getWordAndSentenceCounts(java.util.List<java.util.List<java.lang.String>> sentences)

Get actual word and sentence count.

Parameters:

sentences - List of sentences each containing list of words.

Returns:

Two word int array of sentence and word counts. [0] = # of sentences [1] = # of words

Sentences and words containing only the special separator marker character are not counted.

getWordList

public static TaggedStrings getWordList(java.lang.String wordFileName,
                        java.lang.String posTag,
                        java.lang.String loadedMessage,
                        MorphAdornerSettings adornerSettings,
                        MorphAdornerLogger adornerLogger)

Get word list.

Parameters:

wordFileName - File name of word list.

posTag - Part of speech tag for each word.

loadedMessage - Message to display when words loaded.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

Tagged strings with words.

getExtraWordsList

public static TaggedStrings getExtraWordsList(java.lang.String wordFileName,
                              java.lang.String posTag,
                              java.lang.String loadedMessage,
                              MorphAdornerSettings adornerSettings,
                              MorphAdornerLogger adornerLogger)

Get extra words list.

Parameters:

wordFileName - File name of word list.

posTag - Part of speech tag for each word.

loadedMessage - Message to display when words loaded.

adornerSettings - The adorner settings.

adornerLogger - The adorner logger.

Returns:

Tagged strings with words.

isAdorned

public static boolean isAdorned(java.lang.String xmlFileName,
                int maxLinesToCheck)

Check if file is already adorned.

Parameters:

xmlFileName - File to check for being adorned.

maxLinesToCheck - Maximum # of lines to read looking for a "

logMemoryUsage

public static void logMemoryUsage(MorphAdornerLogger adornerLogger,
                  java.lang.String label)

Log current memory usage.

Parameters:

adornerLogger - The adorner logger.

label - Label for memory usage.