Package edu.northwestern.at.morphadorner.tools.tcp

The tcp package contains utilities aimed at processing Text Creation Partnership texts.

See: Description

Class Summary

Class	Description
AddUnclear	AddUnclear adds type="unclear" attribute to tokens containing character gaps.
CountDividedWords	Count words containing divider characters.
ExtractSoftHyphens	Filter hyphenated words.
FindSoftHyphens	Determine which words containing soft hyphens should actually be hyphenated.
FixWordBreaks	Fix word breaks.
FixWordBreaks.WProcessor	Process an adorned word.
RemoveCruft	Remove long s, brace-enclosed entities, superscripts, etc.
SuperFixer	SuperFixer marks "^" characters with special tags.

Package edu.northwestern.at.morphadorner.tools.tcp Description

The tcp package contains utilities aimed at processing Text Creation Partnership texts.

• AddUnclear adds a type="unclear" attribute to word tokens which contain a character gap marker. Character gaps are indicated by the presence of unicode character \u25CF (the black circle) in a token.
• CountDividedWords counts word tokens which contain soft line break divider characters.
• FindSoftHyphens
• ExtractSoftHyphens
• FixWordBreaks
• RemoveCruft removes "cruft" such as long "s", superscript markers, and other TCP specific markup.