public class ApostrophesAreNotQuotesWordTokenizer extends DefaultWordTokenizer implements WordTokenizer
abbreviations, aposTokens, apostropheCanBeQuote, coalesceAsterisks, coalesceHyphens, contractions, contractionsURL, hyphensMatcher, hyphensPattern, logger, preTokenizer| Constructor and Description |
|---|
ApostrophesAreNotQuotesWordTokenizer()
Create word tokenizer.
|
addWordToSentence, extractWordsfindWordOffsets, getLogger, getPreTokenizer, isClosingQuote, isLetterOrSingleQuote, isMultipleHyphens, isSingleOpeningQuote, loadContractions, preprocessToken, setAbbreviations, setAposTokens, setLogger, setPreTokenizer, splitTokencloseclone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitaddWordToSentence, close, extractWords, findWordOffsets, getPreTokenizer, preprocessToken, setAbbreviations, setAposTokens, setPreTokenizerclose