public class PunktToken
extends java.lang.Object
Modifier and Type | Field and Description |
---|---|
protected int |
startPosition
The starting position of the token.
|
protected java.lang.String |
tokenText
The text of the token.
|
protected PunktTokenType |
tokenType
The Punkt token type.
|
Constructor and Description |
---|
PunktToken(java.lang.String tokenText,
int start,
PunktTokenType tokenType)
Create a Punkt token.
|
PunktToken(java.lang.String tokenText,
PunktTokenType tokenType)
Create a Punkt token.
|
Modifier and Type | Method and Description |
---|---|
int |
getEndPosition()
Get the ending position of the token.
|
int |
getLength()
Get the length of the token.
|
java.lang.String |
getSingleCharCode()
Get the character code for a single character token.
|
int |
getStartPosition()
Get the starting position of the token.
|
java.lang.String |
getTokenText()
Get the token text.
|
PunktTokenType |
getTokenType()
Get the token type.
|
java.lang.String |
toString()
Return token as a string.
|
protected java.lang.String tokenText
protected int startPosition
protected PunktTokenType tokenType
public PunktToken(java.lang.String tokenText, int start, PunktTokenType tokenType)
tokenText
- The text of the token.start
- Starting position of the token.tokenType
- The type of the token.public PunktToken(java.lang.String tokenText, PunktTokenType tokenType)
tokenText
- The text of the token.tokenType
- The type of the token.public java.lang.String getTokenText()
public PunktTokenType getTokenType()
public int getStartPosition()
public int getEndPosition()
public java.lang.String getSingleCharCode()
public int getLength()
public java.lang.String toString()
toString
in class java.lang.Object