edu.northwestern.at.morphadorner.tools.stripwordattributes

Class StripWordAttributes

java.lang.Object

edu.northwestern.at.morphadorner.tools.stripwordattributes.StripWordAttributes

public class StripWordAttributes
extends java.lang.Object

Create derived MorphAdorner file with word elements stripped of attributes.

Usage:

  java edu.northwestern.at.morphadorner.tools.stripwordattributes.StripWordAttributes input.xml output.xml output.tab [/[no]id] [/[no]trim]
  

input.xml	Input MorphAdornerd xml file.
output.xml	Derived adorned file with word element attributes stripped.
output.tab	Tab delimited file of word element attribute values.
/id or /noid	Optional parameter indicating xml:id should be left attached to each word (<w>) element. Default is /noid which removes the xml:id attribute and value.
/trim or /notrim	Optional parameter indicating whether whitespace should be trimmed from the start and end of each XML text line. Default is /notrim, which leaves the original whitespace intact.

The derived adorned output file "output.xml" has all attributes stripped from each <w> tag.

The attribute values for each "<w>" element in the input.xml file are extracted and output to the tab-separated values output.tab file. The order of the attribute lines matches the order of appearance of the <w> elements in the XML output file. When /id is used, the xml:id value in each <w> element in output.xml can be matched with the corresponding xml:id value in output.tab .

The first line in output.tab contains the attribute names for each column. Each subsequent line in the output.tab file contains at least the following information corresponding to a single word "<w>" element. Some adorned files may add extra word attributes, resulting in more columns.

1. xml:id -- the permanent word ID.
2. eos -- the end of sentence flag (1 if word ends a sentence, 0 otherwise)
3. lem -- the lemma.
4. ord -- the word ordinal within the text (starts at 1)
5. part -- the word part flag. "N" for a word which is not split; "I" for the first part of a split word; "M" for the middle parts of a split word; and "F" for the final part of a split word.
6. pos -- the part of speech.
7. reg -- the standard spelling.
8. spe -- the corrected original spelling.
9. tok -- The original token.

Field Summary

Fields

Modifier and Type	Field and Description
protected static java.util.Set<java.lang.String>	attrsToOmit Attributes to omit from output attributes file.
protected static java.util.Map<java.lang.String,java.lang.String>	entitiesMap Standard entities generated by MorphAdorner.
protected static java.util.regex.Matcher	entitiesMatcher Entities pattern matcher.
protected static java.util.regex.Pattern	entitiesPattern Entities pattern.
protected static java.lang.String	LINE_SEPARATOR Line separator.

Constructor Summary

Constructors

Constructor and Description
StripWordAttributes(java.lang.String inputXMLFileName, java.lang.String outputXMLFileName, java.lang.String outputTabFileName, boolean leaveID, boolean trimWhitespace) Create derived adorned files with character offset attributes.

Method Summary

Methods

Modifier and Type	Method and Description
protected static java.lang.String	cleanAttributeValue(java.lang.String attrValue) Cleans an attribute value of enclosing quotes and internal entities.
protected static void	displayUsage() Display program usage.
protected static java.util.Map<java.lang.String,java.lang.String>	fillInMissingAttributes(java.util.Map<java.lang.String,java.lang.String> attrMap, java.lang.String wordText) Fill in missing attribute values.
protected static java.util.Map<java.lang.String,java.lang.String>	getAttributes(java.lang.String attrsText, java.lang.String wordText) Get map of attribute values for a element.
static void	main(java.lang.String[] args) Main program.
protected static void	setMissingValue(java.util.Map<java.lang.String,java.lang.String> attrMap, java.lang.String attrName, java.lang.String defaultAttrValue) Set missing attribute value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LINE_SEPARATOR

protected static final java.lang.String LINE_SEPARATOR

Line separator.

attrsToOmit

protected static java.util.Set<java.lang.String> attrsToOmit

Attributes to omit from output attributes file.

entitiesMap

protected static java.util.Map<java.lang.String,java.lang.String> entitiesMap

Standard entities generated by MorphAdorner.

entitiesPattern

protected static java.util.regex.Pattern entitiesPattern

Entities pattern.

entitiesMatcher

protected static java.util.regex.Matcher entitiesMatcher

Entities pattern matcher.

Constructor Detail

StripWordAttributes

public StripWordAttributes(java.lang.String inputXMLFileName,
                   java.lang.String outputXMLFileName,
                   java.lang.String outputTabFileName,
                   boolean leaveID,
                   boolean trimWhitespace)

Create derived adorned files with character offset attributes.

Parameters:

inputXMLFileName - Input adorned XML file name.

outputXMLFileName - Output modified XML file name.

outputTabFileName - Output attribute values file name.

leaveID - true to leave xml:id in output elements.

trimWhitespace - true to trim whitespace from start and end of input XML lines.

Method Detail

main

public static void main(java.lang.String[] args)

Main program.

displayUsage

protected static void displayUsage()

Display program usage.

getAttributes

protected static java.util.Map<java.lang.String,java.lang.String> getAttributes(java.lang.String attrsText,
                                                             java.lang.String wordText)

Get map of attribute values for a element.

Parameters:

attrsText - String containing attribute values in attr="value" form as extracted from element.

wordText - Word text.

Returns:

Map with attribute name as key and attribute value as value.

cleanAttributeValue

protected static java.lang.String cleanAttributeValue(java.lang.String attrValue)

Cleans an attribute value of enclosing quotes and internal entities.

Parameters:

attrValue - Input attribute value with possible enclosing quotes and internal entities.

Returns:

Attribute value with enclosing quotes removed and internal entities resolved to their character equivalents.

fillInMissingAttributes

protected static java.util.Map<java.lang.String,java.lang.String> fillInMissingAttributes(java.util.Map<java.lang.String,java.lang.String> attrMap,
                                                                       java.lang.String wordText)

Fill in missing attribute values.

Parameters:

attrMap - Map with attribute name -> attribute value entries.

wordText - Word text.

Returns:

Updated map with missing/default entries filled in.

setMissingValue

protected static void setMissingValue(java.util.Map<java.lang.String,java.lang.String> attrMap,
                   java.lang.String attrName,
                   java.lang.String defaultAttrValue)

Set missing attribute value.

Parameters:

attrMap - Map with attribute name -> attribute value entries.

attrName - Attribute name.

defaultAttrValue - Default attribute value for attribute attrName if not present in attrMap.