Class Token


  • public class Token
    extends java.lang.Object
    Contains a parsed token from a Tokenizer.
    • Constructor Summary

      Constructors 
      Constructor Description
      Token()  
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      int getLineNumber()
      Returns the line of this token in the original text.
      int getPosition()
      Returns the position of this token in the original input text.
      java.lang.String getPostpunctuation()
      Returns the postpunctuation characters of this Token.
      java.lang.String getPrepunctuation()
      Returns the prepunctuation characters of this Token.
      java.lang.String getWhitespace()
      Returns the whitespace characters of this Token.
      java.lang.String getWord()
      Returns the string associated with this token.
      void setLineNumber​(int lineNumber)
      Set the line of this token in the original text.
      void setPosition​(int position)
      Sets the position of the token in the original input text.
      void setPostpunctuation​(java.lang.String postpunctuation)
      Sets the postpunctuation characters of this Token.
      void setPrepunctuation​(java.lang.String prepunctuation)
      Sets the prepunctuation characters of this Token.
      void setWhitespace​(java.lang.String whitespace)
      Sets the whitespace characters of this Token.
      void setWord​(java.lang.String word)
      Sets the string of this Token.
      java.lang.String toString()
      Converts this token to a string.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
    • Constructor Detail

      • Token

        public Token()
    • Method Detail

      • getWhitespace

        public java.lang.String getWhitespace()
        Returns the whitespace characters of this Token.
        Returns:
        the whitespace characters of this Token; null if this Token does not use whitespace characters
      • getPrepunctuation

        public java.lang.String getPrepunctuation()
        Returns the prepunctuation characters of this Token.
        Returns:
        the prepunctuation characters of this Token; null if this Token does not use prepunctuation characters
      • getPostpunctuation

        public java.lang.String getPostpunctuation()
        Returns the postpunctuation characters of this Token.
        Returns:
        the postpunctuation characters of this Token; null if this Token does not use postpunctuation characters
      • getPosition

        public int getPosition()
        Returns the position of this token in the original input text.
        Returns:
        the position of this token in the original input text
      • getLineNumber

        public int getLineNumber()
        Returns the line of this token in the original text.
        Returns:
        the line of this token in the original text
      • setWhitespace

        public void setWhitespace​(java.lang.String whitespace)
        Sets the whitespace characters of this Token.
        Parameters:
        whitespace - the whitespace character for this token
      • setPrepunctuation

        public void setPrepunctuation​(java.lang.String prepunctuation)
        Sets the prepunctuation characters of this Token.
        Parameters:
        prepunctuation - the prepunctuation characters
      • setPostpunctuation

        public void setPostpunctuation​(java.lang.String postpunctuation)
        Sets the postpunctuation characters of this Token.
        Parameters:
        postpunctuation - the postpunctuation characters
      • setPosition

        public void setPosition​(int position)
        Sets the position of the token in the original input text.
        Parameters:
        position - the position of the input text
      • setLineNumber

        public void setLineNumber​(int lineNumber)
        Set the line of this token in the original text.
        Parameters:
        lineNumber - the line of this token in the original text
      • getWord

        public java.lang.String getWord()
        Returns the string associated with this token.
        Returns:
        the token if it exists; otherwise null
      • setWord

        public void setWord​(java.lang.String word)
        Sets the string of this Token.
        Parameters:
        word - the word for this token
      • toString

        public java.lang.String toString()
        Converts this token to a string.
        Overrides:
        toString in class java.lang.Object
        Returns:
        the string representation of this object