|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||
java.lang.Objectweka.core.tokenizers.Tokenizer
public abstract class Tokenizer
A superclass for all tokenizer algorithms.
| Constructor Summary | |
|---|---|
Tokenizer()
|
|
| Method Summary | |
|---|---|
java.lang.String[] |
getOptions()
Gets the current option settings for the OptionHandler. |
abstract java.lang.String |
globalInfo()
Returns a string describing the stemmer |
abstract boolean |
hasMoreElements()
Tests if this enumeration contains more elements. |
java.util.Enumeration |
listOptions()
Returns an enumeration of all the available options.. |
abstract java.lang.Object |
nextElement()
Returns the next element of this enumeration if this enumeration object has at least one more element to provide. |
static void |
runTokenizer(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. |
void |
setOptions(java.lang.String[] options)
Sets the OptionHandler's options using the given list. |
abstract void |
tokenize(java.lang.String s)
Sets the string to tokenize. |
static java.lang.String[] |
tokenize(Tokenizer tokenizer,
java.lang.String[] options)
initializes the given tokenizer with the given options and runs the tokenizer over all the remaining strings in the options array. |
| Methods inherited from class java.lang.Object |
|---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
| Methods inherited from interface weka.core.RevisionHandler |
|---|
getRevision |
| Constructor Detail |
|---|
public Tokenizer()
| Method Detail |
|---|
public abstract java.lang.String globalInfo()
public java.util.Enumeration listOptions()
listOptions in interface OptionHandlerpublic java.lang.String[] getOptions()
getOptions in interface OptionHandler
public void setOptions(java.lang.String[] options)
throws java.lang.Exception
setOptions in interface OptionHandleroptions - the list of options as an array of strings
java.lang.Exception - if an option is not supportedpublic abstract boolean hasMoreElements()
hasMoreElements in interface java.util.Enumerationpublic abstract java.lang.Object nextElement()
nextElement in interface java.util.Enumerationpublic abstract void tokenize(java.lang.String s)
s - the string to tokenize
public static java.lang.String[] tokenize(Tokenizer tokenizer,
java.lang.String[] options)
throws java.lang.Exception
tokenizer - the tokenizer to useoptions - the options for the tokenizer
java.lang.Exception - if setting of options or tokenization fails
public static void runTokenizer(Tokenizer tokenizer,
java.lang.String[] options)
tokenizer - the tokenizer to useoptions - the options for the tokenizer
|
||||||||||
| PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
| SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD | |||||||||