protected static class WebcrawlerConnector.EvaluatorTokenStream extends Object
| Modifier and Type | Field and Description |
|---|---|
protected int |
pos |
protected String |
text |
protected WebcrawlerConnector.EvaluatorToken |
token |
| Constructor and Description |
|---|
WebcrawlerConnector.EvaluatorTokenStream(String text)
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
void |
advance()
Go on to next token.
|
protected WebcrawlerConnector.EvaluatorToken |
nextToken() |
WebcrawlerConnector.EvaluatorToken |
peek()
Get current token.
|
protected String text
protected int pos
protected WebcrawlerConnector.EvaluatorToken token
public WebcrawlerConnector.EvaluatorTokenStream(String text)
public WebcrawlerConnector.EvaluatorToken peek() throws ManifoldCFException
ManifoldCFExceptionpublic void advance()
protected WebcrawlerConnector.EvaluatorToken nextToken() throws ManifoldCFException
ManifoldCFException