Question

Tokenizes the current Document in the background, and caches the tokenized rows for future use.

If a certain row is changed, everything below that row is re-tokenized.

Constructors

new BackgroundTokenizer(Tokenizer tokenizer, Editor editor)

Creates a new BackgroundTokenizer object.

tokenizer	Tokenizer	Required. The tokenizer to use
editor	Editor	Required. The editor to associate with

Events

BackgroundTokenizer.on("update", function(Object e))

Fires whenever the background tokeniziers between a range of rows are going to be updated.

e

Object

Required. An object containing two properties, first and last, which indicate the rows of the region being updated.

Methods

BackgroundTokenizer.fireUpdateEvent(Number firstRow, Number lastRow)

Emits the 'update' event. firstRow and lastRow are used to define the boundaries of the region to be updated.

firstRow	Number	Required. The starting row region
lastRow	Number	Required. The final row region

BackgroundTokenizer.getState(Number row)

Returns the state of tokenization at the end of a row.

row

Number

Required. The row to get state at

BackgroundTokenizer.getTokens(Number row)

Gives list of tokens of the row. (tokens are cached)

row

Number

Required. The row to get tokens at

BackgroundTokenizer.setDocument(Document doc)

Sets a new document to associate with this object.

doc

Document

Required. The new document to associate with

BackgroundTokenizer.setTokenizer(Tokenizer tokenizer)

Sets a new tokenizer for this object.

tokenizer

Tokenizer

Required. The new tokenizer to use

BackgroundTokenizer.start(Number startRow)

Starts tokenizing at the row indicated.

startRow

Number

Required. The row to start at

BackgroundTokenizer.stop()

Stops tokenizing.