DBIx::MyParsePP::Token - Lexical tokens extracted by DBIx::MyParsePP::Lexer

Please see the example under DBIx::MyParsePP::Lexer

new($type, $value) creates a new Token object.

type() or getType() returns the type of the Token, as string.

value() or getValue() returns the value of the Token.

setType($new_type) and setValue($new_value) can be used to manipulate the Token.

toString() returns the value of the token, quoted if necessary, as it would appear in a SQL statement. A leading space is added for most tokens in order to facilitate chaining tokens into a larger statement.

Token types are returned as strings, to avoid possible confusion between integer values of constants and tokens whose type is equal to their value. The following types are used by MySQL:

	"IDENT", "IDENT_QUOTED" - database, table or field identifiers or portions thereof.

	"TEXT_STRING", "NCHAR_STRING" - strings in the form 'aaa' and N'aaa'

	"HEX_NUM", "BIN_NUM" - numbers in the form x'ffff' and b'010101'

	"DECIMAL_NUM", "NUM", "LONG_NUM", "ULONGLONG_NUM" - integers of various lengths

	"FLOAT_NUM" - floating-point numbers in scientific notation, e.g -32032.6809e+10

	"UNDERSCORE_CHARSET" - charset modifier before literal string, eg. _utf8

	"LEX_HOSTNAME" - 

	"SET_VAR" - the variable assignment operator :=

Function names and SQL constructs are returned mostly as strings ending in _SYM". The complete list can be found in DBIx::MyParsePP::Symbols. Some functions are returned as "FUNC_ARG1", "FUNC_ARG2" or "FUNC_ARG3", signifying the number of arguments the function expects. In this case, the actual name of the function can be obtained by calling getValue().

The rules that determine which type of "NUM" is returned can be found in DBIx::MyParsePP::Lexer::int_token()