semantic-lex

semantic-lex is an autoloaded compiled Lisp function in `lex.el'.

(semantic-lex START END &optional DEPTH LENGTH)

Lexically analyze text in the current buffer between START and END.
Optional argument DEPTH indicates at what level to scan over entire
lists. The last argument, LENGTH specifies that `semantic-lex'
should only return LENGTH tokens. The return value is a token stream.
Each element is a list, such of the form
(symbol start-expression . end-expression)
where SYMBOL denotes the token type.
See `semantic-lex-tokens' variable for details on token types. END
does not mark the end of the text scanned, only the end of the
beginning of text scanned. Thus, if a string extends past END, the
end of the return token will be larger than END. To truly restrict
scanning, use `narrow-to-region'.