Module token

Parsers extracting tokens from the stream

Functions

• any
  Matches one token
• literal
  Recognizes a literal
• none_of
  Recognize a token that does not match a set of tokens
• one_of
  Recognize a token that matches a set of tokens
• take
  Recognize an input slice containing the first N input elements (I[..N]).
• take_till
  Recognize the longest input slice (if any) till a member of a set of tokens is found.
• take_until
  Recognize the input slice up to the first occurrence of a literal.
• take_while
  Recognize the longest (m <= len <= n) input slice that matches a set of tokens