aboutsummaryrefslogtreecommitdiff
path: root/examples/tokenize.rs
diff options
context:
space:
mode:
authorMarkus Unterwaditzer <markus-honeypot@unterwaditzer.net>2021-11-27 01:08:48 +0100
committerMarkus Unterwaditzer <markus-honeypot@unterwaditzer.net>2021-11-27 01:10:09 +0100
commit96808d0d940e1580cf86e433d0c844e943157e0d (patch)
tree4e4cabd0431dc2b465fad1b68ae54c27358dcd4e /examples/tokenize.rs
parent8c6e4ccbb3fc6830239429c994468bc1ccd6832a (diff)
split up match-arms and tokenizer to isolate some tokenizer-internal state
purpose: don't want to expose self.to_reconsume to the consume() method
Diffstat (limited to 'examples/tokenize.rs')
0 files changed, 0 insertions, 0 deletions