diff options
| author | Markus Unterwaditzer <markus-honeypot@unterwaditzer.net> | 2021-11-27 01:08:48 +0100 |
|---|---|---|
| committer | Markus Unterwaditzer <markus-honeypot@unterwaditzer.net> | 2021-11-27 01:10:09 +0100 |
| commit | 96808d0d940e1580cf86e433d0c844e943157e0d (patch) | |
| tree | 4e4cabd0431dc2b465fad1b68ae54c27358dcd4e /examples/tokenize.rs | |
| parent | 8c6e4ccbb3fc6830239429c994468bc1ccd6832a (diff) | |
split up match-arms and tokenizer to isolate some tokenizer-internal state
purpose: don't want to expose self.to_reconsume to the consume() method
Diffstat (limited to 'examples/tokenize.rs')
0 files changed, 0 insertions, 0 deletions