Age | Commit message (Collapse) | Author | |
---|---|---|---|
2023-09-28 | refactor: move machine impl details to machine module | Martin Fischer | |
This commit separates the public API (the "Tokenizer") from the internal implementation (the "Machine") to make the code more readable. | |||
2023-09-28 | refactor: move utils module under tokenizer::machine | Martin Fischer | |
2023-09-28 | refactor: only use InternalState re-export for feature-gated internal API | Martin Fischer | |
2023-09-28 | refactor: move machine module under tokenizer | Martin Fischer | |