aboutsummaryrefslogtreecommitdiff
path: root/src/tokenizer.rs
AgeCommit message (Collapse)Author
2023-08-19break!: merge Tokenizer::new_with_emitter into Tokenizer::newMartin Fischer
The Tokenizer does not perform any state switching, since proper state switching requires a feedback loop between tokenization and DOM tree building. Using the Tokenizer directly therefore is a bit of a pitfall, since you might not expect it to e.g. tokenize `<script><b>` as: StartTag(StartTag { name: "script", .. }) StartTag(StartTag { name: "b", .. }) Since we don't want to make walking into pitfalls particularly easy, this commit changes the Tokenizer::new method so that you have to specify the Emitter. Since this makes new_with_emitter redundant it is removed.
2023-08-19docs: move note about Reader impls to Reader traitMartin Fischer
2023-08-19break!: remove InfallibleTokenizer in favor of Iterator::flattenMartin Fischer
2023-08-19break!: rename Readable to IntoReaderMartin Fischer
The trait of the standard library is also called IntoIterator and not Iterable.
2023-08-19fix(docs): remove outdated list of Readable implsMartin Fischer
dced8066f77f570dd3e396ec3570c71aa86c454e introduced a Readable impl for std::io::BufReader. Manually listing impls in a doc comment is a bad idea since such lists will just get out of date and there's no need for that since rustdoc automatically lists all implementations on the trait page.
2023-08-19break!: remove set_last_start_tag from EmitterMartin Fischer
2021-12-05spans: add spans to Token::ErrorMartin Fischer
2021-12-05spans: make Emitter generic over ReaderMartin Fischer
2021-12-05spans: make Emitter generic over SpanMartin Fischer
2021-12-05allow setting the Tokenizer to Data, PlainText, RcData, RawText and ↵Martin Fischer
ScriptData states
2021-12-05prepare for introduction of public State enumMartin Fischer
2021-11-28clarify what html5gum isn't, fix #5Markus Unterwaditzer
2021-11-27fix crash in try_read_stringMarkus Unterwaditzer
2021-11-27split up match-arms and tokenizer to isolate some tokenizer-internal stateMarkus Unterwaditzer
purpose: don't want to expose self.to_reconsume to the consume() method