Age | Commit message (Collapse) | Author | |
---|---|---|---|
2023-08-19 | docs: add warning to DefaultEmitter | Martin Fischer | |
2023-08-19 | fix: fix lots of position off-by-ones | Martin Fischer | |
Previously the PosTrackingReader always mysteriously subtracted 1 from the current position ... this wasn't sound at all ... the machine just happens to often call `Tokenizer::unread_char` ... but not always. E.g. for proper comments it didn't which resulted in their offset and spans being off-by-one, which is fixed by this commit (see test_spans.rs). | |||
2023-08-19 | refactor!: make Emitter generic over offset instead of reader | Martin Fischer | |
Emitters should not have access to the reader at all. Also the current position of the reader, at the time an Emitted method is called, very much depends on machine implementation details such as if `Tokenizer::unread_char` is used. Having the Emitter methods take offsets lets the machine take care of providing the right offsets, as evidenced by the next commit. | |||
2023-08-19 | chore: move type param bounds to where clause | Martin Fischer | |
2023-08-19 | feat!: add offset to comments | Martin Fischer | |
2023-08-19 | refactor!: remove Span trait, just use Range | Martin Fischer | |
`std::mem::size_of::<Range<NoopOffset>>()` is 0 so there's no need to abstract over Range. | |||
2023-08-19 | refactor!: make Position generic over offset type | Martin Fischer | |
Previously Span was generic over R just so that it could provide the method: fn from_reader(reader: &R) -> Self; and properly implementing that method again relied on R implementing the Position trait: impl<P: Position> Span<P> for Range<usize> { .. } which was a very roundabout and awkward way of doing things. It makes much more sense to make the Position trait generic over the return type of its method (which previously always had to be usize). Which lets us provide a blanket implementation: impl<R: Reader> Position<NoopOffset> for R { .. } | |||
2023-08-19 | chore: demote missing_docs lint to warn | Martin Fischer | |
`#![deny(missing_docs)]` makes `cargo test` abort immediately if any public API member is missing a doc comment ... which is quite annoying when experimenting with API designs. Also sometimes refactor commits (such as the very next commit) introduce new types that are then immediately removed afterwards, this should be possible without having to add a `/// TODO``` (which contrary to a compiler warning is easy to miss). | |||
2023-08-19 | break!: rename GetPos trait to Position | Martin Fischer | |
More in line with RFC 344.[1] [1]: https://rust-lang.github.io/rfcs/0344-conventions-galore.html#gettersetter-apis | |||
2023-08-19 | refactor: add default for S type param of DefaultEmitter | Martin Fischer | |
2023-08-19 | refactor!: remove current_is_appropriate_end_tag_token from Emitter | Martin Fischer | |
2023-08-19 | refactor: proxy essential Emitter methods through Tokenizer | Martin Fischer | |
2023-08-19 | break!: stop re-exporting reader traits & types | Martin Fischer | |
This is primarily done to make the rustdoc more readable (by grouping Reader, IntoReader, StringReader and BufReadReader in the reader module). Ideally IntoReader is already implemented for your input type and you don't have to concern yourself with these traits / types at all. | |||
2023-08-19 | docs: remove `crate::` from link labels | Martin Fischer | |
2023-08-19 | docs: move `produce ("emit")` clue to Emitter doc | Martin Fischer | |
2023-08-19 | break!: merge Tokenizer::new_with_emitter into Tokenizer::new | Martin Fischer | |
The Tokenizer does not perform any state switching, since proper state switching requires a feedback loop between tokenization and DOM tree building. Using the Tokenizer directly therefore is a bit of a pitfall, since you might not expect it to e.g. tokenize `<script><b>` as: StartTag(StartTag { name: "script", .. }) StartTag(StartTag { name: "b", .. }) Since we don't want to make walking into pitfalls particularly easy, this commit changes the Tokenizer::new method so that you have to specify the Emitter. Since this makes new_with_emitter redundant it is removed. | |||
2023-08-19 | docs: move note about Reader impls to Reader trait | Martin Fischer | |
2023-08-19 | break!: remove Default impl for Attribute | Martin Fischer | |
2023-08-19 | break!: remove Default impls for StartTag and EndTag | Martin Fischer | |
2023-08-19 | break!: privatize PosTrackingReader fields | Martin Fischer | |
2023-08-19 | break!: rename PosTracker to PosTrackingReader | Martin Fischer | |
2023-08-19 | break!: remove Never in favor of std::convert::Infallible | Martin Fischer | |
This change is a backport of 04e6cbe[1] from html5gum. [1]: https://github.com/untitaker/html5gum/commit/04e6cbe44bb7a388bd61d1c9cfe4c618eb3b0e29 | |||
2023-08-19 | break!: remove InfallibleTokenizer in favor of Iterator::flatten | Martin Fischer | |
2023-08-19 | docs: remove Tokenizer::new examples from Reader docs | Martin Fischer | |
2023-08-19 | break!: rename Readable to IntoReader | Martin Fischer | |
The trait of the standard library is also called IntoIterator and not Iterable. | |||
2023-08-19 | fix(docs): remove outdated list of Readable impls | Martin Fischer | |
dced8066f77f570dd3e396ec3570c71aa86c454e introduced a Readable impl for std::io::BufReader. Manually listing impls in a doc comment is a bad idea since such lists will just get out of date and there's no need for that since rustdoc automatically lists all implementations on the trait page. | |||
2023-08-19 | fix(docs): fix Error variant doc saying `$literal` | Martin Fischer | |
2023-08-19 | fix(docs): Span is a byte range (not character range) | Martin Fischer | |
2023-08-19 | fix(docs): StartTag is a start tag | Martin Fischer | |
2023-08-19 | fix(docs): Error::EndTagWithAttributes should be emitted by emit_current_tag | Martin Fischer | |
2023-08-19 | break!: remove StartTag::next_state | Martin Fischer | |
You shouldn't manually have to match tokens yielded by the tokenizer iterator just to correctly handle state transitions. A better NaiveParser API will be introduced. | |||
2023-08-19 | break!: remove set_last_start_tag from Emitter | Martin Fischer | |
2021-12-05 | rename to html5tokenizer, bump versionv0.4.0 | Martin Fischer | |
2021-12-05 | spans: get rid of code duplication by introducing Span trait | Martin Fischer | |
2021-12-05 | spans: refactor to avoid one clone() | Martin Fischer | |
2021-12-05 | rename internal emit_error to push_error (to avoid confusion with trait method) | Martin Fischer | |
2021-12-05 | improve duplicate attribute span | Martin Fischer | |
2021-12-05 | refactor: match btree_map::Entry instead of using and_modify closure | Martin Fischer | |
2021-12-05 | spans: slightly refactor DefaultEmitter | Martin Fischer | |
2021-12-05 | spans: add spans to Token::Error | Martin Fischer | |
2021-12-05 | spans: fix spans for quoted attribute values | Martin Fischer | |
2021-12-05 | spans: support attribute values | Martin Fischer | |
2021-12-05 | spans: support attribute names | Martin Fischer | |
2021-12-05 | spans: add span tests | Martin Fischer | |
2021-12-05 | spans: start implementing SpanEmitter | Martin Fischer | |
2021-12-05 | spans: introduce PosTracker | Martin Fischer | |
2021-12-05 | spans: introduce GetPos trait | Martin Fischer | |
2021-12-05 | spans: rename to SpanEmitter, adjust generics | Martin Fischer | |
2021-12-05 | spans: copy DefaultEmitter to new span module | Martin Fischer | |
2021-12-05 | spans: make Emitter generic over Reader | Martin Fischer | |