aboutsummaryrefslogtreecommitdiff
path: root/src/lib.rs
AgeCommit message (Collapse)Author
2023-09-28refactor: move utils module under tokenizer::machineMartin Fischer
2023-09-28refactor: move machine module under tokenizerMartin Fischer
2023-09-11chore: move DefaultEmitter to own moduleMartin Fischer
2023-09-09refactor: merge token types with attr to new token moduleMartin Fischer
2023-09-09chore: group public modules togetherMartin Fischer
2023-09-03docs: add spans exampleMartin Fischer
2023-09-03feat: make DefaultEmitter public againMartin Fischer
2023-09-03fix!: remove adjusted_current_node_present_and_not_in_html_namespaceMartin Fischer
Conceptually the tokenizer emits tokens, which are then handled in the tree construction stage (which this crate doesn't yet implement). While the tokenizer can operate almost entirely based on its state (which may be changed via Tokenizer::set_state) and its internal state, there is the exception of the 'Markup declaration open state'[1], the third condition of which depends on the "adjusted current node", which in turn depends on the "stack of open elements" only known to the tree constructor. In 82898967320f90116bbc686ab7ffc2f61ff456c4 I tried to address this by adding the adjusted_current_node_present_and_not_in_html_namespace method to the Emitter trait. What I missed was that adding this method to the Emitter trait effectively crippled the composability of the API. You should be able to do the following: struct TreeConstructor<R, O> { tokenizer: Tokenizer<R, O, SomeEmitter<O>>, stack_of_open_elements: Vec<NodeId>, // ... } However this doesn't work if the implementation of SomeEmitter depends on the stack_of_open_elements field. This commits remedies this oversight by removing this method and instead making the Tokenizer yield values of a new Event enum: enum Event<T> { Token(T), CdataOpen } Event::CdataOpen signals that the new Tokenizer::handle_cdata_open method has to be called, which accepts a CdataAction: enum CdataAction { Cdata, BogusComment } the variants of which correspond exactly to the possible outcomes of the third condition of the 'Markup declaration open state'. Removing this method also has the added benefit that the DefaultEmitter is now again spec-compliant, which lets us expose it again in the next commit in good conscience (previously it just hard-coded the method implementation to return false, which is why I had removed the DefaultEmitter from the public API in the last release). [1]: https://html.spec.whatwg.org/multipage/parsing.html#markup-declaration-open-state
2023-09-03fix: BufReadReader skips line on invalid UTF-8Martin Fischer
2023-09-03docs: add changelogMartin Fischer
2023-08-19feat: introduce NaiveParserMartin Fischer
2023-08-19break!: remove DefaultEmitter from public APIMartin Fischer
2023-08-19chore: move internal re-export after public APIMartin Fischer
2023-08-19fix(docs): fix broken relative link in rustdocMartin Fischer
2023-08-19break!: introduce AttributeMapMartin Fischer
This has a number of benefits: * it hides the implementation of the map * it hides the type used for the map values (which lets us e.g. change name_span to name_offset while still being able to provide a convenient `Attribute::name_span` method.) * it lets us provide convenience impls for the map such as `FromIterator<(String, String)>`
2023-08-19chore: move Attribute to attr moduleMartin Fischer
This is done separately so that the following commit has a cleaner diff.
2023-08-19feat!: add offset to commentsMartin Fischer
2023-08-19refactor!: remove Span trait, just use RangeMartin Fischer
`std::mem::size_of::<Range<NoopOffset>>()` is 0 so there's no need to abstract over Range.
2023-08-19chore: demote missing_docs lint to warnMartin Fischer
`#![deny(missing_docs)]` makes `cargo test` abort immediately if any public API member is missing a doc comment ... which is quite annoying when experimenting with API designs. Also sometimes refactor commits (such as the very next commit) introduce new types that are then immediately removed afterwards, this should be possible without having to add a `/// TODO``` (which contrary to a compiler warning is easy to miss).
2023-08-19break!: stop re-exporting reader traits & typesMartin Fischer
This is primarily done to make the rustdoc more readable (by grouping Reader, IntoReader, StringReader and BufReadReader in the reader module). Ideally IntoReader is already implemented for your input type and you don't have to concern yourself with these traits / types at all.
2023-08-19break!: remove Never in favor of std::convert::InfallibleMartin Fischer
This change is a backport of 04e6cbe[1] from html5gum. [1]: https://github.com/untitaker/html5gum/commit/04e6cbe44bb7a388bd61d1c9cfe4c618eb3b0e29
2023-08-19break!: remove InfallibleTokenizer in favor of Iterator::flattenMartin Fischer
2023-08-19break!: rename Readable to IntoReaderMartin Fischer
The trait of the standard library is also called IntoIterator and not Iterable.
2021-12-05spans: support attribute namesMartin Fischer
2021-12-05spans: add span testsMartin Fischer
2021-12-05spans: copy DefaultEmitter to new span moduleMartin Fischer
2021-12-05allow setting the Tokenizer to Data, PlainText, RcData, RawText and ↵Martin Fischer
ScriptData states
2021-12-05prepare for introduction of public State enumMartin Fischer
2021-11-27split up match-arms and tokenizer to isolate some tokenizer-internal stateMarkus Unterwaditzer
purpose: don't want to expose self.to_reconsume to the consume() method
2021-11-26Read html from io::BufRead (#8)Markus Unterwaditzer
2021-11-26clean up reader interfaceMarkus Unterwaditzer
2021-11-24hello worldMarkus Unterwaditzer