| Age | Commit message (Collapse) | Author |
|---|
|
Previously the Token enum contained the offsets using the O generic
type parameter, which could be a usize if you're tracking offsets or
a zero-sized type if you didn't care about offsets. This commit moves
all the byte offset and syntax information to a new Trace enum,
which has several advantages:
* Traces can now easily be stored separately, while the tokens are
fed to the tree builder. (The tree builder only has to keep track
of which tree nodes originate from which tokens.)
* No needless generics for functions that take a token but don't
care about offsets (a tree construction implementation is bound
to have many of such functions).
* The FromIterator<(String, String)> impl for AttributeMap no longer
has to specify arbitrary values for the spans and the value_syntax).
* The PartialEq implementation of Token is now much more useful
(since it no longer includes all the offsets).
* The Debug formatting of Token is now more readable
(since it no longer includes all the offsets).
* Function pointers to functions accepting tokens are possible.
(Since function pointer types may not have generic parameters.)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Conceptually the tokenizer emits tokens, which are then handled in the
tree construction stage (which this crate doesn't yet implement).
While the tokenizer can operate almost entirely based on its state
(which may be changed via Tokenizer::set_state) and its internal state,
there is the exception of the 'Markup declaration open state'[1], the third
condition of which depends on the "adjusted current node", which in turn
depends on the "stack of open elements" only known to the tree constructor.
In 82898967320f90116bbc686ab7ffc2f61ff456c4 I tried to address this
by adding the adjusted_current_node_present_and_not_in_html_namespace
method to the Emitter trait. What I missed was that adding this method
to the Emitter trait effectively crippled the composability of the API.
You should be able to do the following:
struct TreeConstructor<R, O> {
tokenizer: Tokenizer<R, O, SomeEmitter<O>>,
stack_of_open_elements: Vec<NodeId>,
// ...
}
However this doesn't work if the implementation of SomeEmitter
depends on the stack_of_open_elements field.
This commits remedies this oversight by removing this method and
instead making the Tokenizer yield values of a new Event enum:
enum Event<T> { Token(T), CdataOpen }
Event::CdataOpen signals that the new Tokenizer::handle_cdata_open
method has to be called, which accepts a CdataAction:
enum CdataAction { Cdata, BogusComment }
the variants of which correspond exactly to the possible outcomes
of the third condition of the 'Markup declaration open state'.
Removing this method also has the added benefit that the DefaultEmitter
is now again spec-compliant, which lets us expose it again in the next
commit in good conscience (previously it just hard-coded the method
implementation to return false, which is why I had removed the
DefaultEmitter from the public API in the last release).
[1]: https://html.spec.whatwg.org/multipage/parsing.html#markup-declaration-open-state
|
|
|
|
|
|
|
|
|
|
|
|
|
|
This has a number of benefits:
* it hides the implementation of the map
* it hides the type used for the map values
(which lets us e.g. change name_span to name_offset while still
being able to provide a convenient `Attribute::name_span` method.)
* it lets us provide convenience impls for the map
such as `FromIterator<(String, String)>`
|
|
This is done separately so that the following commit has a cleaner diff.
|
|
|
|
`std::mem::size_of::<Range<NoopOffset>>()` is 0
so there's no need to abstract over Range.
|
|
`#![deny(missing_docs)]` makes `cargo test` abort immediately
if any public API member is missing a doc comment ...
which is quite annoying when experimenting with API designs.
Also sometimes refactor commits (such as the very next commit)
introduce new types that are then immediately removed afterwards,
this should be possible without having to add a `/// TODO```
(which contrary to a compiler warning is easy to miss).
|
|
This is primarily done to make the rustdoc more readable
(by grouping Reader, IntoReader, StringReader and BufReadReader
in the reader module). Ideally IntoReader is already implemented
for your input type and you don't have to concern yourself
with these traits / types at all.
|
|
This change is a backport of 04e6cbe[1] from html5gum.
[1]: https://github.com/untitaker/html5gum/commit/04e6cbe44bb7a388bd61d1c9cfe4c618eb3b0e29
|
|
|
|
The trait of the standard library is also
called IntoIterator and not Iterable.
|
|
|
|
|
|
|
|
ScriptData states
|
|
|
|
purpose: don't want to expose self.to_reconsume to the consume() method
|
|
|
|
|
|
|
