aboutsummaryrefslogtreecommitdiff
path: root/tests/test_spans.rs
AgeCommit message (Collapse)Author
2023-09-28refactor: make TracingEmitter only work with usizesMartin Fischer
2023-09-28chore: add BasicEmitter stubMartin Fischer
2023-09-28break!: rename DefaultEmitter to TracingEmitterMartin Fischer
2023-09-28break!: remove Token::ErrorMartin Fischer
An error isn't a token (in general and also according to the spec). You shouldn't have to filter out errors when you're just interested in tokens but most importantly having errors in the Token enum is annoying when implementing tree construction (since the spec conditions exhaustively cover all Token variants except Token::Error).
2023-09-28refactor: make labeler closures take NaiveParserMartin Fischer
The second next commit will move errors out of the Token enum but we still want to be able to test that the spans of errors are character encoding independent.
2023-09-28refactor: make parser helper fn generic over readerMartin Fischer
2023-09-28chore: make assert_char_encoding_independence more readableMartin Fischer
2023-09-28chore: rename internal tokenizer helper fn to parserMartin Fischer
2023-09-03feat: add Doctype::name_spanMartin Fischer
2023-09-03refactor: make test_and_annotate test encoding-independenceMartin Fischer
2023-09-03fix!: make comment data spans encoding-independentMartin Fischer
2023-09-03test: test comment data spans more thoroughlyMartin Fischer
2023-09-03fix: make doctype id spans encoding-independentMartin Fischer
2023-09-03fix!: make set_self_closing encoding-independentMartin Fischer
2023-09-03fix!: make attribute spans encoding-independentMartin Fischer
2023-09-03fix!: make start/end tag name spans encoding-independentMartin Fischer
2023-09-03fix: don't assume UTF-8 in machine/tokenizerMartin Fischer
2023-09-03test: verify that span logic incorrectly assumes UTF-8Martin Fischer
2023-09-03refactor: make span tests tokenizer-independentMartin Fischer
2023-09-03refactor: let comment and doctype tests check multiple casesMartin Fischer
2023-09-03fix: too small char ref error spansMartin Fischer
2023-09-03test: verify too small char ref error spansMartin Fischer
2023-09-03fix: off-by-one missing-semicolon-after-character-reference spanMartin Fischer
2023-09-03test: verify off-by-one missing-semicolon-after-character-reference spanMartin Fischer
2023-09-03chore: rename char ref testMartin Fischer
The tests for character reference errors should be grouped together. So this commit puts "char_ref" first in the function name (since our error tests are ordered by function name).
2023-09-03fix!: off-by-one end-tag-with-trailing-solidus spanMartin Fischer
2023-09-03fix: most error spans mistakenly being emptyMartin Fischer
With codespan_reporting an empty span shows up exactly like a one-byte span, which is why I didn't notice this mistake earlier.
2023-09-03fix: off-by-one eof error spansMartin Fischer
2023-09-03test: add span tests for eof errorsMartin Fischer
2023-09-03fix!: wrong attribute value spans for char refsMartin Fischer
2023-09-03test: verify wrong attribute value spans for char refsMartin Fischer
2023-08-19feat: introduce NaiveParserMartin Fischer
2023-08-19break!: remove DefaultEmitter from public APIMartin Fischer
2023-08-19chore: switch from pretty_assertions to similar-assertsMartin Fischer
In the next commit I'm adding a test that compares the content of files and pretty_assertions doesn't omit large portions of unchanged lines in its diff[1] (contrary to similar-asserts). (Sidenote: We already depend on similar via insta.) [1]: https://github.com/rust-pretty-assertions/rust-pretty-assertions/issues/114
2023-08-19break!: stop abusing Display for Error codesMartin Fischer
Display impls should return human-readable strings. After this commit we're able to introduce a proper Display impl in the future without that being a breaking change.
2023-08-19feat!: add span and offsets to DoctypeMartin Fischer
2023-08-19fix: report value offset/span as None for the empty attr syntaxMartin Fischer
2023-08-19break!: introduce AttributeMapMartin Fischer
This has a number of benefits: * it hides the implementation of the map * it hides the type used for the map values (which lets us e.g. change name_span to name_offset while still being able to provide a convenient `Attribute::name_span` method.) * it lets us provide convenience impls for the map such as `FromIterator<(String, String)>`
2023-08-19feat!: add all-inclusive spans to tagsMartin Fischer
Also more performant since we no longer have to update the name span on every Emitter::push_tag_name call.
2023-08-19fix: fix lots of position off-by-onesMartin Fischer
Previously the PosTrackingReader always mysteriously subtracted 1 from the current position ... this wasn't sound at all ... the machine just happens to often call `Tokenizer::unread_char` ... but not always. E.g. for proper comments it didn't which resulted in their offset and spans being off-by-one, which is fixed by this commit (see test_spans.rs).
2023-08-19refactor!: make Emitter generic over offset instead of readerMartin Fischer
Emitters should not have access to the reader at all. Also the current position of the reader, at the time an Emitted method is called, very much depends on machine implementation details such as if `Tokenizer::unread_char` is used. Having the Emitter methods take offsets lets the machine take care of providing the right offsets, as evidenced by the next commit.
2023-08-19feat!: add offset to commentsMartin Fischer
2023-08-19refactor!: remove Span trait, just use RangeMartin Fischer
`std::mem::size_of::<Range<NoopOffset>>()` is 0 so there's no need to abstract over Range.
2023-08-19test: split up span testMartin Fischer
2023-08-19break!: merge Tokenizer::new_with_emitter into Tokenizer::newMartin Fischer
The Tokenizer does not perform any state switching, since proper state switching requires a feedback loop between tokenization and DOM tree building. Using the Tokenizer directly therefore is a bit of a pitfall, since you might not expect it to e.g. tokenize `<script><b>` as: StartTag(StartTag { name: "script", .. }) StartTag(StartTag { name: "b", .. }) Since we don't want to make walking into pitfalls particularly easy, this commit changes the Tokenizer::new method so that you have to specify the Emitter. Since this makes new_with_emitter redundant it is removed.
2023-08-19break!: privatize PosTrackingReader fieldsMartin Fischer
2023-08-19break!: rename PosTracker to PosTrackingReaderMartin Fischer
2023-08-19break!: remove InfallibleTokenizer in favor of Iterator::flattenMartin Fischer
2023-08-19break!: rename Readable to IntoReaderMartin Fischer
The trait of the standard library is also called IntoIterator and not Iterable.
2021-12-05rename to html5tokenizer, bump versionv0.4.0Martin Fischer
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 01:51:05 +0000