Fork of html5gum with code span support http://git.push-f.com/html5tokenizer/atom/integration_tests/tests?h=v0.5.0 2023-08-19T11:41:55Z 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-19T09:46:27Z urn:sha1:8eb2a95a483bddc8c6a6a8a30dff68c290277075 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-18T14:43:03Z urn:sha1:0c495ba984436cccc6caeed66639a2b61095dbad In the next commit I'm adding a test that compares the content of files and pretty_assertions doesn't omit large portions of unchanged lines in its diff[1] (contrary to similar-asserts). (Sidenote: We already depend on similar via insta.) [1]: https://github.com/rust-pretty-assertions/rust-pretty-assertions/issues/114 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-17T16:07:32Z urn:sha1:11316f041985345dd3a712d14bea749790f937a4 Display impls should return human-readable strings. After this commit we're able to introduce a proper Display impl in the future without that being a breaking change. 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-16T16:54:35Z urn:sha1:c07334e7bb613774d3535663433c90b6dde86943 Just a bit more succinct. And now rustdoc also no longer cuts off the names of these Emitter methods in its sidebar. 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-16T13:58:48Z urn:sha1:0c87a9ec25a45efc9b6b5ab7883cd19ded483909 Making this change made me realize that adding an `impl IntoIterator for T` can be a breaking change if `impl IntoIterator for &T` already exists. See also the cargo-semver-checks issue[1] I filed about that. [1]: https://github.com/obi1kenobi/cargo-semver-checks/issues/518 2023-08-19T11:41:55Z Martin Fischer martin@push-f.com 2023-08-16T07:45:18Z urn:sha1:65aca9cbf0318bd3a2f936641b4f5bc3729c98c2 This has a number of benefits: * it hides the implementation of the map * it hides the type used for the map values (which lets us e.g. change name_span to name_offset while still being able to provide a convenient `Attribute::name_span` method.) * it lets us provide convenience impls for the map such as `FromIterator<(String, String)>` 2023-08-19T09:41:55Z Martin Fischer martin@push-f.com 2023-08-17T13:21:32Z urn:sha1:4d9cf7171836625b61dcfe675bdf9452766166c0 2023-08-19T04:41:55Z Martin Fischer martin@push-f.com 2023-08-12T09:50:46Z urn:sha1:69fdbee250ebfce135bff2671226097bc536d953 This is primarily done to make the rustdoc more readable (by grouping Reader, IntoReader, StringReader and BufReadReader in the reader module). Ideally IntoReader is already implemented for your input type and you don't have to concern yourself with these traits / types at all. 2023-08-19T04:41:55Z Martin Fischer martin@push-f.com 2023-08-12T09:06:02Z urn:sha1:9f1019afa7a8e9102d67356d85bd632044eb2d0c The Tokenizer does not perform any state switching, since proper state switching requires a feedback loop between tokenization and DOM tree building. Using the Tokenizer directly therefore is a bit of a pitfall, since you might not expect it to e.g. tokenize `<script><b>` as: StartTag(StartTag { name: "script", .. }) StartTag(StartTag { name: "b", .. }) Since we don't want to make walking into pitfalls particularly easy, this commit changes the Tokenizer::new method so that you have to specify the Emitter. Since this makes new_with_emitter redundant it is removed. 2023-08-19T04:41:55Z Martin Fischer martin@push-f.com 2023-08-16T15:07:06Z urn:sha1:681404e5036841ec45356f56f77cc5377f3640d9 Previously we mapped the test tokens to our own token type. Now we do the reverse, which makes more sense as it enables us to easily add more detailed fields to our own token variants without having to worry about these fields not being present in the html5lib test data. (An alternative would be to normalize the values of these fields to some arbitrary value so that PartialEq still holds but seeing such normalized fields in the diff printed by pretty_assertions on a test failure would be quite confusing).