| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2023-09-28 | feat: add span to Trace::Char | Martin Fischer | |
| 2023-09-28 | feat: implement BasicEmitter | Martin Fischer | |
| 2023-09-28 | chore: add BasicEmitter stub | Martin Fischer | |
| 2023-09-28 | break!: make DefaultEmitter::emit_eof emit EndOfFile | Martin Fischer | |
| 2023-09-28 | break!: emit chars instead of strings | Martin Fischer | |
| The HTML spec specifies that the tokenizer emits character tokens. That html5gum always emitted strings instead was probably just done to make the token consumption more convenient. When it comes to tree construction character tokens are however actually more convenient than string tokens since the spec defines that specific character tokens should be ignored in specific states (and character tokens let us avoid string manipulation for these conditions). This should also make the DefaultEmitter more performant for cases where you don't actually need the strings at all (or only a few) since it avoids string allocations. Though I haven't benchmarked it. | |||
| 2023-09-03 | docs: add spans example | Martin Fischer | |
| 2023-09-03 | docs: the span logic no longer assumes UTF-8 | Martin Fischer | |
| 2023-09-03 | docs: credit Markus in readme | Martin Fischer | |
| 2023-09-03 | docs: remove description of Emitter trait from readme | Martin Fischer | |
| Implementing Emitter methods as no-ops works great with the NaiveParser but less so when you want spec-compliant HTML parsing since that requires tree construction and most Emitter methods to be implemented. Ideally we'll implement both tree construction and a new way of avoiding unnecessary allocations (without having to implement your own Emitter). | |||
| 2023-09-03 | docs: add 'Compliance & testing' section to readme | Martin Fischer | |
| 2023-09-03 | docs: add Limitations section to readme | Martin Fischer | |
| 2023-09-03 | docs: restore accidentally lost code block info string | Martin Fischer | |
| I accidentally lost it in b125bec9914bd211d77719bd60bc5a23bd9db579. (I should have changed the info string to ```rust ignore.) | |||
| 2023-09-03 | docs: add changelog | Martin Fischer | |
| 2023-08-19 | feat: introduce NaiveParser | Martin Fischer | |
| 2023-08-19 | break!: remove DefaultEmitter from public API | Martin Fischer | |
| 2023-08-19 | fix(docs): fix broken relative link in rustdoc | Martin Fischer | |
| 2023-08-19 | docs: link multipage version of HTML spec | Martin Fischer | |
| 2023-08-19 | chore: use link reference definitions in Markdown | Martin Fischer | |
| 2023-08-19 | refactor!: make Emitter generic over offset instead of reader | Martin Fischer | |
| Emitters should not have access to the reader at all. Also the current position of the reader, at the time an Emitted method is called, very much depends on machine implementation details such as if `Tokenizer::unread_char` is used. Having the Emitter methods take offsets lets the machine take care of providing the right offsets, as evidenced by the next commit. | |||
| 2023-08-19 | refactor: add default for S type param of DefaultEmitter | Martin Fischer | |
| 2023-08-19 | break!: merge Tokenizer::new_with_emitter into Tokenizer::new | Martin Fischer | |
| The Tokenizer does not perform any state switching, since proper state switching requires a feedback loop between tokenization and DOM tree building. Using the Tokenizer directly therefore is a bit of a pitfall, since you might not expect it to e.g. tokenize `<script><b>` as: StartTag(StartTag { name: "script", .. }) StartTag(StartTag { name: "b", .. }) Since we don't want to make walking into pitfalls particularly easy, this commit changes the Tokenizer::new method so that you have to specify the Emitter. Since this makes new_with_emitter redundant it is removed. | |||
| 2023-08-19 | break!: remove InfallibleTokenizer in favor of Iterator::flatten | Martin Fischer | |
| 2021-12-05 | rename to html5tokenizer, bump versionv0.4.0 | Martin Fischer | |
| 2021-11-28 | update wording | Markus Unterwaditzer | |
| 2021-11-28 | restructure readme | Markus Unterwaditzer | |
| 2021-11-28 | add another example | Markus Unterwaditzer | |
| 2021-11-28 | update wording again | Markus Unterwaditzer | |
| 2021-11-28 | fix period | Markus Unterwaditzer | |
| 2021-11-28 | update wording again | Markus Unterwaditzer | |
| 2021-11-28 | fix wording | Markus Unterwaditzer | |
| 2021-11-28 | clarify what html5gum isn't, fix #5 | Markus Unterwaditzer | |
| 2021-11-26 | Read html from io::BufRead (#8) | Markus Unterwaditzer | |
| 2021-11-24 | link issue | Markus Unterwaditzer | |
| 2021-11-24 | hello world | Markus Unterwaditzer | |
 |
index : html5tokenizer | |
| Fork of html5gum with code span support |
| aboutsummaryrefslogtreecommitdiff |