html5tokenizer - Fork of html5gum with code span support

Age	Commit message (Collapse)	Author
2023-09-28	chore: reorder DefaultEmitter methods as well	Martin Fischer
	See the previous commit.
2023-09-28	break!: make DefaultEmitter::emit_eof emit EndOfFile	Martin Fischer

2023-09-28	break!: emit chars instead of strings	Martin Fischer
	The HTML spec specifies that the tokenizer emits character tokens. That html5gum always emitted strings instead was probably just done to make the token consumption more convenient. When it comes to tree construction character tokens are however actually more convenient than string tokens since the spec defines that specific character tokens should be ignored in specific states (and character tokens let us avoid string manipulation for these conditions). This should also make the DefaultEmitter more performant for cases where you don't actually need the strings at all (or only a few) since it avoids string allocations. Though I haven't benchmarked it.
2023-09-28	break!: remove Token::Error	Martin Fischer
	An error isn't a token (in general and also according to the spec). You shouldn't have to filter out errors when you're just interested in tokens but most importantly having errors in the Token enum is annoying when implementing tree construction (since the spec conditions exhaustively cover all Token variants except Token::Error).
2023-09-28	refactor: remove DefaultEmitter::push_error helper fn	Martin Fischer

2023-09-28	break!: rename Emitter::emit_error to report_error	Martin Fischer

2023-09-28	chore: move emit_error method up	Martin Fischer

2023-09-27	break!: remove Emitter::pop_token, use Iterator instead	Martin Fischer

2023-09-12	docs: move warning from DefaultEmitter to Tokenizer	Martin Fischer

2023-09-11	chore: move DefaultEmitter to own module	Martin Fischer