aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2021-11-30refactor: make tests independent of TokenizerOpts::default()Martin Fischer
2021-11-30remove pointless associated type from TokenSink traitMartin Fischer
2021-11-30refactor: use assert_matches! instead of assert_eq!Martin Fischer
2021-11-30refactor: put line numbers first in test tuplesMartin Fischer
2021-11-30docs: update READMEMartin Fischer
2021-11-29set tokenization state depending on tag nameMartin Fischer
2021-11-29docs: fix link & remove outdated commentMartin Fischer
2021-11-29remove Tokenizer::set_plaintext_stateMartin Fischer
2021-11-29refactor: use ? operator for explicit control flowMartin Fischer
2021-11-29refactor: use std::ops::ControlFlowMartin Fischer
2021-11-29refactor: move return out of go! to clarify flowMartin Fischer
#!/usr/bin/env python3 import re text = '' for line in open('src/tokenizer/mod.rs'): if '$me.emit_current' in line: text += re.sub('return (\$me.emit_current_tag\(\));', lambda x: x.group(1), line) elif '$me:ident' in line: text += re.sub('return (ProcessResult::Continue);', lambda x: x.group(1), line) else: text += re.sub('go!\(self: (to|consume_char_ref|emit_tag)', lambda x: 'return ' + x.group(), line) with open('src/tokenizer/mod.rs', 'w') as f: f.write(text)
2021-11-29refactor: remove no longer needed macro rulesMartin Fischer
2021-11-29refactor: split up go! macro callsMartin Fischer
#!/usr/bin/env python3 import re def split_go(match): inner = match.group(1) stmts = inner.split(';') text = '{' for stmt in stmts: if stmt.startswith(' reconsume '): text += 'self.reconsume = true;' stmt = ' to' + stmt[len(' reconsume'):] text += 'go!(self:{});'.format(stmt) return text + '}' text = '' with open('src/tokenizer/mod.rs') as f: for line in f: if '$me:ident' in line: # skip macro rules text += line else: text += re.sub('go!\(self:(.*)\)', split_go, line) with open('src/tokenizer/mod.rs', 'w') as f: f.write(text) import subprocess subprocess.call(['cargo', 'fmt'])
2021-11-29refactor: eliminate line wrapping in go! macroMartin Fischer
2021-11-29refactor: slightly simplify go! DSLMartin Fischer
2021-11-29stop exporting buffer_queue::SetResultMartin Fischer
2021-11-29stop re-exporting Token & TagKind variantsMartin Fischer
2021-11-29stop exporting internal states moduleMartin Fischer
2021-11-19bump version to 0.2.0v0.2.0Martin Fischer
2021-11-19feature gate named-entities (making phf optional)Martin Fischer
2021-11-19make test sink also emit CharacterTokens & ParseErrorMartin Fischer
2021-11-19format code with cargo fmtMartin Fischer
2021-11-12fix named entitiesMartin Fischer
In 462bb0ef0ba9e027f5138c87438328db718d15da I dropped the markup5ever::data dependency but omitted to include its named_entities data. This commit remedies that mistake.
2021-11-12fix compiler warning about semicolons in macrosMartin Fischer
See https://github.com/rust-lang/rust/issues/79813.
2021-04-18fix typo in README.mdMartin Fischer
2021-04-08fix scope, add short module docstringv0.1.0Martin Fischer
2021-04-08stop exporting macrosMartin Fischer
2021-04-08rebrand as html5tokenizerMartin Fischer
2021-04-08drop log dependencyMartin Fischer
2021-04-08drop mac dependencyMartin Fischer
2021-04-08drop tendril dependencyMartin Fischer
2021-04-08drop markup5ever dependencyMartin Fischer
2021-04-08merge Attribute from markup5everMartin Fischer
2021-04-08rm build-dependenciesMartin Fischer
2021-04-08merge buffer_queue and smallcharset from markup5everMartin Fischer
2021-04-08depend on tendril directly (instead of markup5ever::tendril)Martin Fischer
2021-04-08stop relying on markup5ever::dataMartin Fischer
2021-04-08make dependencies on markup5ever explicitMartin Fischer
2021-04-08delete tree_builder, driver and serializeMartin Fischer
2021-04-08add .gitignore fileMartin Fischer
2021-04-08drop path from markup5ever dependencyMartin Fischer
2021-04-08import https://github.com/servo/html5everMartin Fischer
commit d1206daa740305f55a5fa159e43eb33afc359cb4
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-23 02:18:29 +0000