🏡 Back Home

Lexer

Contents

• I wrote a bunch of this stuff for the blog post already, but this should probably be a shorter version.
• This module is fairly compartmentalised. It is given a text stream by Text From Files (for actual source) and Feeds (for generated source) and therefore doesn’t need to hook into the build process more generally.
• The two places are in feeding the lexer where it creates a Wordings wording from the entire feed in, and then the final identification of each word by identifying it in Vocabulary
• I need to check where the external lexer flags are set - Inform TODO list