How to store tokens while lexical analysis
c, compiler-construction, java, lexical-analysis, token
Solution
You need to construct a parse tree, not just a list of tokens. I suggest you continue your reading on the subject before you start coding,
Problem
I'm trying to design a compiler, and am at lexical analysis. Say I take a simple "Hello World!" program as a file of strings and extract tokens from it. What is the best way to store these tokens? In a single data structure, or two or more data structures depending on the type of token?