Skip to content

The parse tree

Viktor Chernev edited this page Jun 26, 2023 · 1 revision

1. Tokens

Token
.Symbol Symbol
.SymbolType Symbol.Type
.object Data
.int State
.Position Position

Position
.int Line
.int Column

Tokens in the Gold Parser are an entity that stores parsed data. This data is stored in the field object Data, which boxes a Reduction or a string - thus I will call those string tokens and reduction tokens. (I would've used two separate classes - StringToken and ReductionToken (or TerminalToken and NonTerminalToken) that inherit an abstract class Token instead, but never mind that) The inbuilt lexer produces the string tokens, while the parser produces the reduction tokens.

The string tokens contain a string - that is a chunk of the source code. They also utilize the Position, which is the position of the first character of that string chunk in the source file.

Reduction tokens are generated by the parser when we have a reduce action. The data of these tokens is a Reduction object that contain other tokens. The position is not utilized and remains (0,0).

Tokens can also remember a past State.

2. Reductions

Reduction
.Production Production
.List<Token> Tokens

Reductions in Gold Parser are an entity that stores parsed data about a Production. This data is a list of tokens. Naturally, those tokens might be string tokens(those represent terminal symbols), or reduction tokens(those represent non-terminal symbols), or both.

A reduction is what we get after a source have been parsed. The final parse tree is a Token that has one Reduction that has Tokens that have their own reductions and so on.

Clone this wiki locally