Skip to content

Refactor scopes tree decoding #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

Refactor scopes tree decoding #1

Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ffa5825
Refactor scopes tree decoding
nicolo-ribaudo Aug 21, 2025
7c8e68f
Update spec.emu
nicolo-ribaudo Aug 22, 2025
da98899
Update spec.emu
nicolo-ribaudo Aug 22, 2025
e5d5689
Keep null
nicolo-ribaudo Aug 28, 2025
d6821f7
Handle Scopes:OriginalScopeTreeList case
nicolo-ribaudo Aug 28, 2025
0043d9c
Format
nicolo-ribaudo Aug 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
296 changes: 160 additions & 136 deletions spec.emu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1617,27 +1617,27 @@
</tr>
<tr>
<td>[[ScopeLine]]</td>
<td>a non-negative Number</td>
<td>an Index Accumulator Record</td>
<td>|ScopeLine|</td>
</tr>
<tr>
<td>[[ScopeColumn]]</td>
<td>a non-negative Number</td>
<td>an Index Accumulator Record</td>
<td>|ScopeColumn|</td>
</tr>
<tr>
<td>[[ScopeNameIndex]]</td>
<td>a non-negative Number</td>
<td>an Index Accumulator Record</td>
<td>|ScopeNameOrKind|</td>
</tr>
<tr>
<td>[[ScopeKindIndex]]</td>
<td>a non-negative Number</td>
<td>an Index Accumulator Record</td>
<td>|ScopeNameOrKind| and |ScopeKind|</td>
</tr>
<tr>
<td>[[ScopeVariableIndex]]</td>
<td>a non-negative Number</td>
<td>an Index Accumulator Record</td>
<td>|ScopeVariable|</td>
</tr>
<tr>
Expand Down Expand Up @@ -1672,6 +1672,42 @@
</tr>
</table>
</emu-table>

<p>An <dfn>Index Accumulator Record</dfn> is used to keep track of an index that is expressed in consecutive relative increments. It has the following fields:</p>
<emu-table id="table-index-accumulator-record-fields" caption="Index Accumulator Record Fields">
<table>
<thead>
<tr>
<th>Field Name</th>
<th>Type</th>
</tr>
</thead>
<tr>
<td>[[Index]]</td>
<td>a non-negative integer</td>
</tr>
</table>
</emu-table>

<emu-clause id="sec-AccumulateIndex" type="abstract operation">
<h1>
AccumulateIndex (
_accumulator_: an Index Accumulator Record,
_increment_: an integer,
): a non-negative integer
</h1>
<dl class="header">
<dt>description</dt>
<dd>Adds _increment_ to the accumulator and returns the new value.</dd>
</dl>
<emu-alg>
1. Set _accumulator_.[[Index]] to _accumulator_.[[Index]] + _increment_.
1. If _accumulator_.[[Index]] &lt; 0, then
1. Optionally report an error.
1. Set _accumulator_.[[Index]] to 0.
1. Return _accumulator_.[[Index]].
</emu-alg>
</emu-clause>
</emu-clause>

<emu-clause id="sec-DecodeScopesInfo" type="abstract operation">
Expand Down Expand Up @@ -1708,34 +1744,10 @@
OriginalScopeTreeList `,` TopLevelItemList
</emu-grammar>
<emu-alg>
1. Perform DecodeScopesInfoItem of |OriginalScopeTreeList| with arguments _info_, _state_ and _names_.
1. Let _originalScopes_ be the DecodedOriginalScopeTrees of |OriginalScopeTreeList| with arguments _state_ and _names_.
1. Set _info_.[[Scopes]] to _originalScopes_.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we need to set the result on DecodeOriginalScopeTrees now on _info_.[[Scopes]], do we also need to duplicate this for the production:

Scopes :
  OriginalScopeTreeList

Since the application of the chain rule only performs the SDO but does not set t on _info_?

1. Perform DecodeScopesInfoItem of |TopLevelItemList| with arguments _info_, _state_ and _names_.
</emu-alg>
<emu-grammar>
OriginalScopeTreeList :
OriginalScopeTreeList `,` OriginalScopeTreeItem
</emu-grammar>
<emu-alg>
1. Perform DecodeScopesInfoItem of |OriginalScopeTreeList| with arguments _info_, _state_ and _names_.
1. Perform DecodeScopesInfoItem of |OriginalScopeTreeItem| with arguments _info_, _state_ and _names_.
</emu-alg>
<emu-grammar>
OriginalScopeTreeItem :
OriginalScopeTree
</emu-grammar>
<emu-alg>
1. Set _state_.[[ScopeLine]] to 0.
1. Set _state_.[[ScopeColumn]] to 0.
1. Let _scope_ be DecodeOriginalScope(|OriginalScopeTree|, _state_, _names_).
1. Append _scope_ to _info_.[[Scopes]].
</emu-alg>
<emu-grammar>
OriginalScopeTreeItem :
[empty]
</emu-grammar>
<emu-alg>
1. Append *null* to _info_.[[Scopes]].
</emu-alg>
<emu-grammar>
TopLevelItemList :
TopLevelItemList `,` TopLevelItem
Expand All @@ -1755,159 +1767,171 @@
</emu-clause>
</emu-clause>

<emu-clause id="sec-DecodeOriginalScope" type="abstract operation">
<h1>
DecodeOriginalScope (
_scopeTree_: a |OriginalScopeTree| Parse Node,
_state_: a Decode Scope State Record,
_names_: a List of Strings,
): a Original Scope Record
</h1>
<dl class="header"></dl>
<emu-alg>
1. Let _scope_ be a new Original Scope Record with all fields default initialized.
1. Perform DecodeOriginalScopeItem of _scopeTree_ with arguments _scope_, _state_ and _names_.
1. Append _scope_ to _state_.[[FlatScopes]].
1. Return _scope_.
</emu-alg>

<emu-clause id="sec-DecodeOriginalScopeItem" type="sdo">
<emu-clause id="sec-DecodedOriginalScopeTrees" type="sdo">
<h1>
DecodeOriginalScopeItem (
_scope_: a Original Scope Record,
DecodedOriginalScopeTrees (
_state_: a Decode Scope State Record,
_names_: a List of Strings,
)
): a List of Original Scope Record
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is important that we preserve the OriginalScopeTreeItem: [empty] case with a null.

The motivation is that if a source map has a list of N sources, then we expect N original scope trees. To signify that a specific source file doesn't have an original scope tree, we currently use the empty production and add null.

The PR currently removes the null, so afterwards we can't associate source records with their respective original scope tree.

I don't have a strong opinion if we want to do this differently. Either by having an "empty" tag, or by adding a "sourceFileIndex" field or item.

If we keep the "null" route we'd need to:

  • Return here a List of either a Original Scope Record or ~null~
  • Change the empty production to return a single element list: 1. Return « ~null~ »

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also add null to the flattened scopes?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say no. Reason being that the "definition index" in a generated range is basically the Nth "original scope start" item. I.e. an index into the flattened scopes without pushing the top-level null scopes.

</h1>
<dl class="header"></dl>
<emu-grammar>
OriginalScopeTree :
OriginalScopeStart OriginalScopeVariablesItem? OriginalScopeItemList? `,` OriginalScopeEnd
OriginalScopeItemList : OriginalScopeItemList `,` OriginalScopeItem
</emu-grammar>
<emu-alg>
1. Perform DecodeOriginalScopeItem of |OriginalScopeStart| with arguments _scope_, _state_ and _names_.
1. If |OriginalScopeVariablesItem| is present, perform DecodeOriginalScopeItem of |OriginalScopeVariablesItem| with arguments _scope_, _state_ and _names_.
1. If |OriginalScopeItemList| is present, perform DecodeOriginalScopeItem of |OriginalScopeItemList| with arguments _scope_, _state_ and _names_.
1. Perform DecodeOriginalScopeItem of |OriginalScopeEnd| with arguments _scope_, _state_ and _names_.
1. Let _left_ be the DecodedOriginalScopeTrees of |OriginalScopeItemList| with arguments _state_ and _names_.
1. Let _right_ be the DecodedOriginalScopeTrees of |OriginalScopeItem| with arguments _state_ and _names_.
1. Return the list-concatenation of _left_ and _right_.
</emu-alg>
<emu-grammar>
OriginalScopeStart :
`B` ScopeFlags ScopeLine ScopeColumn ScopeNameOrKind? ScopeKind?
OriginalScopeTreeItem: [empty]
</emu-grammar>
<emu-alg>
1. Perform DecodeOriginalScopeItem of |ScopeLine| with arguments _scope_, _state_ and _names_.
1. Set _scope_.[[Start]].[[Line]] to _state_.[[ScopeLine]].
1. Perform DecodeOriginalScopeItem of |ScopeColumn| with arguments _scope_, _state_ and _names_.
1. Set _scope_.[[Start]].[[Column]] to _state_.[[ScopeColumn]].
1. Let _flags_ be the VLQUnsignedValue of |ScopeFlags|.
1. If _flags_ & 0x3 = 0x1, then
1. Assert: |ScopeNameOrKind| is present and |ScopeKind| is not present.
1. DecodeOriginalScopeName(|ScopeNameOrKind|, _scope_, _state_, _names_).
1. Else if _flags_ & 0x3 = 0x2, then
1. Assert: |ScopeNameOrKind| is present and |ScopeKind| is not present.
1. DecodeOriginalScopeKind(|ScopeNameOrKind|, _scope_, _state_, _names_).
1. Else if _flags_ & 0x3 = 0x3, then
1. Assert: |ScopeNameOrKind| is present and |ScopeKind| is present.
1. DecodeOriginalScopeName(|ScopeNameOrKind|, _scope_, _state_, _names_).
1. DecodeOriginalScopeKind(|ScopeKind|, _scope_, _state_, _names_).
1. If _flags_ & 0x4 = 0x4, set _scope_.[[IsStackFrame]] to *true*.
1. Return « ».
</emu-alg>
<emu-grammar>
ScopeVariableList :
ScopeVariableList ScopeVariable
</emu-grammar>
<emu-alg>
1. Perform DecodeOriginalScopeItem of |ScopeVariableList| with arguments _scope_, _state_ and _names_.
1. Perform DecodeOriginalScopeItem of |ScopeVariable| with arguments _scope_, _state_ and _names_.
</emu-alg>
<emu-grammar>
OriginalScopeItemList :
OriginalScopeItemList `,` OriginalScopeItem
</emu-grammar>
<emu-alg>
1. Perform DecodeOriginalScopeItem of |OriginalScopeItemList| with arguments _scope_, _state_ and _names_.
1. Perform DecodeOriginalScopeItem of |OriginalScopeItem| with arguments _scope_, _state_ and _names_.
</emu-alg>
<emu-grammar>
OriginalScopeItem :
OriginalScopeTree
OriginalScopeTree :
OriginalScopeStart OriginalScopeVariablesItem? OriginalScopeItemList? `,` OriginalScopeEnd
</emu-grammar>
<emu-alg>
1. Let _childScope_ be DecodeOriginalScope(|OriginalScopeTree|, _state_, _names_).
1. Append _childScope_ to _scope_.[[Children]].
1. Let _start_ be the OriginalScopePosition of |OriginalScopeStart| with arguments _state_.[[ScopeLine]] and _state_.[[ScopeColumn]].
1. Let _name_ be the OriginalScopeName of |OriginalScopeStart| with arguments _state_.[[ScopeNameIndex]] and _names_.
1. Let _kind_ be the OriginalScopeKind of |OriginalScopeStart| with arguments _state_.[[ScopeKindIndex]] and _names_.
1. Let _flags_ be the VLQUnsignedValue of |OriginalScopeStart|'s |ScopeFlags| nonterminal.
1. If _flags_ & 0x4 = 0x4, then
1. Let _isStackFrame_ be *true*.
1. Else,
1. Let _isStackFrame_ be *false*.
1. If |OriginalScopeVariablesItem| is present, then
1. Let _variables_ be the OriginalScopeVariables of |OriginalScopeVariablesItem| with arguments _state_.[[ScopeVariableIndex]] and _names_.
1. Else,
1. Let _variables_ be « ».
1. If |OriginalScopeItemList| is present, then
1. Let _children_ be the DecodedOriginalScopeTrees of |OriginalScopeItemList| with arguments _state_ and _names_.
1. Else,
1. Let _children_ be « ».
1. Let _end_ be the OriginalScopePosition or |OriginalScopeEnd| with arguments _state_.[[ScopeLine]] and _state_.[[ScopeColumn]].
1. Let _originalScope_ be the Original Scope Record { [[Start]]: _start_, [[End]]: _end_, [[Name]]: _name_, [[Kind]]: _kind_, [[Variables]]: _variables_, [[Children]]: _children_, [[IsStackFrame]]: _isStackFrame_ }.
1. Append _originalScope_ to _state_.[[FlatOriginalScopes]].
1. Return « _originalScope_ ».
</emu-alg>
</emu-clause>

<emu-clause id="sec-OriginalScopeStart" type="sdo">
<h1>
OriginalScopePosition(
_lineAccumulator_: an Index Accumulator Record,
_columnAccumulator_: an Index Accumulator Record,
): a Position Record
</h1>
<dl class="header"></dl>
<emu-grammar>
OriginalScopeStart :
`B` ScopeFlags ScopeLine ScopeColumn ScopeNameOrKind? ScopeKind?

OriginalScopeEnd :
`C` ScopeLine ScopeColumn
</emu-grammar>
<emu-alg>
1. Perform DecodeOriginalScopeItem of |ScopeLine| with arguments _scope_, _state_ and _names_.
1. Set _scope_.[[End]].[[Line]] to _state_.[[ScopeLine]].
1. Perform DecodeOriginalScopeItem of |ScopeColumn| with arguments _scope_, _state_ and _names_.
1. Set _scope_.[[End]].[[Column]] to _state_.[[ScopeColumn]].
</emu-alg>
<emu-grammar>
ScopeLine :
Vlq
</emu-grammar>
<emu-alg>
1. Let _relativeLine_ be the VLQUnsignedValue of |Vlq|.
1. Set _state_.[[ScopeLine]] to _state_.[[ScopeLine]] + _relativeLine_.
1. [id="step-reset-scope-column"] If _relativeLine_ > 0, set _state_.[[ScopeColumn]] to 0.
1. Let _relativeLine_ be the VLQUnsignedValue of |ScopeLine|.
1. Let _line_ be AccumulateIndex(_lineAccumulator_, _relativeLine_).
1. [id="step-reset-scope-column"] If _relativeLine_ > 0, set _columnAccumulator_.[[Index]] to 0.
1. Let _relativeColumn_ be the VLQUnsignedValue of |ScopeColumn|.
1. Let _column_ be AccumulateIndex(_columnAccumulator_, _relativeColumn_).
1. Return the Position Record { [[Line]]: _line_, [[Column]]: _column_ }.
</emu-alg>
<emu-note>
Step <emu-xref href="#step-reset-scope-column"></emu-xref> makes it so |ScopeColumn| is encoded relative if the preceding |OriginalScopeStart| or |OriginalScopeEnd| is on the same line (i.e. _relativeLine_ is 0), or absolute otherwise.
</emu-note>
</emu-clause>

<emu-clause id="sec-OriginalScopeName" type="sdo">
<h1>
OriginalScopeName(
_accumulator_: an Index Accumulator Record,
_names_: a List of Strings,
): a String or *null*
</h1>
<dl class="header"></dl>
<emu-grammar>
ScopeColumn :
Vlq
OriginalScopeStart :
`B` ScopeFlags ScopeLine ScopeColumn ScopeNameOrKind? ScopeKind?
</emu-grammar>
<emu-alg>
1. Let _relativeColumn_ be the VLQUnsignedValue of |Vlq|.
1. Set _state_.[[ScopeColumn]] to _state_.[[ScopeColumn]] + _relativeColumn_.
1. Let _flags_ be the VLQUnsignedValue of |ScopeFlags|.
1. If _flags_ & 0x1 ≠ 0x1, return *null*.
1. Assert: |ScopeNameOrKind| is present.
1. Return RelativeName(|ScopeNameOrKind|, _accumulator_, _names_).
</emu-alg>
</emu-clause>

<emu-clause id="sec-OriginalScopeKind" type="sdo">
<h1>
OriginalScopeKind(
_accumulator_: an Index Accumulator Record,
_names_: a List of Strings,
): a String or *null*
</h1>
<dl class="header"></dl>
<emu-grammar>
ScopeVariable :
Vlq
OriginalScopeStart :
`B` ScopeFlags ScopeLine ScopeColumn ScopeNameOrKind? ScopeKind?
</emu-grammar>
<emu-alg>
1. Let _relativeVariable_ be the VLQSignedValue of |Vlq|.
1. Set _state_.[[ScopeVariableIndex]] to _state_.[[ScopeVariableIndex]] + _relativeVariable_.
1. Append _names_[_state_.[[ScopeVariableIndex]]] to _scope_.[[Variables]].
1. Let _flags_ be the VLQUnsignedValue of |ScopeFlags|.
1. If _flags_ & 0x2 ≠ 0x2, return *null*.
1. Assert: |ScopeNameOrKind| is present.
1. If _flags_ & 0x1 = 0x1, then
1. Assert: |ScopeKind| is present.
1. Return RelativeName(|ScopeKind|, _accumulator_, _names_).
1. Return RelativeName(|ScopeNameOrKind|, _accumulator_, _names_).
</emu-alg>
</emu-clause>

<emu-clause id="sec-DecodeOriginalScopeName" type="abstract operation">
<emu-clause id="sec-OriginalScopeVariables" type="sdo">
<h1>
DecodeOriginalScopeName (
_name_: a |Vlq| Parse Node,
_scope_: a Original Scope Record,
_state_: a Decode Scope State Record,
OriginalScopeVariables(
_accumulator_: an Index Accumulator Record,
_names_: a List of Strings,
)
): a List of Strings
</h1>
<dl class="header"></dl>
<emu-grammar>
ScopeVariableList : ScopeVariableList ScopeVariable
</emu-grammar>
<emu-alg>
1. Let _relativeName_ be the VLQSignedValue of _name_.
1. Set _state_.[[ScopeNameIndex]] to _state_.[[ScopeNameIndex]] + _relativeName_.
1. Set _scope_.[[Name]] to _names_[_state_.[[ScopeNameIndex]]].
1. Let _left_ be the OriginalScopeVariables of |ScopeVariableList| with arguments _accumulator_ and _names_.
1. Let _right_ be the OriginalScopeVariables of |ScopeVariable| with arguments _accumulator_ and _names_.
1. Return the list-concatenation of _left_ and _right_.
</emu-alg>
<emu-grammar>
ScopeVariable : Vlq
</emu-grammar>
<emu-alg>
1. TODO: Handle *null* returned by RelativeName.
1. Return « RelativeName(|Vlq|, _accumulator_, _names_) ».
</emu-alg>
</emu-clause>

<emu-clause id="sec-DecodeOriginalScopeKind" type="abstract operation">
<emu-clause id="sec-RelativeName" type="abstract operation">
<h1>
DecodeOriginalScopeKind (
_kind_: a |Vlq| Parse Node,
_scope_: a Original Scope Record,
_state_: a Decode Scope State Record,
RelativeName(
_vlq_: a |Vlq| Parse Node,
_accumulator_: an Index Accumulator Record,
_names_: a List of Strings,
)
): a String or *null*
</h1>
<dl class="header"></dl>
<dl class="header">
<dt>description</dt>
<dd>It returns the name referenced by _vlq_, considering it as a relative index in _names_.</dd>
</dl>
<emu-alg>
1. Let _relativeKind_ be the VLQSignedValue of _kind_.
1. Set _state_.[[ScopeKindIndex]] to _state_.[[ScopeKindIndex]] + _relativeKind_.
1. Set _scope_.[[Kind]] to _names_[_state_.[[ScopeKindIndex]]].
1. Let _relativeIndex_ be the VLQSignedValue of _vlq_.
1. Let _index_ be AccumulateIndex(_accumulator_, _relativeIndex_).
1. If _index_ > the length of _names_, then
1. Optionally report an error.
1. Return *null*.
1. Return _names_[_index_].
</emu-alg>
</emu-clause>
</emu-clause>
Expand Down
Loading