ECSQL: support IS / IS NOT operator between operands (null-safe comparison)

[khanaffan]

imodel-native: iTwin/imodel-native#1476

closes: #9439

This pull request adds support for using the IS and IS NOT operators between two operands in ECSQL, extending their previous usage (which was limited to comparisons with NULL, boolean literals, or type predicates). Now, these operators can be used for null-safe comparisons between properties, including multi-column types like points and navigation properties, with appropriate column-wise expansion. The changes include parser, lexer, and documentation updates, as well as comprehensive tests to verify the new behavior.

ECSQL Syntax and Semantics Enhancements:

• Added support for IS and IS NOT between two operands (e.g., prop1 IS [NOT] prop2), with null-safe comparison semantics consistent with SQLite. For multi-column operands, IS joins per-column comparisons with AND, and IS NOT with OR.
• Updated the ECSQL version to 2.0.4.0 in documentation, code, and tests to reflect the new syntax support.

[aruniverse]

Adversarial review (automated) — 4 inline findings below. One is a blocker (result-table column mismatch that will fail the .ecsql.md test runner) and should be fixed before merge. The others are low/informational.

[aruniverse]

Two related follow-ups on the disambiguation note wording — same single-word fix needed in both NextVersion.md and Operators.md. The sqlbison.y grammar comment in imodel-native#1476 is already accurate; these two docs just need to match it.

[aruniverse]

Same 'qualified' gap as Operators.md — unqualified class names were never type predicates

"a right-hand operand that is exactly NULL/TRUE/FALSE/UNKNOWN, or a parenthesized (ClassName), keeps its original meaning"

The grammar requires a qualified name (schema.Class), an ONLY/ALL prefix, or a comma-separated list to route to the type-predicate path. A bare unqualified name like IS (MyClass) reduces as a value_exp (null-safe comparison) regardless of whether MyClass is a real ECClass — it was never reachable as a type predicate.

Suggested fix:

or a parenthesized *qualified* class name (e.g. `(schema.Class)`), an `ONLY`/`ALL`-prefixed form, or a comma-separated list keeps its original meaning. A single unqualified name in parentheses is always parsed as a value expression.

[aruniverse]

⚠️ Partially addressed — the qualified keyword was added and the wording for unqualified names is correct. However, the IS (alias.prop) qualified-name fallback (added in imodel-native commit 593ff15eb9) is not mentioned here. A qualified name that doesn't resolve to a class — e.g. S1 IS (Foo.S2) or Status IS (ts.Status.Active) — previously failed with 'class not found' and now works as a null-safe comparison. This is the most developer-visible part of the latest commit and should appear in the changelog. New inline comment posted on line 185 with a suggested addition.

[khanaffan]

Good catch — fixed in 0e701b9. The note now reads "a parenthesized qualified class name such as (bis.Element)" and clarifies that a parenthesized unqualified name like (prop2) is read as a value expression. This matches the sqlbison.y comment in imodel-native#1476.

[aruniverse]

✅ Addressed in 782b6e6f87. NextVersion.md now distinguishes qualified class predicates from unqualified value expressions, documents the qualified non-class fallback such as (alias.prop), and states that the class/type-predicate reading wins for qualified collisions.

[aruniverse]

'qualified' missing — same issue as NextVersion.md line 185

"a right-hand operand that is exactly NULL/TRUE/FALSE/UNKNOWN, or a parenthesized (ClassName), keeps its original meaning"

The note correctly warns about the disambiguation, but omits the key constraint: the type-predicate path requires a qualified class name (schema.Class). An unqualified name like IS (MyClass) — even if MyClass is a real ECClass — has always resolved as a value expression. The sqlbison.y comment (imodel-native#1476) already says this precisely; this note should match it.

Additionally, the example x IS y + 1 vs x IS (y + 1) doesn't illustrate the real disambiguation risk — (y + 1) cannot be parsed as a type predicate regardless of parentheses. The genuine risk is a qualified identifier: IS (schema.myProp) where schema.myProp looks like a class reference. Optionally reword to:

> Note: … Those forms take precedence: a right-hand operand that is exactly
> `NULL`/`TRUE`/`FALSE`/`UNKNOWN`, a **qualified** class name in parentheses
> (e.g. `(schema.Class)`), an `ONLY`/`ALL`-prefixed form, or a comma-separated list
> keeps its original meaning. A single unqualified name in parentheses (e.g. `(MyProp)`)
> is always a value expression, even if a class by that name exists. To avoid ambiguity,
> write qualified names without outer parentheses (e.g. `x IS schema.prop` rather than
> `x IS (schema.prop)`).

[khanaffan]

ECSQL does not support unqualified class names. So I am not sure why we have to explicitly say qualified.

[aruniverse]

⚠️ Partially addressed — qualified was added and the Note is significantly improved. One remaining issue: the parenthetical "(if a name is both a class and a property, the property reading wins)" contradicts imodel-native's CHANGES.md which says "the class (type-predicate) reading wins" for the qualified case. Both can be reconciled in context (unqualified names are never classes in ECSQL, so 'property wins' is vacuously true for that sub-case), but a user reading Operators.md in isolation will take the parenthetical as the general rule and be wrong about qualified ambiguous names. New inline comment posted on line 78 with a rewrite suggestion.

[khanaffan]

Follow-up to my earlier note: you're right that an ECSQL class reference is always schema-qualified — but the reason the note now spells out "qualified" is the parenthesized-operand disambiguation. (bis.Element) takes the type-predicate path, whereas a parenthesized unqualified token like (MyProperty) parses as a value expression (the property). Calling out qualified vs unqualified is what makes that precise, so I incorporated it in 0e701b9 (Operators.md, NextVersion.md, and the test comment). Thanks for pushing on it.

[aruniverse]

✅ Addressed in 782b6e6f87. Operators.md now explains why qualification matters, treats unqualified (MyProperty) as a value expression, documents qualified non-class fallback, and preserves type-predicate precedence when a qualified name resolves to a class.

[hl662]

Nambot 🤖 (powered by GPT-5.5)

[aruniverse]

Two documentation gaps found after reviewing the IS (alias.prop) semantic fallback added in imodel-native commit 593ff15eb9. Both are in the itwinjs-core changelog/reference docs and don't reflect the full new behavior.

[aruniverse]

Tie-breaking parenthetical says 'property reading wins' — contradicts iModelCore/ECDb/CHANGES.md which says 'class reading wins'

Current text in this Note:

"a parenthesized unqualified name such as (MyProperty) is read as that property (if a name is both a class and a property, the property reading wins))"

The imodel-native CHANGES.md (commit 593ff15eb9) says the opposite for the qualified case:

"When a parenthesized qualified name is both a class and a valid property path, the class (type-predicate) reading wins."

Both statements can be technically reconciled (unqualified names are never classes in ECSQL, so 'property wins' is vacuously correct for unqualified names), but a reader who encounters the Operators.md parenthetical will take it as the general rule — and it's wrong for qualified names.

The parenthetical should either be removed or replaced with the correct general rule. Suggested rewrite:

Any other parenthesized right-hand operand is a value expression:
a parenthesized *unqualified* name such as `(MyProperty)` is read as that property (ECSQL
class names must always be schema-qualified, so there is no ambiguity here);
a parenthesized *qualified* name such as `(alias.prop)` that does **not** resolve to a class
is likewise a null-safe value comparison — if it **does** resolve to a class, the
type-predicate reading wins.

[khanaffan]

Fixed in 782b6e6f87 — reworded the tie-break: ECSQL class names are always schema-qualified so "property wins" was vacuously true for unqualified names, and I now explicitly state that when a qualified name resolves to a class, the type-predicate reading takes precedence (matching imodel-native CHANGES.md).

[aruniverse]

✅ Addressed in 782b6e6f87. The incorrect general "property reading wins" parenthetical was removed. The note now explicitly says that a qualified name resolving to a class takes the type-predicate reading, matching TryParseParenthesizedNameAsValueExp and native CHANGES.md.

[aruniverse]

Missing: the IS (alias.prop) qualified-name fallback is not mentioned here

The current text only covers unqualified names:

"A parenthesized unqualified name such as (prop2) is instead read as a value expression, so prop1 IS (prop2) is a null-safe comparison."

However, imodel-native commit 593ff15eb9 specifically added support for qualified names that don't resolve to a class — e.g. S1 IS (Foo.S2), Status IS (ts.Status.Active), f.S1 IS (f.Info.Code). These were previously rejected with a 'class not found' error. A user upgrading who writes x IS (alias.prop) would find no changelog mention that this is now valid.

Suggested addition after the existing unqualified-name sentence:

A parenthesized *qualified* name (e.g. `(alias.prop)` or `(ts.Status.Active)`) that does not
resolve to a known ECClass is also treated as a null-safe value expression instead of failing
with 'class not found'. When a qualified name is both a valid class and a valid property path,
the type-predicate (class) reading takes precedence.

[khanaffan]

Fixed in 782b6e6f87 — added a sentence to NextVersion.md noting that a parenthesized qualified name not resolving to a known ECClass (e.g. (alias.prop), (ts.Status.Active)) is now treated as a null-safe value expression instead of failing with "class not found", and that the type-predicate reading still wins if the qualified name does resolve to a class.

[aruniverse]

✅ Addressed in 782b6e6f87. NextVersion.md now calls out (alias.prop) / (ts.Status.Active) fallback behavior and the class-wins tie-break, so the late native behavior is represented in the user-facing change history.

[mergify]

This pull request is now in conflicts. Could you fix it @khanaffan? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

[mergify]

This pull request is now in conflicts. Could you fix it @khanaffan? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

[aruniverse]

The fallback is limited to dot-qualified names. The linked native implementation deliberately keeps schema:class / tablespace.schema:class colon forms as class-only syntax, even when class lookup fails. Please change "a parenthesized qualified name" to "a parenthesized dot-qualified name" here and in NextVersion.md, so the docs do not imply an unresolved colon form becomes a value expression.

[aruniverse]

This still states the old general tie-break. For an unqualified token like S2, ECSQL class references are not possible, so it simply takes the property/value-expression reading. For a qualified name that is both a class and a property path, the parser checks the class first and the type-predicate reading wins—as the updated docs now correctly say. Please reword this sentence to preserve that distinction.