feature - add portable plan-validation backend (#22)

.github/workflows/ci.yml

@@ -19,8 +19,8 @@ concurrency:
 
 env:
   CARGO_TERM_COLOR: always
-  INCAN_REF: release/v0.3
-  EXPECTED_INCAN_VERSION: 0.3.0
+  INCAN_REF: release/v0.4
+  EXPECTED_INCAN_VERSION: 0.4.0
   RUST_BACKTRACE: 1
   INCAN_NO_BANNER: 1
   INCAN_GENERATED_CARGO_TARGET_DIR: ${{ github.workspace }}/.incan-generated-cargo-target

docs/language/reference/execution_context.md

@@ -10,6 +10,7 @@ This page documents the public execution surface in the IncQL package. Normative
 - `BackendSelection` is the portable backend selection envelope stored by a session.
 - `BackendOption` carries adapter-specific configuration without adding one field per backend to `Session`.
 - `backends.DataFusion()` is the current reference backend configuration entry point.
+- `backends.PlanValidation(require_root_relation=true)` selects the portable validation backend. It validates plan and binding shape without executing rows.
 
 ## Construction
 
@@ -19,6 +20,7 @@ This page documents the public execution surface in the IncQL package. Normative
 | `Session.builder()`                                                | Create a builder for backend selection and configuration            |
 | `Session.builder().with_backend(selection).build()`                | Build a session from a portable backend-selection envelope           |
 | `Session.builder().with_datafusion(backends.DataFusion()).build()` | Build an explicit DataFusion-backed session                         |
+| `Session.builder().with_plan_validation(backends.PlanValidation(require_root_relation=true)).build()` | Build a validation-only backend session                             |
 
 ## Read and registration surface
 
@@ -45,6 +47,8 @@ All read APIs return `LazyFrame[T]`. They create deferred logical work; they do
 - `execute(...)` proves the plan can bind, lower, and run.
 - `collect(...)` performs that same work and materializes a local `DataFrame[T]`.
 
+The `plan_validation` backend supports `execute(...)` as a portable validation checkpoint. It does not materialize rows or write sinks, so `collect(...)`, `write(...)`, `write_csv(...)`, and `write_parquet(...)` return typed backend errors when selected. Use DataFusion for local row execution.
+
 ## Execution observations
 
 Observed execution methods preserve the ordinary session contracts while also returning runtime evidence. The ordinary `execute`, `collect`, and `write` methods use the same execution path internally and keep returning `Result[...]` values for compact application code.
@@ -73,7 +77,7 @@ Observed execution methods preserve the ordinary session contracts while also re
 | `context_targets`                       | `list[SemanticTarget]`        | Session or binding context targets attached to the attempt     |
 | `operation`                             | `ExecutionOperationKind`      | Operation family: `execute`, `collect`, or `write`             |
 | `status`                                | `ExecutionObservationStatus`  | Terminal status                                                |
-| `backend_name`                          | `str`                         | Selected backend name, currently `datafusion` by default       |
+| `backend_name`                          | `str`                         | Selected backend name, such as `datafusion` or `plan_validation` |
 | `adapter_version`                       | `Option[str]`                 | Adapter version when reported by the backend                   |
 | `requested_semantic_profile_id`         | `Option[str]`                 | Requested semantic profile identity when one is bound          |
 | `observed_semantic_profile_id`          | `Option[str]`                 | Observed semantic profile identity when the adapter reports one |

docs/release_notes/v0_1.md

@@ -31,6 +31,7 @@ Entries will be filled in as work lands (link RFCs and PRs when applicable).
 - **Quality observations:** RFC 034 adds quality assertion declarations, `quality { ... }` / `quality:` assertion-list syntax, and `Session.observe_quality(...)` / `Session.observe_quality_pair(...)` so relation, field, group, and explicit cross-relation checks produce structured quality observations. The first helper set covers row-count thresholds, null-rate thresholds, uniqueness, group row-count thresholds, and cross-relation row-count equality without treating failed checks as implicit filters or pipeline gates.
 - **Governed evidence:** RFC 035 adds governed attribute and policy checkpoint records so callers can attach provenance, confidence, status, authority, lifetime, visibility, diagnostics, and evidence references to semantic targets. Local plan inspection now emits conservative schema-derived governed attributes and planning observation checkpoints without turning IncQL into a policy engine.
 - **Execution:** Session-oriented read, execute, and write (reference backend per RFC 004), with `collect(...)` now producing structured `DataFrame` materialization metadata plus preview text instead of treating rendered text as the canonical contract. Session execution dispatch now routes through a backend adapter boundary over Substrait plans; DataFusion remains the first adapter rather than being encoded directly into Session state.
+- **Backend validation:** `backends.PlanValidation(...)` adds a validation-only backend selection for portable plan and binding checks. It supports `Session.execute(...)` as a plan-validation checkpoint and returns typed backend errors for row materialization or sink writes instead of delegating to DataFusion.
 - **Session API:** `Session.write(data, target)` now accepts typed sink descriptors such as `csv_sink(uri)` and `parquet_sink(uri)`, while the file-specific `write_csv(...)` and `write_parquet(...)` helpers remain as convenience methods.
 - **Documentation:** Current package behavior is documented under `docs/language/`, while RFCs remain design records rather than implementation diaries.
 

src/backends.incn

@@ -18,9 +18,21 @@ pub class DataFusion:
         return DataFusion(enable_optimizer=self.enable_optimizer)
 
 
+@derive(Clone)
+pub class PlanValidation:
+    """Portable backend configuration for plan and binding validation without local row execution."""
+
+    pub require_root_relation: bool
+
+    def clone(self) -> Self:
+        """Return one cloned plan-validation backend configuration."""
+        return PlanValidation(require_root_relation=self.require_root_relation)
+
+
 @derive(Clone)
 pub enum BackendKind(str):
     DataFusionEngine = "datafusion"
+    PlanValidationEngine = "plan_validation"
 
 
 @derive(Clone)
@@ -84,6 +96,19 @@ pub def datafusion_backend_from_selection(selection: BackendSelection) -> DataFu
     return DataFusion(enable_optimizer=_backend_bool_option(selection, "enable_optimizer", true))
 
 
+pub def plan_validation_backend_selection(backend: PlanValidation) -> BackendSelection:
+    """Encode plan-validation backend options into one portable backend-selection envelope."""
+    return BackendSelection(
+        kind=BackendKind.PlanValidationEngine,
+        options=[BackendOption(key="require_root_relation", value=_bool_option_value(backend.require_root_relation))],
+    )
+
+
+pub def plan_validation_backend_from_selection(selection: BackendSelection) -> PlanValidation:
+    """Decode plan-validation options from one backend-selection envelope."""
+    return PlanValidation(require_root_relation=_backend_bool_option(selection, "require_root_relation", true))
+
+
 pub def backend_kind_name(selection: BackendSelection) -> str:
     """Return one stable backend name for diagnostics and UX."""
     return selection.kind.value()

src/lib.incn

@@ -398,12 +398,15 @@ pub from backends import (
     BackendOption,
     BackendSelection,
     DataFusion,
+    PlanValidation,
     TableSource,
     arrow_source,
     csv_source,
     datafusion_backend_from_selection,
     datafusion_backend_selection,
     parquet_source,
+    plan_validation_backend_from_selection,
+    plan_validation_backend_selection,
 )
 pub from metadata import incql_version
 pub from session.domain import SinkKind, SinkTarget, csv_sink, parquet_sink, sink_kind_name

src/session/backend_dispatch.incn

@@ -2,8 +2,9 @@
 
 import std.async
 from rust::substrait::proto import Plan
+from rust::substrait::proto::plan_rel import RelType as PlanRelType
 from rust::incan_stdlib::async::runtime import block_on
-from backends import BackendKind, BackendSelection
+from backends import BackendKind, BackendSelection, plan_validation_backend_from_selection
 from dataset.materialization import DataFrameMaterialization
 from session.backend_types import BackendError, BackendErrorKind, BackendRegistration, backend_error
 from session.domain import SinkKind
@@ -26,6 +27,7 @@ pub def backend_execute_plan(
             match block_on(datafusion_execute_async(registrations, plan)):
                 Ok(result) => return result
                 Err(err) => return Err(backend_error(BackendErrorKind.RuntimeInitError, err.message()))
+        BackendKind.PlanValidationEngine => return _plan_validation_execute(selection, plan)
 
 
 pub def backend_collect_plan(
@@ -39,6 +41,13 @@ pub def backend_collect_plan(
             match block_on(datafusion_collect_materialization_async(registrations, plan)):
                 Ok(result) => return result
                 Err(err) => return Err(backend_error(BackendErrorKind.RuntimeInitError, err.message()))
+        BackendKind.PlanValidationEngine =>
+            return Err(
+                backend_error(
+                    BackendErrorKind.BackendExecutionError,
+                    "plan_validation backend validates plans but does not materialize rows",
+                ),
+            )
 
 
 pub def backend_write_plan(
@@ -60,3 +69,28 @@ pub def backend_write_plan(
                     match block_on(datafusion_write_parquet_async(registrations, plan, uri)):
                         Ok(result) => return result
                         Err(err) => return Err(backend_error(BackendErrorKind.RuntimeInitError, err.message()))
+        BackendKind.PlanValidationEngine =>
+            return Err(
+                backend_error(
+                    BackendErrorKind.BackendSinkError,
+                    "plan_validation backend validates plans but does not write sinks",
+                ),
+            )
+
+
+def _plan_validation_execute(selection: BackendSelection, plan: Plan) -> Result[None, BackendError]:
+    """Validate one portable plan shape without handing execution to a row engine."""
+    backend = plan_validation_backend_from_selection(selection)
+    if backend.require_root_relation and not _plan_has_root_relation(plan):
+        return Err(backend_error(BackendErrorKind.BackendPlanningError, "plan has no root relation"))
+    return Ok(None)
+
+
+def _plan_has_root_relation(plan: Plan) -> bool:
+    """Return whether the first plan relation is a real root relation with input."""
+    if len(plan.relations) == 0:
+        return false
+    if let Some(PlanRelType.Root(root)) = plan.relations[0].rel_type:
+        if let Some(_) = root.input:
+            return true
+    return false

src/session/types.incn

@@ -6,6 +6,7 @@ from aggregate_builders import aggregate_as
 from backends import (
     BackendSelection,
     DataFusion,
+    PlanValidation,
     TableSource,
     backend_kind_name,
     datafusion_backend_from_selection,
@@ -15,6 +16,8 @@ from backends import (
     csv_source,
     default_backend_selection,
     parquet_source,
+    plan_validation_backend_from_selection,
+    plan_validation_backend_selection,
 )
 from dataset import BoundedDataSet, DataFrame, LazyFrame, lazy_frame_named_table
 from evidence import (
@@ -153,6 +156,10 @@ pub class Session:
         """Return the current DataFusion backend configuration."""
         return datafusion_backend_from_selection(self._backend)
 
+    def plan_validation_backend(self) -> PlanValidation:
+        """Return the current plan-validation backend configuration."""
+        return plan_validation_backend_from_selection(self._backend)
+
     def registration_count(self) -> int:
         """Return the number of registered logical sources in this Session."""
         return len(self._registrations)
@@ -444,6 +451,10 @@ pub class SessionBuilder:
         """Select DataFusion backend options for the Session being built."""
         return self.with_backend(datafusion_backend_selection(backend.clone()))
 
+    def with_plan_validation(self, backend: PlanValidation) -> Self:
+        """Select the portable plan-validation backend for the Session being built."""
+        return self.with_backend(plan_validation_backend_selection(backend.clone()))
+
     def build(self) -> Session:
         """Build one Session from the current builder backend selection."""
         return Session(_backend=self._backend.clone(), _registrations=[])

tests/test_session_backend_plan_validation.incn

@@ -0,0 +1,170 @@
+"""Session tests for the portable plan-validation backend."""
+
+from rust::substrait::proto import Plan, PlanRel
+from rust::substrait::proto::plan_rel import RelType as PlanRelType
+from backends import PlanValidation, csv_source, plan_validation_backend_selection
+from dataset import LazyFrame
+from session import Session, SessionErrorKind, csv_sink
+from session.backend_dispatch import backend_execute_plan
+from session.backend_types import BackendErrorKind
+from substrait.plans import empty_plan
+from substrait.relations import read_named_table_rel
+from std.testing import assert_is_err, assert_is_ok
+
+
+@derive(Clone)
+pub model Order:
+    """Minimal row model used by backend validation tests."""
+
+    pub id: int
+
+
+def _plan_validation_session() -> Session:
+    """Return one Session configured with the plan-validation backend."""
+    return Session.builder().with_plan_validation(PlanValidation(require_root_relation=true)).build()
+
+
+def _non_root_plan() -> Plan:
+    """Return one plan whose first relation is not a root relation."""
+    return Plan(
+        version=None,
+        extension_urns=[],
+        extensions=[],
+        relations=[PlanRel(rel_type=Some(PlanRelType.Rel(read_named_table_rel("orders"))))],
+        advanced_extensions=None,
+        expected_type_urls=[],
+        parameter_bindings=[],
+        type_aliases=[],
+    )
+
+
+def test_session_backend_plan_validation__builder_selects_backend() -> None:
+    """Session builders should select and preserve the plan-validation backend envelope."""
+    # -- Arrange --
+    selection = plan_validation_backend_selection(PlanValidation(require_root_relation=false))
+
+    # -- Act --
+    session = Session.builder().with_backend(selection).build()
+    backend = session.plan_validation_backend()
+
+    # -- Assert --
+    assert session.backend_name() == "plan_validation", "builder should select the plan-validation backend"
+    assert backend.require_root_relation is false, "builder should preserve plan-validation backend options"
+
+
+def test_session_backend_plan_validation__execute_validates_registered_plan() -> None:
+    """execute should accept a bound named-table plan without using a row engine."""
+    # -- Arrange --
+    mut session = _plan_validation_session()
+
+    # -- Act & Assert --
+    assert_is_ok(
+        session.register("orders", csv_source("tests/fixtures/orders.csv")),
+        "fixture registration should succeed",
+    )
+    lazy: LazyFrame[Order] = assert_is_ok(session.table("orders"), "registered table lookup should succeed")
+
+    assert_is_ok(session.execute(lazy), "plan-validation backend should accept a bound named-table plan")
+
+
+def test_session_backend_plan_validation__read_csv_executes_validation_plan() -> None:
+    """read_csv should keep the portable validation backend on the normal read path."""
+    # -- Arrange --
+    mut session = _plan_validation_session()
+
+    # -- Act --
+    lazy: LazyFrame[Order] = assert_is_ok(
+        session.read_csv("orders", "tests/fixtures/orders.csv"),
+        "read_csv should bind the fixture source",
+    )
+
+    # -- Assert --
+    assert_is_ok(session.execute(lazy), "plan-validation backend should validate read_csv plans")
+
+
+def test_session_backend_plan_validation__requires_real_root_relation() -> None:
+    """execute should reject empty and non-root plans when root validation is required."""
+    # -- Arrange --
+    selection = plan_validation_backend_selection(PlanValidation(require_root_relation=true))
+
+    # -- Act --
+    empty_err = assert_is_err(
+        backend_execute_plan(selection.clone(), [], empty_plan()),
+        "empty plans should fail root validation",
+    )
+    non_root_err = assert_is_err(
+        backend_execute_plan(selection, [], _non_root_plan()),
+        "non-root plan relations should fail root validation",
+    )
+
+    # -- Assert --
+    assert empty_err.kind == BackendErrorKind.BackendPlanningError, empty_err.message
+    assert non_root_err.kind == BackendErrorKind.BackendPlanningError, non_root_err.message
+    assert empty_err.message.contains("no root relation"), empty_err.message
+    assert non_root_err.message.contains("no root relation"), non_root_err.message
+
+
+def test_session_backend_plan_validation__collect_reports_typed_unsupported_error() -> None:
+    """collect should report a typed backend error instead of pretending to materialize rows."""
+    # -- Arrange --
+    mut session = _plan_validation_session()
+    assert_is_ok(
+        session.register("orders", csv_source("tests/fixtures/orders.csv")),
+        "fixture registration should succeed",
+    )
+    lazy: LazyFrame[Order] = assert_is_ok(session.table("orders"), "registered table lookup should succeed")
+
+    # -- Act --
+    err = assert_is_err(session.collect(lazy), "plan-validation backend should not materialize rows")
+
+    # -- Assert --
+    assert err.kind == SessionErrorKind.BackendExecutionError, err.error_message()
+    assert err.message.contains("does not materialize rows"), err.error_message()
+
+
+def test_session_backend_plan_validation__write_reports_typed_unsupported_error() -> None:
+    """write should report a typed backend error instead of creating sink output."""
+    # -- Arrange --
+    mut session = _plan_validation_session()
+    assert_is_ok(
+        session.register("orders", csv_source("tests/fixtures/orders.csv")),
+        "fixture registration should succeed",
+    )
+    lazy: LazyFrame[Order] = assert_is_ok(session.table("orders"), "registered table lookup should succeed")
+
+    # -- Act --
+    err = assert_is_err(
+        session.write(lazy, csv_sink("tests/target/plan_validation_output.csv")),
+        "plan-validation backend should not write sinks",
+    )
+
+    # -- Assert --
+    assert err.kind == SessionErrorKind.BackendSinkError, err.error_message()
+    assert err.message.contains("does not write sinks"), err.error_message()
+
+
+def test_session_backend_plan_validation__write_helpers_report_typed_unsupported_error() -> None:
+    """write_csv and write_parquet should preserve typed unsupported sink errors."""
+    # -- Arrange --
+    mut session = _plan_validation_session()
+    assert_is_ok(
+        session.register("orders", csv_source("tests/fixtures/orders.csv")),
+        "fixture registration should succeed",
+    )
+    lazy: LazyFrame[Order] = assert_is_ok(session.table("orders"), "registered table lookup should succeed")
+
+    # -- Act --
+    csv_err = assert_is_err(
+        session.write_csv(lazy.clone(), "tests/target/plan_validation_output.csv"),
+        "plan-validation backend should not write CSV sinks",
+    )
+    parquet_err = assert_is_err(
+        session.write_parquet(lazy, "tests/target/plan_validation_output.parquet"),
+        "plan-validation backend should not write Parquet sinks",
+    )
+
+    # -- Assert --
+    assert csv_err.kind == SessionErrorKind.BackendSinkError, csv_err.error_message()
+    assert parquet_err.kind == SessionErrorKind.BackendSinkError, parquet_err.error_message()
+    assert csv_err.message.contains("does not write sinks"), csv_err.error_message()
+    assert parquet_err.message.contains("does not write sinks"), parquet_err.error_message()