feat: add pipelined stack execution (#242)

Implement `StatementStack` and `StackOperation` for managing SQL execution sequences

Author: cofin

.gitignore

@@ -19,6 +19,7 @@ target/
 .vscode/
 .cursor/
 .zed/
+.cache
 .coverage*
 # files
 **/*.so

.pre-commit-config.yaml

@@ -17,7 +17,7 @@ repos:
       - id: mixed-line-ending
       - id: trailing-whitespace
   - repo: https://github.com/charliermarsh/ruff-pre-commit
-    rev: "v0.14.4"
+    rev: "v0.14.5"
     hooks:
       - id: ruff
         args: ["--fix"]
@@ -43,6 +43,7 @@ repos:
     rev: "v1.0.1"
     hooks:
       - id: sphinx-lint
+        args: ["--jobs", "1"]
   - repo: local
     hooks:
       - id: pypi-readme

AGENTS.md

@@ -185,6 +185,25 @@ SQLSpec is a type-safe SQL query mapper designed for minimal abstraction between
 - **Single-Pass Processing**: Parse once → transform once → validate once - SQL object is single source of truth
 - **Abstract Methods with Concrete Implementations**: Protocol defines abstract methods, base classes provide concrete sync/async implementations
 
+### Query Stack Implementation Guidelines
+
+- **Builder Discipline**
+    - `StatementStack` and `StackOperation` are immutable (`__slots__`, tuple storage). Every push helper returns a new stack; never mutate `_operations` in place.
+    - Validate inputs at push time (non-empty SQL, execute_many payloads, reject nested stacks) so drivers can assume well-formed operations.
+- **Adapter Responsibilities**
+    - Add a single capability gate per adapter (e.g., Oracle pipeline version check, `psycopg.capabilities.has_pipeline()`), return `super().execute_stack()` immediately when unsupported.
+    - Preserve `StackResult.result` by building SQL/Arrow results via `create_sql_result()` / `create_arrow_result()` instead of copying row data.
+    - Honor manual toggles via `driver_features={"stack_native_disabled": True}` and document the behavior in the adapter guide.
+- **Telemetry + Tracing**
+    - Always wrap adapter overrides with `StackExecutionObserver(self, stack, continue_on_error, native_pipeline=bool)`.
+    - Do **not** emit duplicate metrics; the observer already increments `stack.execute.*`, logs `stack.execute.start/complete/failed`, and publishes the `sqlspec.stack.execute` span.
+- **Error Handling**
+    - Wrap driver exceptions in `StackExecutionError` with `operation_index`, summarized SQL (`describe_stack_statement()`), adapter name, and execution mode.
+    - Continue-on-error stacks append `StackResult.from_error()` and keep executing. Fail-fast stacks roll back (if they started the transaction) before re-raising the wrapped error.
+- **Testing Expectations**
+    - Add integration tests under `tests/integration/test_adapters/<adapter>/test_driver.py::test_*statement_stack*` that cover native path, sequential fallback, and continue-on-error.
+    - Guard base behavior (empty stacks, large stacks, transaction boundaries) via `tests/integration/test_stack_edge_cases.py`.
+
 ### Driver Parameter Profile Registry
 
 - All adapter parameter defaults live in `DriverParameterProfile` entries inside `sqlspec/core/parameters.py`.

docs/changelog.rst

@@ -10,6 +10,14 @@ SQLSpec Changelog
 Recent Updates
 ==============
 
+Query Stack Documentation Suite
+--------------------------------
+
+- Expanded the :doc:`/reference/query-stack` API reference (``StatementStack``, ``StackResult``, driver hooks, and ``StackExecutionError``) with the high-level workflow, execution modes, telemetry, and troubleshooting tips.
+- Added :doc:`/examples/patterns/stacks/query_stack_example` that runs the same stack against SQLite and AioSQLite.
+- Captured the detailed architecture and performance guidance inside the internal specs workspace for future agent runs.
+- Updated every adapter reference with a **Query Stack Support** section so behavior is documented per database.
+
 Migration Convenience Methods on Config Classes
 ------------------------------------------------
 

docs/examples/README.md

@@ -6,6 +6,7 @@ This directory now mirrors the way developers explore SQLSpec:
 - `frameworks/` groups runnable apps (Litestar for now) that rely on lightweight backends (aiosqlite, duckdb).
 - `adapters/` holds connection-focused snippets for production drivers such as asyncpg, psycopg, and oracledb.
 - `patterns/` demonstrates SQL builder usage, migrations, and multi-tenant routing.
+- `arrow/` collects Arrow integration demos so advanced exports stay discoverable without bloating other folders.
 - `loaders/` shows how to hydrate SQL from files for quick demos.
 - `extensions/` keeps integration-specific samples (Adapter Development Kit in this pass).
 

docs/examples/arrow/__init__.py

@@ -0,0 +1,3 @@
+"""Arrow integration examples for SQLSpec."""
+
+__all__ = ()

docs/examples/arrow_basic_usage.py renamed to docs/examples/arrow/arrow_basic_usage.py

@@ -0,0 +1,16 @@
+Arrow: Basic Usage
+==================
+
+Demonstrate the ``select_to_arrow()`` helper across multiple adapters and
+conversion targets (native Arrow, pandas, polars, and Parquet exports).
+
+.. code-block:: console
+
+   uv run python docs/examples/arrow/arrow_basic_usage.py
+
+Source
+------
+
+.. literalinclude:: arrow_basic_usage.py
+   :language: python
+   :linenos:

docs/examples/arrow/arrow_basic_usage.rst

@@ -91,6 +91,19 @@ Patterns
      - Routing requests to dedicated SQLite configs per tenant slug.
    * - ``patterns/configs/multi_adapter_registry.py``
      - Register multiple adapters on a single SQLSpec registry.
+   * - ``patterns/stacks/query_stack_example.py``
+     - Immutable StatementStack workflow executed against SQLite and AioSQLite drivers.
+
+Arrow
+-----
+
+.. list-table:: Arrow-powered exports
+   :header-rows: 1
+
+   * - File
+     - Scenario
+   * - ``arrow/arrow_basic_usage.py``
+     - ``select_to_arrow()`` walkthrough covering native Arrow, pandas, polars, and Parquet exports.
 
 Loaders
 -------
@@ -142,4 +155,6 @@ Shared Utilities
    frameworks/starlette/aiosqlite_app
    frameworks/flask/sqlite_app
    patterns/configs/multi_adapter_registry
+   patterns/stacks/query_stack_example
+   arrow/arrow_basic_usage
    README

docs/examples/index.rst

@@ -0,0 +1,3 @@
+"""Statement stack examples for SQLSpec documentation."""
+
+__all__ = ()