This repository was archived by the owner on Apr 21, 2026. It is now read-only.
feat: Sail - #2
Draft
shehabgamin wants to merge 6 commits into
Draft
Conversation
Thin wrapper around dbt-spark using the same pattern as dbt-redshift wrapping dbt-postgres. Connects to Sail via Spark Connect protocol with embedded (in-process) and remote connection modes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Updated test classes to reflect Spark-specific naming conventions, replacing "Sail" with "Spark" in class names. - Added pytest markers to skip tests for specific profiles, ensuring compatibility with Apache Spark and Databricks environments. - Refactored model definitions and configurations to align with Spark's requirements, including adjustments for data types and incremental strategies. - Introduced new tests for query timeout functionality and enhanced existing tests for incremental models, ensuring robust coverage for Spark integration.
Prune unsupported Sail adapter tests, capture the spark session debug report, and add shard-aware spark session runners so the branch reflects the intended session-focused coverage.
Move dbt-sail Hatch config into its own file, reuse dbt-spark relation macros, and keep the snapshot test on Delta so the adapter package and repro flow exercise the intended Sail path.
Match the Spark adapter test setup by forcing Delta for the snapshot check-cols test so it no longer falls back to parquet and fail before exercising snapshot behavior.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.