feat(packaging): conda recipes for polars-bio and polars-config-meta - #426
Open
mwiewior wants to merge 4 commits into
Open
feat(packaging): conda recipes for polars-bio and polars-config-meta#426mwiewior wants to merge 4 commits into
mwiewior wants to merge 4 commits into
Conversation
Design for the first adoption sub-project: conda distribution and parallel-by-default. Records the survey finding that external friction sits entirely at the install and I/O edges rather than in the engine, and sequences the #421 write-ordering fix ahead of flipping the target_partitions default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings the bioconda recipe up to date and adds the conda-forge recipe for the one dependency that is not yet packaged, so polars-bio can be installed with conda rather than pip only. bioconda recipe (recipes/polars-bio): - Bump 0.21.0 -> 0.33.0 with the current sdist sha256. - Correct the pins to match pyproject.toml: datafusion >=53,<54 (was 50), pyarrow >=23.0.1,<25 (was >=21,<23). Drop typing-extensions, which is no longer a dependency. - Require Python >=3.11 rather than >=3.10, matching requires-python. - Move to a build.sh, which is needed to remove rust-toolchain.toml (the conda build supplies its own rustc and has no rustup to honour the pin) and to clear the -Ctarget-cpu / -Dwarnings RUSTFLAGS that upstream CI sets for PyPI wheels but which are wrong for a redistributable package. - Add git, cmake and make to the build environment: six dependencies resolve from git tags. - Replace the print-only test with an actual overlap query, so a package that imports but cannot execute fails the build. conda-forge recipe (staged-recipes/polars-config-meta): - Source the GitHub tag archive rather than the PyPI sdist, which omits LICENSE. - Declare polars as a run dependency. Upstream lists it as an optional extra but imports it at module scope, so the package is broken on import without it; a local conda-build caught this. - Fix the build backend to uv_build; the earlier draft said pdm-backend. Release automation: - Poll PyPI for the sdist instead of downloading immediately. Publishing is a separate workflow, so the release event can win the race and 404. - Sync the fork against upstream before branching, so the pull request does not carry unrelated commits from a stale fork. - Anchor the sed expressions and verify the substitutions took effect. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Matched against bioconda's existing Rust+maturin genomics recipe (oxbow):
- Add {{ stdlib('c') }}, which current conda-forge pinning requires and
whose absence fails dependency resolution.
- Run cargo-bundle-licenses and ship THIRDPARTY.yml in license_file, so the
licences of the 662-crate dependency graph are recorded.
- Add pkg-config to the build environment.
Also document a local-only trap in the README: conda-build without
conda-forge's pinning picks a macOS deployment target from the host SDK,
which can exceed the running OS. maturin then tags the wheel with that
version and the test phase fails pip check with "not supported on this
platform". Bioconda pins the target well below the runner, so this does not
affect CI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
bioconda-utils lint fails a recipe without run_exports (missing_run_exports). Pinned to the minor version rather than the major one, since polars-bio is still 0.x with an evolving API. Verified: 'bioconda-utils lint --packages polars-bio' reports All checks OK, and a local conda-build of the recipe produces polars-bio-0.33.0-py312h7217057_0.conda with pip check and the overlap smoke test passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Track A of the adoption roadmap (
docs/superpowers/specs/2026-07-27-adoption-zero-config-first-run-design.md): make polars-bio installable with conda, so it can be picked up by nf-core modules, Snakemake wrappers and biocontainers.Upstream pull requests opened from this work:
polars-config-meta— linter and linux_64 / osx_64 / win_64 builds all greenpolars-bio— draft, deliberately blocked on the aboveWhat changed
The recipe in
bioconda-recipe/was already tracked but stale: version 0.21.0,datafusion >=50,<51,pyarrow >=21,<23, atyping-extensionsdependency we no longer have, andpy<310. It is now 0.33.0 with pins matchingpyproject.toml.It also moves to a
build.sh, which is necessary rather than cosmetic:rust-toolchain.tomlpins channel 1.91.0. Conda builds supply their ownrustcand have norustup, so the pin has to be removed or the build fails outright.-Ctarget-cpu=skylake/apple-m1and-Dwarnings. The first emits instructions that fault on older hardware and the second turns any new compiler lint into a build failure, soRUSTFLAGSis cleared for the conda build.Plus
gitin the build environment (six dependencies resolve from git tags),{{ stdlib('c') }},run_exports, andcargo-bundle-licensesrecording the 662-crate licence graph intoTHIRDPARTY.yml.Two real bugs found while verifying
polars-config-metais broken on import without polars. It declaresdependencies = []and lists polars as an optional extra, but__init__.pyimports it at module scope. A localconda-buildcaught this; the conda-forge recipe declares polars as a run dependency.Verification
bioconda-utils lint --packages polars-bio→All checks OKconda-buildagainst conda-forge pinning producedpolars-bio-0.33.0-py312h7217057_0.conda;pip checkand the packaged overlap test both passpolars-config-metabuilt and tested aspolars-config-meta-0.3.4-py_0.condaNot included
Installation docs still say pip only. That should be updated when the packages are actually available on the channels, not before.
🤖 Generated with Claude Code