diff --git a/.claude/agents/compat-tester.md b/.claude/agents/compat-tester.md index f37309f7..f330f2d3 100644 --- a/.claude/agents/compat-tester.md +++ b/.claude/agents/compat-tester.md @@ -14,14 +14,14 @@ tests with candle as the torch backend. - **Config**: `compat/transformers/models.yaml` (model tiers), `compat/transformers/xfail.yaml` (known failures) - **Patches**: `compat/transformers/conftest.py` (version spoof, module stubs, etc.) - **Source code**: `src/candle/` -- **Conda env**: `mindnlp` (`source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp ...`) +- **Conda env**: `candle` (`source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle ...`) ## Workflow ### Step 1: Run Tests ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ env USE_CANDLE=1 python compat/transformers/run.py --model MODEL \ --json-report /tmp/compat-report.json -v --tb=short ``` diff --git a/.claude/agents/pytorch-tester.md b/.claude/agents/pytorch-tester.md index 41e83a3f..0dd8638d 100644 --- a/.claude/agents/pytorch-tester.md +++ b/.claude/agents/pytorch-tester.md @@ -14,14 +14,14 @@ test suite with candle as the torch backend. - **Config**: `compat/pytorch/tests.yaml` (test tiers), `compat/pytorch/xfail.yaml` (known failures) - **Patches**: `compat/pytorch/conftest.py` (version spoof, module stubs, etc.) - **Source code**: `src/candle/` -- **Conda env**: `mindnlp` (`source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp ...`) +- **Conda env**: `candle` (`source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle ...`) ## Workflow ### Step 1: Run Tests ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ env USE_CANDLE=1 python compat/pytorch/run.py --tier TIER \ --json-report /tmp/pt-report.json -v --tb=short ``` @@ -29,7 +29,7 @@ source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ Or for a single file: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ env USE_CANDLE=1 python compat/pytorch/run.py --file FILE \ --json-report /tmp/pt-report.json -v --tb=short ``` diff --git a/.claude/agents/test-runner.md b/.claude/agents/test-runner.md index fd651192..b5098c49 100644 --- a/.claude/agents/test-runner.md +++ b/.claude/agents/test-runner.md @@ -6,10 +6,10 @@ You are a specialized test execution and bug-fixing agent for the Candle project Candle is a PyTorch-compatible ML framework (`import candle as torch`) with custom dispatch system, autograd engine, and multi-backend support (CPU, MPS, CUDA, NPU). -- **Test Command**: `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests// -v --tb=short` +- **Test Command**: `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests// -v --tb=short` - **Test Directories**: `tests/cpu/`, `tests/mps/`, `tests/contract/`, `tests/npu/`, `tests/cuda/` - **Source Code**: `src/candle/` -- **Conda Environment**: `mindnlp` (via `conda run -n mindnlp`) +- **Conda Environment**: `candle` (via `conda run -n candle`) ## Your Responsibilities @@ -31,15 +31,15 @@ git pull upstream main ```bash # CPU tests -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/ tests/contract/ -v --tb=short # MPS tests (macOS Apple Silicon only) -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/mps/ -v --tb=short # Specific test file -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/test_ops_cpu.py -v --tb=short ``` @@ -70,7 +70,7 @@ Apply targeted fixes: Re-run the same tests plus the full suite to check for regressions: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/ tests/contract/ -v --tb=short ``` diff --git a/.claude/commands/compat.md b/.claude/commands/compat.md index b14e6230..e9f96f76 100644 --- a/.claude/commands/compat.md +++ b/.claude/commands/compat.md @@ -19,7 +19,7 @@ Run transformers compatibility tests against candle and report results. ```bash source ~/miniconda3/etc/profile.d/conda.sh && \ -conda run -n mindnlp env USE_CANDLE=1 \ +conda run -n candle env USE_CANDLE=1 \ python compat/transformers/run.py $PARSED_ARGS \ --json-report /tmp/compat-report.json \ -v --tb=short diff --git a/.github/ISSUE_TEMPLATE/ai-bug-report.yml b/.github/ISSUE_TEMPLATE/ai-bug-report.yml index a2baf7a3..5690eed9 100644 --- a/.github/ISSUE_TEMPLATE/ai-bug-report.yml +++ b/.github/ISSUE_TEMPLATE/ai-bug-report.yml @@ -53,7 +53,7 @@ body: description: Must be runnable as-is. Use `import candle as torch`. render: shell placeholder: | - source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ + source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -c "import candle as torch; ..." validations: required: true diff --git a/.github/PULL_REQUEST_TEMPLATE/ai.md b/.github/PULL_REQUEST_TEMPLATE/ai.md index f9758879..c29866dd 100644 --- a/.github/PULL_REQUEST_TEMPLATE/ai.md +++ b/.github/PULL_REQUEST_TEMPLATE/ai.md @@ -25,7 +25,7 @@ ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest ... +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest ... pylint src/candle/ --rcfile=.github/pylint.conf ``` diff --git a/.github/PULL_REQUEST_TEMPLATE/human.md b/.github/PULL_REQUEST_TEMPLATE/human.md index 4b85d80e..2f8e5567 100644 --- a/.github/PULL_REQUEST_TEMPLATE/human.md +++ b/.github/PULL_REQUEST_TEMPLATE/human.md @@ -12,8 +12,8 @@ ## Test plan - [ ] Not run — explain why below -- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short` -- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/mps/ -v --tb=short` (if MPS-related) +- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short` +- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/mps/ -v --tb=short` (if MPS-related) - [ ] `pylint src/candle/ --rcfile=.github/pylint.conf` ## Checklist diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index c46f875c..198af0aa 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -12,8 +12,8 @@ ## Test plan - [ ] Not run — explain why below -- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short` -- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/mps/ -v --tb=short` (if MPS-related) +- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short` +- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/mps/ -v --tb=short` (if MPS-related) - [ ] `pylint src/candle/ --rcfile=.github/pylint.conf` ## Checklist diff --git a/.github/scripts/trigger_openi_upstream_ccache.txt b/.github/scripts/trigger_openi_upstream_ccache.txt new file mode 100644 index 00000000..ad449f53 --- /dev/null +++ b/.github/scripts/trigger_openi_upstream_ccache.txt @@ -0,0 +1 @@ +# single smoke rerun 1774847023 diff --git a/CLAUDE.md b/CLAUDE.md index e4596acb..81571629 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -46,7 +46,7 @@ candle/ ## Environment -- **Conda**: `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp ...` +- **Conda**: `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle ...` - **Platform**: macOS Apple Silicon (Darwin), local MPS hardware available - **Python**: 3.11 @@ -184,11 +184,11 @@ Do NOT open a PR if pylint fails. Fix all issues first. ```bash # CPU + contract tests (always required for src/candle/ changes) -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/ tests/contract/ -v --tb=short # MPS tests (required on macOS Apple Silicon when src/candle/ changes) -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/mps/ -v --tb=short ``` @@ -242,11 +242,11 @@ CI is a safety net, NOT a substitute for local testing. Do NOT push untested cod ```bash # CPU tests -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/ tests/contract/ -v --tb=short # MPS tests (macOS Apple Silicon only) -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/mps/ -v --tb=short ``` @@ -288,7 +288,7 @@ source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ ## Troubleshooting -- **Tests not running**: Ensure conda env is activated (`conda run -n mindnlp`) +- **Tests not running**: Ensure conda env is activated (`conda run -n candle`) - **MPS tests skipped locally**: Verify `pyobjc-framework-Metal` is installed - **MPS tests skipped on CI**: Check that `requirements-test-mps.txt` includes `pyobjc-framework-Metal` - **Git push fails**: Check push access, uncommitted changes, branch existence on remote diff --git a/docs/plans/2026-03-08-mindtorch-to-candle-migration.md b/docs/plans/2026-03-08-mindtorch-to-candle-migration.md index e4585814..ed743ae8 100644 --- a/docs/plans/2026-03-08-mindtorch-to-candle-migration.md +++ b/docs/plans/2026-03-08-mindtorch-to-candle-migration.md @@ -2,7 +2,7 @@ > **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. -**Goal:** Migrate the mindtorch_v2 codebase from the mindnlp repository to the standalone candle repository, renaming all references from `mindtorch_v2` to `candle`, and setting up CI/CD, tests, and packaging. +**Goal:** Migrate the mindtorch_v2 codebase from the legacy source repository to the standalone candle repository, renaming all references from `mindtorch_v2` to `candle`, and setting up CI/CD, tests, and packaging. **Architecture:** The candle repository will be a standalone Python package with `src/candle/` layout. All internal references to `mindtorch_v2` will be replaced with `candle`. GitHub Actions will provide pylint checks, unit testing, and wheel releases. The torch proxy system will be preserved for transparent PyTorch compatibility. @@ -12,8 +12,8 @@ ## Source Analysis -- **Source code:** `/Users/lvyufeng/Projects/mindnlp/src/mindtorch_v2/` (384 files) -- **Tests:** `/Users/lvyufeng/Projects/mindnlp/tests/mindtorch_v2/` (306 files) +- **Source code:** `/path/to/legacy-repo/src/mindtorch_v2/` (384 files) +- **Tests:** `/path/to/legacy-repo/tests/mindtorch_v2/` (306 files) - **Target:** `/Users/lvyufeng/Projects/candle/` (empty repo, MIT license) - **References to rename:** 192 files, ~584 occurrences of `mindtorch_v2` - **Also rename:** `MINDTORCH_TEST_FORCE_CPU_ONLY` → `CANDLE_TEST_FORCE_CPU_ONLY` @@ -23,12 +23,12 @@ ### Task 1: Copy Source Code to Candle Repository **Files:** -- Copy: `mindnlp/src/mindtorch_v2/**` → `candle/src/candle/**` +- Copy: `legacy-repo/src/mindtorch_v2/**` → `candle/src/candle/**` **Step 1: Copy the source tree** ```bash -cp -r /Users/lvyufeng/Projects/mindnlp/src/mindtorch_v2/ /Users/lvyufeng/Projects/candle/src/candle/ +cp -r /path/to/legacy-repo/src/mindtorch_v2/ /Users/lvyufeng/Projects/candle/src/candle/ ``` **Step 2: Remove `__pycache__` directories** @@ -58,12 +58,12 @@ git commit -m "feat: copy mindtorch_v2 source code as candle base" ### Task 2: Copy Tests to Candle Repository **Files:** -- Copy: `mindnlp/tests/mindtorch_v2/**` → `candle/tests/` +- Copy: `legacy-repo/tests/mindtorch_v2/**` → `candle/tests/` **Step 1: Copy the test tree** ```bash -cp -r /Users/lvyufeng/Projects/mindnlp/tests/mindtorch_v2/ /Users/lvyufeng/Projects/candle/tests/ +cp -r /path/to/legacy-repo/tests/mindtorch_v2/ /Users/lvyufeng/Projects/candle/tests/ ``` **Step 2: Remove `__pycache__` directories** @@ -323,11 +323,11 @@ git commit -m "feat: add pytest configuration" **Files:** - Create: `candle/.github/pylint.conf` -**Step 1: Copy pylint.conf from mindnlp** +**Step 1: Copy pylint.conf from the legacy repository** ```bash mkdir -p /Users/lvyufeng/Projects/candle/.github -cp /Users/lvyufeng/Projects/mindnlp/.github/pylint.conf /Users/lvyufeng/Projects/candle/.github/pylint.conf +cp /path/to/legacy-repo/.github/pylint.conf /Users/lvyufeng/Projects/candle/.github/pylint.conf ``` **Step 2: Commit** @@ -498,7 +498,7 @@ git commit -m "feat: add GitHub Actions release workflow" **Step 1: Create run_test.py** -Adapt from `mindnlp/tests/run_test_v2.py`, replacing all `mindtorch_v2` references with `candle`: +Adapt from `legacy-repo/tests/run_test_v2.py`, replacing all `mindtorch_v2` references with `candle`: - Line 34: `from candle._torch_proxy import install` (if torch_proxy exists) - Line 39: docstrings referencing `candle` instead of `mindtorch_v2` @@ -591,14 +591,14 @@ git commit -m "chore: final cleanup and verification" ## Summary of Changes -| Component | Source (mindnlp) | Target (candle) | +| Component | Source (legacy repo) | Target (candle) | |-----------|-----------------|-----------------| | Source code | `src/mindtorch_v2/` | `src/candle/` | | Tests | `tests/mindtorch_v2/` | `tests/` | | Package name | `mindtorch_v2` | `candle` | | CI workflow | `.github/workflows/ci_pipeline.yaml` | `.github/workflows/ci.yaml` | | Release workflow | `.github/workflows/make_wheel_releases.yml` | `.github/workflows/release.yaml` | -| Setup | `setup.py` (shared with mindnlp) | `pyproject.toml` (standalone) | +| Setup | `setup.py` (shared in the legacy repo) | `pyproject.toml` (standalone) | | Pylint config | `.github/pylint.conf` | `.github/pylint.conf` | | Test runner | `tests/run_test_v2.py` | `tests/run_test.py` | | Env var | `MINDTORCH_TEST_FORCE_CPU_ONLY` | `CANDLE_TEST_FORCE_CPU_ONLY` | diff --git a/docs/plans/2026-03-10-fsdp2-mvp-impl-plan.md b/docs/plans/2026-03-10-fsdp2-mvp-impl-plan.md index 99b166ab..f71ef505 100644 --- a/docs/plans/2026-03-10-fsdp2-mvp-impl-plan.md +++ b/docs/plans/2026-03-10-fsdp2-mvp-impl-plan.md @@ -12,7 +12,7 @@ **Test command:** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ -v --tb=short -x ``` --- @@ -122,7 +122,7 @@ def test_hook_removal(): **Step 2: Run tests to verify they fail** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_module_hooks.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_module_hooks.py -v --tb=short -x ``` Expected: `test_forward_pre_hook_prepend` FAILS (prepend ignored, order is `["first", "prepended"]`). @@ -195,7 +195,7 @@ def __call__(self, *args, **kwargs): **Step 4: Run tests to verify they pass** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_module_hooks.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_module_hooks.py -v --tb=short ``` Expected: ALL PASS @@ -203,7 +203,7 @@ Expected: ALL PASS **Step 5: Run existing tests to verify no regression** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 ``` Expected: No new failures. Existing hooks use the old format (just `hook`, not tuple), so the `isinstance(hook_entry, tuple)` check provides backward compatibility. @@ -338,7 +338,7 @@ def test_plain_tensors_skip_torch_function(): **Step 2: Run tests to verify they fail** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_torch_function.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_torch_function.py -v --tb=short -x ``` Expected: FAIL — `__torch_function__` not called because `_functional.py` doesn't check for it yet. @@ -440,7 +440,7 @@ For MVP, add the check to `add` and `matmul` first to prove the mechanism works. **Step 5: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_torch_function.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_torch_function.py -v --tb=short ``` Expected: ALL PASS @@ -448,7 +448,7 @@ Expected: ALL PASS **Step 6: Run existing tests for regression** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 ``` Expected: No new failures (`_has_torch_function` fast-returns `False` for plain Tensors). @@ -546,7 +546,7 @@ def test_plain_tensors_unaffected(): **Step 2: Run tests to verify they fail** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_torch_dispatch.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_torch_dispatch.py -v --tb=short -x ``` Expected: FAIL — no `__torch_dispatch__` mechanism exists yet. @@ -642,7 +642,7 @@ def __torch_dispatch__(cls, func, types, args=(), kwargs=None): **Step 7: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_torch_dispatch.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_torch_dispatch.py -v --tb=short ``` Expected: ALL PASS @@ -650,7 +650,7 @@ Expected: ALL PASS **Step 8: Run full test suite for regression** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 ``` Expected: No new failures. The `Python` key is only added when subclass tensors are detected, so plain tensor operations are unaffected. Key reorder only affects priority when the `Python` key is present. @@ -724,7 +724,7 @@ def test_shard_repr(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_placement.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_placement.py -v --tb=short -x ``` **Step 3: Implement placement types** @@ -795,7 +795,7 @@ Note: `DTensor` will be added to the import in Task 6. **Step 5: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_placement.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_placement.py -v --tb=short ``` **Step 6: Commit** @@ -849,7 +849,7 @@ def test_device_mesh_size(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_device_mesh.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_device_mesh.py -v --tb=short -x ``` **Step 3: Implement DeviceMesh** @@ -930,7 +930,7 @@ def init_device_mesh(device_type, mesh_shape, *, mesh_dim_names=None): **Step 4: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_device_mesh.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_device_mesh.py -v --tb=short ``` **Step 5: Commit** @@ -1044,7 +1044,7 @@ def test_dtensor_sharded_blocks_compute(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_dtensor.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_dtensor.py -v --tb=short -x ``` **Step 3: Implement DTensor** @@ -1226,7 +1226,7 @@ __all__ = ["DTensor", "DTensorSpec", "TensorMeta", "Placement", "Shard", "Replic **Step 5: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_dtensor.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_dtensor.py -v --tb=short ``` **Step 6: Commit** @@ -1341,7 +1341,7 @@ def test_fsdp_param_group_lifecycle(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fsdp_param.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fsdp_param.py -v --tb=short -x ``` **Step 3: Create the FSDP directory structure** @@ -1563,7 +1563,7 @@ Create `src/candle/distributed/_composable/fsdp/__init__.py`: **Step 8: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fsdp_param.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fsdp_param.py -v --tb=short ``` **Step 9: Commit** @@ -1653,7 +1653,7 @@ def test_fsdp_state_post_forward_reshards(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fsdp_state.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fsdp_state.py -v --tb=short -x ``` **Step 3: Implement FSDPState** @@ -1770,7 +1770,7 @@ def _extract_tensors_from_output(output): **Step 4: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fsdp_state.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fsdp_state.py -v --tb=short ``` **Step 5: Commit** @@ -1899,7 +1899,7 @@ def test_fully_shard_excludes_child_params(): **Step 2: Run to verify failure** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fully_shard.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fully_shard.py -v --tb=short -x ``` **Step 3: Implement `fully_shard()` and FSDPModule** @@ -2061,13 +2061,13 @@ from .fsdp import fully_shard, FSDPModule **Step 5: Run tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fully_shard.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fully_shard.py -v --tb=short ``` **Step 6: Run full test suite** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 ``` **Step 7: Commit** @@ -2192,7 +2192,7 @@ def test_fsdp_params_are_dtensor_between_iterations(): **Step 2: Run integration tests** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_fsdp_integration.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_fsdp_integration.py -v --tb=short ``` **Step 3: Debug and fix any issues** @@ -2206,7 +2206,7 @@ If tests fail, investigate and fix. Common issues: **Step 4: Run full CPU test suite** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/ tests/contract/ -v --tb=short -x 2>&1 | tail -30 ``` **Step 5: Commit** @@ -2292,7 +2292,7 @@ def test_fsdp2_gloo_forward_backward(): **Step 2: Run multi-process test** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_fsdp2_gloo.py -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_fsdp2_gloo.py -v --tb=short -x ``` **Step 3: Debug and fix issues** diff --git a/docs/plans/2026-03-23-generated-autograd-drift-registration-followup-plan.md b/docs/plans/2026-03-23-generated-autograd-drift-registration-followup-plan.md index 40303ddb..a2732641 100644 --- a/docs/plans/2026-03-23-generated-autograd-drift-registration-followup-plan.md +++ b/docs/plans/2026-03-23-generated-autograd-drift-registration-followup-plan.md @@ -63,7 +63,7 @@ def test_compiled_variable_type_surface_matches_generated_safe_registration_subs Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -v --tb=short ``` @@ -143,7 +143,7 @@ This is a placeholder red test you will strengthen immediately after inventory l Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -k "manual_wrapper" -v --tb=short ``` @@ -177,7 +177,7 @@ Do the same for `diff`. If you cannot safely express it in the current generator Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m tools.autograd.gen_autograd ``` @@ -194,7 +194,7 @@ Update the placeholder so it checks that every “manual” wrapper is either: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -v --tb=short ``` @@ -239,7 +239,7 @@ def test_registration_generic_aliases_have_backing_wrapper_symbols(): Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -k "generic_aliases_have_backing_wrapper_symbols" -v --tb=short ``` @@ -268,7 +268,7 @@ If a generic alias is semantically ambiguous, then `gen_registration.py` should Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m tools.autograd.gen_autograd ``` @@ -312,7 +312,7 @@ def test_registration_splits_compiled_safe_and_python_legacy_sections(): Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -k "splits_compiled_safe_and_python_legacy_sections" -v --tb=short ``` @@ -357,7 +357,7 @@ The generated-safe/legacy split should be data-driven from the inventory created Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m tools.autograd.gen_autograd ``` @@ -383,9 +383,9 @@ def test_sum_to_size_post_registration_still_resolves_with_compiled_modules_pres Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -v --tb=short -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/test_declarative_autograd.py -k "registration_imports_when_compiled_variable_type_is_present or sum_to_size_post_registration_still_resolves" -v --tb=short ``` @@ -427,7 +427,7 @@ def test_registration_keeps_python_fallback_for_legacy_ops(): Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/test_declarative_autograd.py -k "prefers_compiled_generated_variable_type_for_generated_safe_ops or keeps_python_fallback_for_legacy_ops" -v --tb=short ``` @@ -438,7 +438,7 @@ Expected: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python setup.py build_ext --inplace ``` @@ -449,7 +449,7 @@ Expected: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/test_declarative_autograd.py -k "generated_cython or registration or generated_package" -v --tb=short ``` @@ -474,7 +474,7 @@ git commit -m "test: verify compiled generated registration split" Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python setup.py build_ext --inplace ``` @@ -484,7 +484,7 @@ Expected: success. Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/contract/test_generated_registration_coverage.py -v --tb=short ``` @@ -494,7 +494,7 @@ Expected: PASS. Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python -m pytest tests/cpu/test_declarative_autograd.py -v --tb=short ``` @@ -504,7 +504,7 @@ Expected: no new import-time or registration-time failures. Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ pylint src/candle/ --rcfile=.github/pylint.conf ``` diff --git a/docs/plans/2026-03-23-parallel-roadmap-implementation-plan.md b/docs/plans/2026-03-23-parallel-roadmap-implementation-plan.md index 437c9c2b..3806e142 100644 --- a/docs/plans/2026-03-23-parallel-roadmap-implementation-plan.md +++ b/docs/plans/2026-03-23-parallel-roadmap-implementation-plan.md @@ -82,7 +82,7 @@ Suggested test names: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_work_async.py tests/distributed/test_ddp_async_overlap.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_work_async.py tests/distributed/test_ddp_async_overlap.py -v --tb=short ``` Expected: @@ -143,7 +143,7 @@ Do not delete the Python fallback. Make it clearly secondary and keep import fal Run: ```bash python setup.py build_ext --inplace -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_work_async.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_work_async.py -v --tb=short ``` Expected: @@ -192,7 +192,7 @@ The hot path must be Cython. At minimum: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_work_async.py tests/distributed/test_ddp_async_overlap.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_work_async.py tests/distributed/test_ddp_async_overlap.py -v --tb=short ``` Expected: @@ -239,7 +239,7 @@ Keep Python responsible only for module-level orchestration and hook registratio Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_ddp_async_overlap.py tests/distributed/test_ddp_bucket_fastpath.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_ddp_async_overlap.py tests/distributed/test_ddp_bucket_fastpath.py -v --tb=short ``` Expected: @@ -281,8 +281,8 @@ Document these as explicit success criteria in the benchmark script header: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python benchmarks/distributed_ddp_overlap.py -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_ddp_async_overlap.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python benchmarks/distributed_ddp_overlap.py +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_ddp_async_overlap.py -v --tb=short ``` Expected: @@ -327,7 +327,7 @@ Replace hard-fail exports with a deliberate public API decision. Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_fsdp_public_api.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_fsdp_public_api.py -v --tb=short ``` Expected: @@ -383,7 +383,7 @@ Move repeated per-parameter loops into Cython. Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_fsdp_public_api.py tests/distributed/test_fsdp_shard_fastpath.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_fsdp_public_api.py tests/distributed/test_fsdp_shard_fastpath.py -v --tb=short ``` Expected: @@ -419,7 +419,7 @@ Cover at least: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_fsdp_npu_integration.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_fsdp_npu_integration.py -v --tb=short ``` Expected: @@ -462,7 +462,7 @@ Make sure checkpoint planning understands: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_distributed_checkpoint.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_distributed_checkpoint.py -v --tb=short ``` Expected: @@ -503,7 +503,7 @@ Planner policy can stay Python. The repeated metadata and chunk math should move Run: ```bash python setup.py build_ext --inplace -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_distributed_checkpoint.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_distributed_checkpoint.py -v --tb=short ``` Expected: @@ -548,7 +548,7 @@ Cover: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cuda/test_nccl_collectives.py tests/cuda/test_cuda_stream_api.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cuda/test_nccl_collectives.py tests/cuda/test_cuda_stream_api.py -v --tb=short ``` Expected: @@ -601,7 +601,7 @@ Implement at least: Run: ```bash python setup.py build_ext --inplace -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cuda/test_nccl_collectives.py tests/cuda/test_cuda_stream_api.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cuda/test_nccl_collectives.py tests/cuda/test_cuda_stream_api.py -v --tb=short ``` Expected: @@ -648,7 +648,7 @@ Cover: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_dtensor_redistribute.py tests/distributed/test_tensor_parallel.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_dtensor_redistribute.py tests/distributed/test_tensor_parallel.py -v --tb=short ``` Expected: @@ -692,7 +692,7 @@ Python should choose the algorithm. Cython should do the repeated layout math an Run: ```bash python setup.py build_ext --inplace -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_dtensor_redistribute.py tests/distributed/test_tensor_parallel.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_dtensor_redistribute.py tests/distributed/test_tensor_parallel.py -v --tb=short ``` Expected: @@ -733,7 +733,7 @@ Do not add a full scheduler zoo. Start with: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/distributed/test_pipeline_parallel.py -v --tb=short +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/distributed/test_pipeline_parallel.py -v --tb=short ``` Expected: @@ -787,7 +787,7 @@ Examples: Run: ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest tests/cpu/test_threading_api.py tests/distributed -v --tb=short -x +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest tests/cpu/test_threading_api.py tests/distributed -v --tb=short -x ``` Expected: diff --git a/docs/superpowers/plans/2026-03-10-pytorch-test-migration.md b/docs/superpowers/plans/2026-03-10-pytorch-test-migration.md index 7fd3d3d6..2f2c0352 100644 --- a/docs/superpowers/plans/2026-03-10-pytorch-test-migration.md +++ b/docs/superpowers/plans/2026-03-10-pytorch-test-migration.md @@ -156,7 +156,7 @@ See each subdirectory's README for usage. - [ ] **Step 6: Verify transformers compat still works** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ env USE_CANDLE=1 python compat/transformers/run.py --model bert \ --json-report /tmp/tf-verify.json --tb=line python compat/transformers/run.py --summarize /tmp/tf-verify.json @@ -871,16 +871,16 @@ git commit -m "feat: add candle.testing._internal.common_device_type (instantiat - [ ] **Step 1: Clone pytorch and setup** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ pip install -r compat/pytorch/requirements.txt -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ python compat/pytorch/run.py --setup-only ``` - [ ] **Step 2: Run test_tensor.py smoke test** ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp \ +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle \ env USE_CANDLE=1 python compat/pytorch/run.py \ --file test_tensor.py \ --json-report /tmp/pytorch-smoke.json \ diff --git a/docs/superpowers/plans/2026-03-24-github-templates.md b/docs/superpowers/plans/2026-03-24-github-templates.md index 616b24b2..dcf9002b 100644 --- a/docs/superpowers/plans/2026-03-24-github-templates.md +++ b/docs/superpowers/plans/2026-03-24-github-templates.md @@ -606,7 +606,7 @@ git commit -m "docs(github): add AI issue templates" ## Test plan - [ ] Not run (explain why) -- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest ...` +- [ ] `source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest ...` - [ ] `pylint src/candle/ --rcfile=.github/pylint.conf` ## Checklist @@ -643,7 +643,7 @@ git commit -m "docs(github): add AI issue templates" ### Commands run ```bash -source ~/miniconda3/etc/profile.d/conda.sh && conda run -n mindnlp python -m pytest ... +source ~/miniconda3/etc/profile.d/conda.sh && conda run -n candle python -m pytest ... pylint src/candle/ --rcfile=.github/pylint.conf ```