Skip to content

Commit fcf50da

Browse files
committed
CUDA backend: add a PTE device-id remap tool under backends/cuda/scripts
A CUDA program records the devices from its target_device compile spec in its planned memory table, in tensor placement info, and in each delegate's target_device compile spec itself. All three must agree, so a program exported with an explicit non-zero target_device (cuda:3, say) does not load on a machine whose GPUs have other indices, and the loader has no way to move it. scripts/remap_pte_devices.py rewrites all three places in the file with one mapping, dense by default (3, 5, 7 become 0, 1, 2, keeping their order, with a warning unless the result lands entirely on cuda:0) or explicit (--mapping 3=0,5=0,7=0 collapses everything onto cuda:0). The output is an ordinary PTE, so the runtime needs no remapping support of its own. The companion .ptd is untouched: tensor data carries no device ids. A mapping that does not cover a recorded device is refused, so a file is never rewritten only in part. The tool starts a new backends/cuda/scripts directory for standalone CUDA backend tools, alongside the library code. Its CPU-only tests join pytest.ini testpaths and a Buck python_unittest, so serializer and schema changes run them without a GPU. Test Plan: - backends/cuda/tests/test_remap_pte_devices.py (also registered as a Buck python_unittest): 22 tests covering collection from all three places across multiple execution plans, dense and explicit mappings, coverage and validation errors, bare/uppercase/non-CUDA/ malformed target_device specs, shared compile specs, serialize round trips with named data, identity-remap byte equality, failed-write atomicity, the not-entirely-cuda:0 warning, an exported program (DeviceAwarePartitioner, cuda:3) remapped to equal a native cuda:0 export, and the CLI dry-run, input-error and write paths. All pass locally.
1 parent 679e49e commit fcf50da

8 files changed

Lines changed: 947 additions & 0 deletions

File tree

‎backends/cuda/BUCK‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,34 @@ fbcode_target(
164164
],
165165
)
166166

167+
fbcode_target(
168+
_kind = runtime.python_library,
169+
name = "scripts_atomic_output",
170+
srcs = [
171+
"scripts/__init__.py",
172+
"scripts/_atomic_output.py",
173+
],
174+
visibility = [
175+
"//executorch/backends/cuda/...",
176+
],
177+
)
178+
179+
fbcode_target(
180+
_kind = runtime.python_library,
181+
name = "remap_pte_devices",
182+
srcs = [
183+
"scripts/remap_pte_devices.py",
184+
],
185+
visibility = ["PUBLIC"],
186+
deps = [
187+
":scripts_atomic_output",
188+
"//executorch/exir/_serialize:lib",
189+
"//executorch/exir/backend:compile_spec_schema",
190+
"//executorch/exir/passes:propagate_device_pass",
191+
"//executorch/exir:schema",
192+
],
193+
)
194+
167195
fbcode_target(
168196
_kind = runtime.python_library,
169197
name = "cuda_partitioner",

‎backends/cuda/scripts/__init__.py‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Copyright (c) Meta Platforms, Inc. and affiliates.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# Copyright (c) Meta Platforms, Inc. and affiliates.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
6+
7+
"""Write a CUDA tool's output file in one atomic step.
8+
9+
The tools in this package rewrite model files that other steps then
10+
load, so a half-written output must never be visible, a failed write
11+
must not leave its temp file behind, and the result must stay
12+
readable by whoever could read the input (a bare NamedTemporaryFile
13+
would leave it owner-only at 0600).
14+
"""
15+
16+
import os
17+
import stat
18+
import tempfile
19+
from pathlib import Path
20+
from typing import BinaryIO, Callable, Optional
21+
22+
23+
def write_atomically(
24+
output: Path,
25+
write: Callable[[BinaryIO], None],
26+
mode_from: Path,
27+
) -> None:
28+
"""Write `output` via a temp file in its directory, then replace it.
29+
30+
`write` receives the open temp file. The output takes
31+
`mode_from`'s permission bits (required, so no caller can fall
32+
back to the temp file's owner-only mode by omission).
33+
"""
34+
output.parent.mkdir(parents=True, exist_ok=True)
35+
temporary_path: Optional[Path] = None
36+
try:
37+
with tempfile.NamedTemporaryFile(
38+
dir=output.parent,
39+
prefix=f".{output.name}.",
40+
delete=False,
41+
) as temporary:
42+
temporary_path = Path(temporary.name)
43+
write(temporary)
44+
temporary.flush()
45+
os.fsync(temporary.fileno())
46+
os.chmod(temporary_path, stat.S_IMODE(mode_from.stat().st_mode))
47+
os.replace(temporary_path, output)
48+
temporary_path = None
49+
finally:
50+
if temporary_path is not None:
51+
temporary_path.unlink(missing_ok=True)

0 commit comments

Comments
 (0)