Skip to content

Commit e01c6a0

Browse files
authored
CUDA backend: create the temporary kernel library file safely (#24783)
### Summary The CUDA backend writes each method's compiled kernel library to a temporary file and loads it from there. The name came from the process id and a counter, and the file was opened with a plain `std::ofstream`. That is fragile in a temporary folder shared with other users, like `/tmp`: - If anything already exists at that name, the load can fail. - The backend can end up opening a file it did not create. - Other users can usually read the file. This change adds `write_temp_library()` to the code that all AOTI backends share (`backends/aoti/temp_library.h`, built as `aoti_temp_library` in CMake and `:temp_library` in Buck). Every AOTI backend loads its compiled library from a file, so any of them can use it. The CUDA backend's `init()` uses it now. Other backends, such as Metal, can switch to it in separate changes. - On Linux it calls `mkostemps()`. That picks a random, unused name (`executorch_aoti_<n>_XXXXXX.so`), creates the file exclusively so an existing path is never opened, and gives it owner-only permissions (`0600`). - On Windows it picks a random name of the same form and creates the file with `CreateFileW(CREATE_NEW)`. - `<n>` is a process-wide counter, so a path never repeats (the loader would reuse an already loaded library). Other effects: - `get_process_id()` is removed from the CUDA platform layer (it was only used for the old name). - A failed load no longer removes a file the backend did not create. - A missing temporary folder is now a load error instead of an exception. The file still goes in the `TMPDIR` folder. On Linux it is still removed when `init()` fails or the delegate is destroyed. On Windows a loaded DLL cannot be deleted, so the file stays, as before. Names no longer repeat, so these files now pile up in `%TEMP%` across runs. On Linux, a file left behind by a crash now stays too: before, a restarted process with the same process id (common in a container restart loop) overwrote it, but now every crash adds a new file to `TMPDIR`. The library is still loaded by path, so this relies on only the user and root being able to rename the file or any folder on its path, as with `/tmp`. ### Test plan New tests in `backends/cuda/tests/test_temp_library_file.py`: - Load a small CUDA program with `TMPDIR` set to a fresh folder. Check for exactly one new file with mode `0600`, the right output, and no file after the program is freed. - Load with `TMPDIR` set to a missing folder, and check that the load fails with `AccessFailed`. - Make the write fail partway with a file size limit. Check for `AccessFailed` and no leftover file. The first two fail on main. The third fails if the half-written file is not removed. On Linux x86_64 (A100): - All CUDA backend C++ runtime tests pass. - `temp_library.cpp` compiles for Windows (MinGW headers) and Linux x86_64 and aarch64. Not built with MSVC.
1 parent 2c40e57 commit e01c6a0

10 files changed

Lines changed: 364 additions & 44 deletions

File tree

‎backends/aoti/CMakeLists.txt‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,37 @@ endif()
2525
include(${EXECUTORCH_ROOT}/tools/cmake/Utils.cmake)
2626
find_package_torch()
2727

28+
# ==============================================================================
29+
# Writes a compiled AOTI library to a new private temporary file, for every AOTI
30+
# backend that loads that library by path
31+
# ==============================================================================
32+
add_library(aoti_temp_library STATIC temp_library.cpp)
33+
target_include_directories(
34+
aoti_temp_library
35+
PUBLIC $<BUILD_INTERFACE:${EXECUTORCH_ROOT}> $<INSTALL_INTERFACE:include>
36+
$<BUILD_INTERFACE:${EXECUTORCH_ROOT}/..>
37+
)
38+
target_compile_options(
39+
aoti_temp_library
40+
PRIVATE
41+
$<$<AND:$<COMPILE_LANGUAGE:CXX>,$<CXX_COMPILER_ID:MSVC>>:/EHsc>
42+
$<$<AND:$<COMPILE_LANGUAGE:CXX>,$<NOT:$<CXX_COMPILER_ID:MSVC>>>:-fexceptions
43+
-fPIC>
44+
)
45+
# Same choice as cuda_platform: a shared library that links this one must reach
46+
# the runtime that ships beside it, or the process gets two platform layers.
47+
if(EXECUTORCH_BUILD_SHARED)
48+
target_link_libraries(aoti_temp_library PRIVATE executorch_shared)
49+
else()
50+
target_link_libraries(aoti_temp_library PRIVATE executorch_core)
51+
endif()
52+
53+
install(
54+
TARGETS aoti_temp_library
55+
EXPORT ExecuTorchTargets
56+
DESTINATION ${CMAKE_INSTALL_LIBDIR}
57+
)
58+
2859
# ==============================================================================
2960
# AOTI common shims using ETensor (for Metal backend)
3061
# TODO(gasoonjia): Remove this after metal migration

‎backends/aoti/targets.bzl‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,22 @@ def define_common_targets():
2929
],
3030
)
3131

32+
# Writes a compiled AOTI library to a new private temporary file, for every
33+
# AOTI backend that loads that library by path.
34+
runtime.cxx_library(
35+
name = "temp_library",
36+
srcs = [
37+
"temp_library.cpp",
38+
],
39+
exported_headers = [
40+
"temp_library.h",
41+
],
42+
visibility = ["PUBLIC"],
43+
exported_deps = [
44+
"//executorch/runtime/core:core",
45+
],
46+
)
47+
3248
# AOTI model container functionality
3349
runtime.cxx_library(
3450
name = "delegate_handle",

‎backends/aoti/temp_library.cpp‎

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
/*
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
* All rights reserved.
4+
*
5+
* This source code is licensed under the BSD-style license found in the
6+
* LICENSE file in the root directory of this source tree.
7+
*/
8+
9+
#include <executorch/backends/aoti/temp_library.h>
10+
#include <atomic>
11+
#include <string>
12+
13+
#ifdef _WIN32
14+
#include <windows.h>
15+
#include <cstdio>
16+
#include <random>
17+
#else // Posix
18+
#include <fcntl.h>
19+
#include <unistd.h>
20+
#include <cerrno>
21+
#include <cstdlib>
22+
#include <cstring>
23+
#endif
24+
25+
namespace executorch {
26+
namespace backends {
27+
namespace aoti {
28+
29+
executorch::runtime::Result<std::filesystem::path> write_temp_library(
30+
const void* data,
31+
size_t size) {
32+
std::error_code temp_dir_error;
33+
const std::filesystem::path temp_dir =
34+
std::filesystem::temp_directory_path(temp_dir_error);
35+
if (temp_dir_error) {
36+
ET_LOG(
37+
Error,
38+
"Failed to find the temporary directory: %s",
39+
temp_dir_error.message().c_str());
40+
return executorch::runtime::Error::AccessFailed;
41+
}
42+
// The loader hands back an already loaded library when given the same path
43+
// again, and libraries are never unloaded, so a path must not repeat within
44+
// a process.
45+
static std::atomic<uint64_t> counter{0};
46+
const std::string prefix = "executorch_aoti_" +
47+
std::to_string(counter.fetch_add(1, std::memory_order_relaxed)) + "_";
48+
const char* bytes = static_cast<const char*>(data);
49+
size_t remaining = size;
50+
#ifdef _WIN32
51+
std::random_device random;
52+
char suffix[24];
53+
snprintf(suffix, sizeof(suffix), "%08x%08x.so", random(), random());
54+
std::filesystem::path path = temp_dir / (prefix + suffix);
55+
const auto u8 = path.u8string();
56+
const std::string utf8(u8.begin(), u8.end());
57+
HANDLE file = CreateFileW(
58+
path.c_str(),
59+
GENERIC_WRITE,
60+
0,
61+
nullptr,
62+
CREATE_NEW,
63+
FILE_ATTRIBUTE_NORMAL,
64+
nullptr);
65+
if (file == INVALID_HANDLE_VALUE) {
66+
ET_UNUSED const DWORD err = GetLastError();
67+
ET_LOG(Error, "Failed to create %s with error %lu", utf8.c_str(), err);
68+
return executorch::runtime::Error::AccessFailed;
69+
}
70+
// WriteFile takes a 32-bit length, so write in chunks.
71+
constexpr DWORD kMaxChunk = 1u << 30;
72+
DWORD err = ERROR_SUCCESS;
73+
while (remaining > 0 && err == ERROR_SUCCESS) {
74+
const DWORD chunk =
75+
remaining > kMaxChunk ? kMaxChunk : static_cast<DWORD>(remaining);
76+
DWORD written = 0;
77+
if (!WriteFile(file, bytes, chunk, &written, nullptr)) {
78+
err = GetLastError();
79+
} else if (written == 0) {
80+
err = ERROR_WRITE_FAULT;
81+
} else {
82+
bytes += written;
83+
remaining -= written;
84+
}
85+
}
86+
if (!CloseHandle(file) && err == ERROR_SUCCESS) {
87+
err = GetLastError();
88+
}
89+
if (err != ERROR_SUCCESS) {
90+
DeleteFileW(path.c_str());
91+
ET_LOG(Error, "Failed to write %s with error %lu", utf8.c_str(), err);
92+
return executorch::runtime::Error::AccessFailed;
93+
}
94+
ET_LOG(Info, "Wrote %zu bytes to %s", size, utf8.c_str());
95+
return path;
96+
#else
97+
// mkostemps replaces the Xs with random characters and creates the file with
98+
// O_EXCL and mode 0600, retrying if the name is taken.
99+
std::string path = (temp_dir / (prefix + "XXXXXX.so")).string();
100+
const int fd = mkostemps(path.data(), /*suffixlen=*/3, O_CLOEXEC);
101+
if (fd == -1) {
102+
ET_UNUSED const int err = errno;
103+
ET_LOG(Error, "Failed to create %s: %s", path.c_str(), strerror(err));
104+
return executorch::runtime::Error::AccessFailed;
105+
}
106+
int err = 0;
107+
while (remaining > 0 && err == 0) {
108+
const ssize_t written = write(fd, bytes, remaining);
109+
if (written > 0) {
110+
bytes += written;
111+
remaining -= static_cast<size_t>(written);
112+
} else if (written == 0) {
113+
err = EIO;
114+
} else if (errno != EINTR) {
115+
err = errno;
116+
}
117+
}
118+
if (close(fd) != 0 && err == 0) {
119+
err = errno;
120+
}
121+
if (err != 0) {
122+
ET_LOG(Error, "Failed to write %s: %s", path.c_str(), strerror(err));
123+
unlink(path.c_str());
124+
return executorch::runtime::Error::AccessFailed;
125+
}
126+
ET_LOG(Info, "Wrote %zu bytes to %s", size, path.c_str());
127+
// The caller only takes over the file once this returns, so remove it here
128+
// if building the path throws.
129+
try {
130+
return std::filesystem::path(path);
131+
} catch (...) {
132+
unlink(path.c_str());
133+
throw;
134+
}
135+
#endif
136+
}
137+
138+
} // namespace aoti
139+
} // namespace backends
140+
} // namespace executorch

‎backends/aoti/temp_library.h‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
/*
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
* All rights reserved.
4+
*
5+
* This source code is licensed under the BSD-style license found in the
6+
* LICENSE file in the root directory of this source tree.
7+
*/
8+
9+
#pragma once
10+
11+
#include <executorch/runtime/core/result.h>
12+
#include <filesystem>
13+
14+
namespace executorch {
15+
namespace backends {
16+
namespace aoti {
17+
18+
/**
19+
* Writes a shared library to a new file in the system temporary directory.
20+
*
21+
* The file gets a name that is random and never repeats within the process,
22+
* and it is created exclusively, so an existing file is never opened or
23+
* overwritten. Outside Windows, only the current user can read or write it.
24+
* The directory is std::filesystem::temp_directory_path(), which honors TMPDIR
25+
* on POSIX. A caller that loads the file by path relies on only the user and
26+
* root being able to rename the file or any directory on its path, as with
27+
* /tmp. The caller removes the file when it is no longer needed.
28+
*
29+
* @param[in] data The library contents.
30+
* @param[in] size The size of the library contents, in bytes.
31+
* @returns The path of the new file, or Error::AccessFailed if it could not be
32+
* created or written.
33+
*/
34+
executorch::runtime::Result<std::filesystem::path> write_temp_library(
35+
const void* data,
36+
size_t size);
37+
38+
} // namespace aoti
39+
} // namespace backends
40+
} // namespace executorch

‎backends/cuda/CMakeLists.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,7 @@ target_link_libraries(
416416
PUBLIC cuda_platform extension_tensor ${_cuda_runtime_libraries}
417417
extension_cuda ${CMAKE_DL_LIBS}
418418
)
419+
target_link_libraries(aoti_cuda_backend PRIVATE aoti_temp_library)
419420
if(EXECUTORCH_BUILD_EXTENSION_LLM)
420421
target_link_libraries(aoti_cuda_backend PUBLIC extension_llm_cache)
421422
target_compile_definitions(

‎backends/cuda/runtime/cuda_backend.cpp‎

Lines changed: 7 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
#include <array>
2121
#include <atomic>
2222
#include <filesystem>
23-
#include <fstream>
2423
#include <mutex>
2524
#include <optional>
2625
#include <string>
@@ -42,6 +41,7 @@
4241

4342
// Include our shim layer headers
4443
#include <executorch/backends/aoti/aoti_delegate_handle.h>
44+
#include <executorch/backends/aoti/temp_library.h>
4545
#include <executorch/backends/aoti/utils.h>
4646
#include <executorch/backends/cuda/runtime/backend_options.h>
4747
#include <executorch/backends/cuda/runtime/cuda_delegate_handle.h>
@@ -512,18 +512,12 @@ class ET_EXPERIMENTAL CudaBackend final
512512
so_blob_key.c_str(),
513513
static_cast<uint32_t>(aoti_dso_buffer.error()));
514514

515-
// Generate a unique temporary file path. so_blob_key already selected the
516-
// blob above and is deliberately kept out of the filename: it can be an
517-
// untrusted, variable-length payload key, so embedding it risks path
518-
// traversal and cross-key collisions. Uniqueness comes from the pid
519-
// (across processes) and an atomic counter (within a process, since two
520-
// identical CUDA partitions would otherwise clobber each other's .so).
521-
static std::atomic<uint64_t> so_file_counter{0};
522-
filesystem::path temp_dir = filesystem::temp_directory_path();
523-
filesystem::path so_path = temp_dir /
524-
("executorch_cuda_" + to_string(get_process_id()) + "_" +
525-
to_string(so_file_counter.fetch_add(1, std::memory_order_relaxed)) +
526-
".so");
515+
Result<filesystem::path> so_path_res =
516+
write_temp_library(aoti_dso_buffer->data(), aoti_dso_buffer->size());
517+
if (!so_path_res.ok()) {
518+
return so_path_res.error();
519+
}
520+
const filesystem::path& so_path = so_path_res.get();
527521

528522
// A failed init returns no handle, so destroy() never runs for it. Remove
529523
// the file here unless init succeeds.
@@ -541,26 +535,6 @@ class ET_EXPERIMENTAL CudaBackend final
541535
remove_error.message().c_str());
542536
});
543537

544-
// Create a temporary file
545-
ofstream outfile(so_path, ios::binary);
546-
547-
// Write the ELF buffer to the temporary file
548-
ET_LOG(
549-
Info,
550-
"Writing %zu bytes to %s",
551-
aoti_dso_buffer->size(),
552-
so_path.c_str());
553-
554-
outfile.write(
555-
static_cast<const char*>(aoti_dso_buffer->data()),
556-
aoti_dso_buffer->size());
557-
558-
ET_CHECK_OR_RETURN_ERROR(
559-
outfile, AccessFailed, "Failed to write to file %s", so_path.c_str());
560-
561-
// Finish writing the file to disk
562-
outfile.close();
563-
564538
// Free the buffer immediately after writing to disk
565539
aoti_dso_buffer->Free();
566540
// Load the lib

‎backends/cuda/runtime/platform/platform.cpp‎

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
#include <string>
2121
#else // Posix
2222
#include <dlfcn.h>
23-
#include <unistd.h>
2423
#include <cstdlib>
2524
#endif
2625

@@ -166,14 +165,6 @@ executorch::runtime::Result<void*> get_function(
166165
// probably explicitly pack the bytes.
167166
}
168167

169-
int32_t get_process_id() {
170-
#ifdef _WIN32
171-
return GetCurrentProcessId();
172-
#else
173-
return getpid();
174-
#endif
175-
}
176-
177168
void* aligned_alloc(size_t alignment, size_t size) {
178169
#ifdef _WIN32
179170
return _aligned_malloc(size, alignment);

‎backends/cuda/runtime/platform/platform.h‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,6 @@ executorch::runtime::Result<void*> get_function(
2727
void* lib_handle,
2828
const std::string& fn_name);
2929

30-
int32_t get_process_id();
31-
3230
void* aligned_alloc(size_t alignment, size_t size);
3331

3432
void aligned_free(void* ptr);

‎backends/cuda/runtime/targets.bzl‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@ def define_common_targets(is_fbcode = False):
156156
":cuda_platform",
157157
":runtime_shims",
158158
"//executorch/backends/aoti:aoti_common_slim",
159+
"//executorch/backends/aoti:temp_library",
159160
"//executorch/backends/aoti/slim/factory:empty",
160161
"//executorch/backends/aoti/slim/factory:from_blob",
161162
"//executorch/backends/aoti/slim/factory:from_etensor",

0 commit comments

Comments
 (0)