Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions Dockerfile.standalone
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION.
# SPDX-License-Identifier: Apache-2.0
#
# Environment and runner for standalone C build (see build_standalone_c.sh in this dir).
# Build args: CUDA_VERSION (default 13.0), PYTHON_VERSION (default 3.11).
#
# End-user instructions: see ci/README-standalone-c.md
# Quick run: docker run -v $(pwd):/workspace -v $(pwd)/build:/build <image> [--build-tests]

ARG CUDA_VERSION=13.0
ARG PYTHON_VERSION=3.11
FROM rapidsai/ci-wheel:26.04-cuda${CUDA_VERSION}-rockylinux8-py${PYTHON_VERSION}

ARG NINJA_VERSION=v1.13.1

# System packages required by build_standalone_c.sh
RUN dnf install -y \
patch \
tar \
unzip \
wget \
&& dnf clean all

# Install ninja (architecture-aware)
RUN set -euo pipefail; \
if ! command -V ninja >/dev/null 2>&1; then \
case "$(uname -m)" in \
x86_64) \
wget --no-hsts -q -O /tmp/ninja-linux.zip "https://github.com/ninja-build/ninja/releases/download/${NINJA_VERSION}/ninja-linux.zip"; \
;; \
aarch64) \
wget --no-hsts -q -O /tmp/ninja-linux.zip "https://github.com/ninja-build/ninja/releases/download/${NINJA_VERSION}/ninja-linux-aarch64.zip"; \
;; \
*) \
echo "Unrecognized platform '$(uname -m)'" >&2; exit 1; \
;; \
esac; \
unzip -d /usr/bin /tmp/ninja-linux.zip; \
chmod +x /usr/bin/ninja; \
rm /tmp/ninja-linux.zip; \
fi

# CMake; sccache/date are configured at runtime by build_standalone_c.sh
RUN rapids-pip-retry install cmake \
&& pyenv rehash

# Output directory for the standalone archive. Bind-mount a host folder here
# (e.g. -v $(pwd)/build:/build) so libcuvs_c.tar.gz is written to the host.
ENV BUILD_OUTPUT_DIR=/build

# Run from repo root; the repo is expected to be bind-mounted at /workspace.
WORKDIR /workspace

# Run the build script, then copy the standalone tarball into the mounted output dir.
# Example: docker run -v $(pwd):/workspace -v $(pwd)/build:/build <image> [--build-tests]
ENTRYPOINT ["/bin/bash", "-c", "ci/standalone_c/build_standalone_c.sh \"$@\" && cp -v libcuvs_c.tar.gz \"$BUILD_OUTPUT_DIR/\"", "--"]
20 changes: 20 additions & 0 deletions docs/source/advanced_topics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Advanced Topics

- [Just-in-Time Compilation](#just-in-time-compilation)

## Just-in-Time Compilation

cuVS uses the Just-in-Time (JIT) [Link-Time Optimization (LTO)](https://developer.nvidia.com/blog/cuda-12-0-compiler-support-for-runtime-lto-using-nvjitlink-library/) compilation technology to compile certain kernels. When a JIT compilation is triggered, cuVS will compile the kernel for your architecture and automatically cache it in-memory and on-disk. The validity of the cache is as follows:

1. In-memory cache is valid for the lifetime of the process.
2. On-disk cache is valid until a CUDA driver upgrade is performed. The cache can be portably shared between machines in network or cloud storage and we strongly recommend that you store the cache in a persistent location. For more details on how to configure the on-disk cache, look at CUDA documentation on [JIT Compilation](https://docs.nvidia.com/cuda/cuda-programming-guide/05-appendices/environment-variables.html#jit-compilation). Specifically, the environment variables of interest are: <span class="title-ref">CUDA_CACHE_PATH</span> and <span class="title-ref">CUDA_CACHE_MAX_SIZE</span>.

Thus, the JIT compilation is a one-time cost and you can expect no loss in real performance after the first compilation. We recommend that you run a "warmup" to trigger the JIT compilation before the actual usage.

Currently, the following capabilities will trigger a JIT compilation: - IVF Flat search APIs: `cuvs::neighbors::ivf_flat::search() <cpp_api/neighbors_ivf_flat>`

<div class="toctree" maxdepth="2">

jit_lto_guide

</div>
22 changes: 0 additions & 22 deletions docs/source/advanced_topics.rst

This file was deleted.

79 changes: 79 additions & 0 deletions docs/source/api_basics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# cuVS API Basics

- [Memory management](#memory-management)
- [Resource management](#resource-management)

## Memory management

Centralized memory management allows flexible configuration of allocation strategies, such as sharing the same CUDA memory pool across library boundaries. cuVS uses the [RMM](https://github.com/rapidsai/rmm) library, which eases the burden of configuring different allocation strategies globally across GPU-accelerated libraries.

RMM currently has APIs for C++ and Python.

### C++

Here's an example of configuring RMM to use a pool allocator in C++ (derived from the RMM example [here](https://github.com/rapidsai/rmm?tab=readme-ov-file#example)):

``` c++
rmm::mr::cuda_memory_resource cuda_mr;
// Construct a resource that uses a coalescing best-fit pool allocator
// With the pool initially half of available device memory
auto initial_size = rmm::percent_of_free_device_memory(50);
rmm::mr::pool_memory_resource pool_mr{cuda_mr, initial_size};
rmm::mr::set_current_device_resource(pool_mr);
auto mr = rmm::mr::get_current_device_resource_ref();
```

### Python

And the corresponding code in Python (derived from the RMM example [here](https://github.com/rapidsai/rmm?tab=readme-ov-file#memoryresource-objects)):

``` python
import rmm
pool = rmm.mr.PoolMemoryResource(
rmm.mr.CudaMemoryResource(),
initial_pool_size=2**30,
maximum_pool_size=2**32)
rmm.mr.set_current_device_resource(pool)
```

## Resource management

cuVS uses an API from the [RAFT](https://github.com/rapidsai/raft) library of ML and data mining primitives to centralize and reuse expensive resources, such as memory management. The below code examples demonstrate how to create these resources for use throughout this guide.

See RAFT's [resource API documentation](https://docs.rapids.ai/api/raft/nightly/cpp_api/core_resources/) for more information.

### C

``` c
#include <cuda_runtime.h>
#include <cuvs/core/c_api.h>

cuvsResources_t res;
cuvsResourcesCreate(&res);

// ... do some processing ...

cuvsResourcesDestroy(res);
```

### C++

``` c++
#include <raft/core/device_resources.hpp>

raft::device_resources res;
```

### Python

``` python
import pylibraft

res = pylibraft.common.DeviceResources()
```

### Rust

``` rust
let res = cuvs::Resources::new()?;
```
90 changes: 0 additions & 90 deletions docs/source/api_basics.rst

This file was deleted.

10 changes: 10 additions & 0 deletions docs/source/api_docs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# API Reference

<div class="toctree" maxdepth="3">

c_api.rst cpp_api.rst python_api.rst rust_api/index.rst

</div>

- `genindex`
- `search`
13 changes: 0 additions & 13 deletions docs/source/api_docs.rst

This file was deleted.

101 changes: 101 additions & 0 deletions docs/source/api_interoperability.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Interoperability

## DLPack (C)

Approximate nearest neighbor (ANN) indexes provide an interface to build and search an index via a C API. [DLPack v0.8](https://github.com/dmlc/dlpack/blob/main/README.md), a tensor interface framework, is used as the standard to interact with our C API.

Representing a tensor with DLPack is simple, as it is a POD struct that stores information about the tensor at runtime. At the moment, <span class="title-ref">DLManagedTensor</span> from DLPack v0.8 is compatible with out C API however we will soon upgrade to <span class="title-ref">DLManagedTensorVersioned</span> from DLPack v1.0 as it will help us maintain ABI and API compatibility.

Here's an example on how to represent device memory using \`DLManagedTensor\`:

``` c
#include <dlpack/dlpack.h>

// Create data representation in host memory
float dataset[2][1] = {{0.2, 0.1}};
// copy data to device memory
float *dataset_dev;
cuvsRMMAlloc(&dataset_dev, sizeof(float) * 2 * 1);
cudaMemcpy(dataset_dev, dataset, sizeof(float) * 2 * 1, cudaMemcpyDefault);

// Use DLPack for representing the data as a tensor
DLManagedTensor dataset_tensor;
dataset_tensor.dl_tensor.data = dataset;
dataset_tensor.dl_tensor.device.device_type = kDLCUDA;
dataset_tensor.dl_tensor.ndim = 2;
dataset_tensor.dl_tensor.dtype.code = kDLFloat;
dataset_tensor.dl_tensor.dtype.bits = 32;
dataset_tensor.dl_tensor.dtype.lanes = 1;
int64_t dataset_shape[2] = {2, 1};
dataset_tensor.dl_tensor.shape = dataset_shape;
dataset_tensor.dl_tensor.strides = nullptr;

// free memory after use
cuvsRMMFree(dataset_dev);
```

Please refer to [cuVS C API documentation](c_api.rst) to learn more.

## Multi-dimensional span (C++)

cuVS is built on top of the GPU-accelerated machine learning and data mining primitives in the [RAFT](https://github.com/rapidsai/raft) library. Most of the C++ APIs in cuVS accept [mdspan](https://arxiv.org/abs/2010.06474) multi-dimensional array view for representing data in higher dimensions similar to the <span class="title-ref">ndarray</span> in the Numpy Python library. RAFT also contains the corresponding owning <span class="title-ref">mdarray</span> structure, which simplifies the allocation and management of multi-dimensional data in both host and device (GPU) memory.

The <span class="title-ref">mdarray</span> is an owning object that forms a convenience layer over RMM and can be constructed in RAFT using a number of different helper functions:

``` c++
#include <raft/core/device_mdarray.hpp>

int n_rows = 10;
int n_cols = 10;

auto scalar = raft::make_device_scalar<float>(handle, 1.0);
auto vector = raft::make_device_vector<float>(handle, n_cols);
auto matrix = raft::make_device_matrix<float>(handle, n_rows, n_cols);
```

The <span class="title-ref">mdspan</span> is a lightweight non-owning view that can wrap around any pointer, maintaining shape, layout, and indexing information for accessing elements.

We can construct <span class="title-ref">mdspan</span> instances directly from the above <span class="title-ref">mdarray</span> instances:

``` c++
// Scalar mdspan on device
auto scalar_view = scalar.view();

// Vector mdspan on device
auto vector_view = vector.view();

// Matrix mdspan on device
auto matrix_view = matrix.view();
```

Since the <span class="title-ref">mdspan</span> is just a lightweight wrapper, we can also construct it from the underlying data handles in the <span class="title-ref">mdarray</span> instances above. We use the extent to get information about the <span class="title-ref">mdarray</span> or <span class="title-ref">mdspan</span>'s shape.

``` c++
#include <raft/core/device_mdspan.hpp>

auto scalar_view = raft::make_device_scalar_view(scalar.data_handle());
auto vector_view = raft::make_device_vector_view(vector.data_handle(), vector.extent(0));
auto matrix_view = raft::make_device_matrix_view(matrix.data_handle(), matrix.extent(0), matrix.extent(1));
```

Of course, RAFT's <span class="title-ref">mdspan</span>/\`mdarray\` APIs aren't just limited to the <span class="title-ref">device</span>. You can also create <span class="title-ref">host</span> variants:

``` c++
#include <raft/core/host_mdarray.hpp>
#include <raft/core/host_mdspan.hpp>

int n_rows = 10;
int n_cols = 10;

auto scalar = raft::make_host_scalar<float>(handle, 1.0);
auto vector = raft::make_host_vector<float>(handle, n_cols);
auto matrix = raft::make_host_matrix<float>(handle, n_rows, n_cols);

auto scalar_view = raft::make_host_scalar_view(scalar.data_handle());
auto vector_view = raft::make_host_vector_view(vector.data_handle(), vector.extent(0));
auto matrix_view = raft::make_host_matrix_view(matrix.data_handle(), matrix.extent(0), matrix.extent(1));
```

Please refer to RAFT's [mdspan documentation](https://docs.rapids.ai/api/raft/stable/cpp_api/mdspan/) to learn more.

## CUDA array interface (Python)
Loading
Loading